How do you systematically measure, optimize, and improve the performance of LLM applications (like those powered by RAG or tool use)? Ragas is an open source effort that has been trying to answer this question comprehensively, and they are promoting a “Metrics Driven Development” approach. Shahul from Ragas joins us to discuss Ragas in this episode, and we dig into specific metrics, the difference between benchmarking models and evaluating LLM apps, generating synthetic test data and more.

Ryan Worl, Co-founder and CTO at WarpStream, joins us to talk about the world of Kafka and data streaming and how WarpStream redesigned the idea of Kafka to run in modern cloud environments directly on top of object storage. Last year they posted a blog titled, “Kafka is dead, long live Kafka” that hit the top of Hacker News to put WarpStream on the map. We get the backstory on Kafka and why it’s so widely used, who created it and for what purpose, and the behind the scenes on all things WarpStream.

Simon Wijckmans from c/side joins Jerod & Nick to discuss the Pollyfill attack in detail. What does it mean for web developers & client-side security going forward?

On this episode, Angelica is joined by Go community leaders from around the world: meetup organizers from Guadalajara, St. Louis, New York & Go Bridge Atlanta. Together, they explore the ins & outs of organizing meetups, the benefits of attending, the Go Developer Network (GDN) & the current state of the Go Meetup community.

Waymo cars make bad neighbors, Leonardo Creed pulls together wisdom from Linus Torvalds & the Art of Unix Programming to conclude what good programmers worry about, Max Schmitt makes the argument that toast notifications create a bad user experience, ChartDB is a web-based database diagramming editor, Simon Tatham makes a list of code review anti-patterns & scientists confirm that ‘flow state’ is very much a thing.

Adam & Jerod catch up with our ol’ friend, Suz Hinton! It’s been a couple years since Suz was a regular on JS Party. Since then, she moved back to Australia, earned a degree in cyber security & won a fidget spinner from the NSA… but that’s not all!

Silvia Botros joins Justin & Autumn for a phenomenal conversation about databases, her career path & the ins/outs of writing High Performance MySQL.

Eric Bailey joins Jerod to discuss everything Dungeons & Dragons taught him about writing alt text, building accessible websites, Primer, the problem with a11y overlays & more.

If you have questions at the intersection of Cybersecurity and AI, you need to know Donato at WithSecure! Donato has been threat modeling AI applications and seriously applying those models in his day-to-day work. He joins us in this episode to discuss his LLM application security canvas, prompt injections, alignment, and more.

Flavors of Ship It on The Changelog — if you’re not subscribed to Ship It yet, do so at shipit.show or by searching for “Ship it” wherever you listen to podcasts. Every week Justin Garrison and Autumn Nash explore everything that happens after git push — and today’s flavors include running infrastructure in space, managing millions of machines at Meta, and what it takes to control your 3D printer with OctoPrint.

This episode focuses on the art of delivering concise Lightning Talks, a popular format at conferences worldwide where speakers present in a short timeframe. Joined by some of this year’s GopherCon Lightning Talkers, we’ll discuss their experiences, challenges & tips for effective communication within a limited time.

Chris Stjernlöf got nerd-sniped and ended up writing down his practices of reliable software design, Ben Visness has had enough with the npm community’s propensity to pull in micro-libraries to suit every need, “Stay SaaSy” makes three metaphors for problem solving categories, Troy Hunt takes us inside the “3 billion people” National Public Data breach & Dasel is one data tool to rule them all.

You won’t believe the bizarre secrets Jordan Eldredge found investigating corrupt Winamp skins (#7 will shock you)! You also won’t believe how long we can wax nostalgic about the era of Napster, Aladdin & Pearl Jam.

Justin & Autumn are joined by Steven Wu from Scanner. Scanner built logging infrastructure focused on security teams and occasional querying. We dive deep into how architectural decisions affect your business.

Raphael Landaverde & Jake Shirley work on Minecraft full-time. How cool is that?! On this episode, they join Jerod to tell us all about the web tech that drives Minecraft’s scripting infrastructure, how they incrementally change a massive / always-moving target, the best / worst parts of the job & much more.

Andreas Kling and Chris Wanstrath have joined forces to form a non-profit called Ladybird Browser Initiative to manage the newly forked Ladybird browser. We discuss what it’s going to take to get to alpha, the why behind Ladybird, avoiding incentives other than those of the users, their plans for incremental adoption of Swift as the successor language over C++, and of course what they hope Ladybird can achieve as a truly independent open source browser that’s for the people.

You might have heard that “AI is only as good as the data.” What does that mean and what data are we talking about? Chris and Daniel dig into that topic in the episode exploring the categories of data that you might encounter working in AI (for training, testing, fine-tuning, benchmarks, etc.). They also discuss the latest developments in AI regulation with the EU’s AI Act coming into force.

Kris, Angelica & Johnny react to the recently announced Go team changes, discuss the finding that 80% of developers surveyed by Stack Overflow are unhappy & disagree about the concept of tech debt (but agree that something’s gotta give).

Jimmy Miller tells us about the best, worst codebase he’s ever seen, The Phylum Research Team follows up on the great npm garbage patch, Zach Leatherman logs his findings on sneaky serverless costs, David Cain wants you to go on quests instead of goals & Ashley Janssen gives us szeven rules for effective meeting culture.

Tim Banks joins Justin and Autumn — there’s nothing quite like being punched in the face by Zookeeper or being taken down by a “hot” shard.

Database aficionado, Ben Johnson, joins Jerod to answer the age ol’ question: which database should you use? Answering that isn’t always easy, which means it’s time to play the “It Depends” jingle & weigh (some of) the options.

We’re talking OpenAPI this week! Kris & Johnny are joined by Jamie Tanna, one of the maintainers of oapi-codegen, to discuss OpenAPI, API design philosophies, versioning, and open source maintenance and sustainability. In addition to the usual laughs and unpopular opinions, this week’s episode includes a Changelog++ section that you don’t want to miss.

Dennis E. Taylor joins the show to take us “Into the Bobiverse” and other books he’s written. Dennis shares the backstory on how he went from programmer to author/writer and creator of Audible’s Best Science Fiction Book of 2016, his process for iterating and developing the story as he writes, plans for a Bobiverse movie, and what’s next in book 5 coming out in September 2024.

There is an increasing desire for and effort towards GPU alternatives for AI workloads and an ability to run GenAI models on CPUs. Ben and Greg from Intel join us in this episode to help us understand Intel’s strategy as it related to AI along with related projects, hardware, and developer communities. We dig into Intel’s Gaudi processors, open source collaborations with Hugging Face, and AI on CPU/Xeon processors.

The latest Stack Overflow Developer Survey has some concerning results, Joeri Sebrechts helps you do plain vanilla web dev, MIT’s “missing semester” course looks pretty amazing, a dive into the fascinating history of CSV & a tool to get request analytics from the nginx access logs.

Adam Jacob goes solo with Adam for an epic pod into his journey to get to System Initiative. From SysAdmin at 8 years old, to discovering Linux and working for Mom-and-pop ISPs, to open source changing his life and starting Opscode and building Chef. Buckle up. This is a different flavor of “Friends” for you. Enjoy.

Deploying new applications can be tough. Deploying configuration management safely at scale with stores around the world is different. Martin Jackson joins us to discuss.

Node.js makes big TypeScript & SQLite moves, ECMAScript 2024 adds some niceties to the language (but not the ones you’re probably excited for) & we review the State of React 2023 results. Emergency?! Nick!

We discussed “🥦 Broccoli AI” a couple weeks ago, which is the kind of AI that is actually good/healthy for a real world business. Bengsoon Chuah, a data scientist working in the energy sector, joins us to discuss developing and deploying NLP pipelines in that environment. We talk about good/healthy ways of introducing AI in a company that uses on-prem infrastructure, has few data science professionals, and operates in high risk environments.

Joseph Jacks (JJ) is back! We discuss the latest in COSS funding, his thesis for investing in commercial open source companies, the various rug pulls happening out there in open source licensing, and Zuck/Meta’s generosity releasing Llama 3.1 as “open source.”