In this week’s episode Cameron Dutro, a software engineer at GitHub, Ship It listener and someone with an extraordinary attention to detail, joins us to talk about Kuby, a convention-over-configuration approach to deploying Rails apps.

The question that we will be trying to answer is what happened to Rails Active Deployment. The path to that promise land is paved with good intentions, but it’s complicated.

This week we have the pleasure of Rich Burroughs, Senior Developer Advocate at Loft Labs and host of the Kube Cuddle podcast.

We talk about multitenancy in Kubernetes and how to run Kubernetes in Kubernetes with vcluster. If you are using KiND, you will find this episode interesting, and maybe even helpful.

We also talk about the role that Kelsey Hightower played in Rich joining the CNCF ecosystem. The key take-away is that people make all the difference.

ADHD is something that Rich thinks about often. Gerhard was curious about the difference between ADHD and burnout, as well as this Twitter thread on re-reading sent emails.

In today’s episode, Gerhard is talking to Mauricio Salatino (@salaboy) about the Continuous Delivery for Kubernetes book that he is currently writing.

Mauricio is a Staff Engineer at VMware where he spends most of his time contributing to Knative, an open source platform for running serverless workloads on Kubernetes. Gerhard & Mauricio spent a few months in 2021 working on Knative Eventing, and they both appreciate shipping great software continuously. Mauricio helped ship Knative 1.0.

The from-monolith-to-k8s application used throughout this book has been a few years in the making. It doubles-up as a workshop-style guide for rearchitecting a Java monolith to a Cloud Native architecture running in Kubernetes.

We finally did it! All our static files are served from AWS S3. This is the most significant improvement to our app’s architecture in years, and now we have unlocked the next level: multi-cloud. We talk about that at length, and how it fits in our 2022 setup. The TL;DR is that changelog.com will fly, both literally and figuratively.

We also address Steve’s comment that he left on our previous Kaizen episode - thanks Steve!

Towards the end, we talk about Gerhard’s new beginnings at Dagger, where he gets to work with a world-class team and build the next-gen CI/CD. That’s right, Gerhard is now walking the Ship It talk all day, every day. If you want to watch him code live, you can do so every Thursday, in our weekly community session.

Kaizen!

This week we are talking to Robin Morero, the person behind fabled.se, a DevOps consultancy from Gothenburg, Sweden. Their motto is “move faster and prosper”, which Gerhard prefers to the initial “move fast and break things”.

Fabled works with startups primarily, and after 26 years, Robin has a few interesting insights to share. What do you think, are haunted codebases real? At what point do pull requests become harmful? What about k3s running on KVM as a simple starting point for production? If this reminds you of #7, and the follow-up YouTube stream with Lars, it’s no coincidence.

Gunnar Holwerda (Engineering Manager) and Tom Pansino (DevOps Team Lead) share with us a few stories about how the teams at opensesame.com manage AWS operational complexity. The first link in the episode show notes are the slides that Tom & Gunnar prepared for this conversation. Check them out as you hear us speak about the Inverse Conway Manoeuvre, and why you should always go for the bananas.

If you like this episode, and have a similar story to share, please reach out to us. We all love real-world stories that we can learn from, and perhaps contribute to.

MLOps is an increasingly popular topic that is no longer just a subset of DevOps. Go is a great choice for infrastructure. What role does Go play in MLOps?

Vincent Ambo –the person behind nixery.dev, tvl.fyi, and a former Google engineer– shares his take on monorepos, Nix, and fully declarative systems without any Flux, Argo or Kubernetes.

While the tooling is impressive, it’s the principles behind it that captivated Gerhard’s imagination. Vincent has a rather interesting take on the monorepository idea, including one change - one version - one deploy. There are a lot of interesting links in the show notes, including all the code that Vincent uses to manage infrastructure.

As a result of this conversation, Gerhard is running Nix on one of his Macs, and also started experimenting with his first NixOS production instance.

Gerhard loves simple ideas executed well, which is why he is excited to be speaking today with Ildar Iskhakov & Matvey Kukuy about their startup Amixr, a.k.a. Grafana OnCall.

Ildar & Matvey started with a simple idea and a simple stack - Django, Celery, RabbitMQ & MySQL - all running on Kubernetes. Because they kept their main thing their main thing, and kept improving it every day for a couple of years, now your on-call can be simple too.

This is another Big Tent philosophy story with a Black Swan moment towards the end.

In our first 2022 episode, Alexis Richardson, co-founder and CEO of Weaveworks, is talking to Gerhard about going fully remote, what a great team looks like, and GitOps. While you may have heard of GitOps, now is a good time to check out opengitops.dev.

The most interesting part of today’s conversation is the missing cloud native App Store. While Apple revolutionised the world with the App Store and the iPhone, we don’t yet have something similar for cloud native apps. You may be thinking “But what about OperatorHub?”, or all the Helm registries out there? The registry fragmentation, operator deprecations and lack of curation are not what people have in mind when they think App Store. But there is more to it, so let’s hear how Alexis thinks about this.

Merry Shipmas! This is our special Christmas episode which sums up two months of very early mornings and a few late nights. After many twists and turns, stuff which didn’t work out, as well as pleasant surprises, this is what we ended up with:

• 🎁 PR #395 - CI/CD Lego set with Guillaume de Rouville & Joel Longtine
• 🎁 PR #396 - Continuous CPU profiling with Frederic Branczyk
• 🎁 PR #399 - Auto-restoring Kubernetes clusters with Dan Mangum & Muvaffak Onuş

While we initially intended to have five Christmas presents in total, only three got delivered in time. We planned, worked hard and eventually shipped the best we could just in time for this special Christmas episode. Our hope is that the latest additions to our changelog.com GitHub repository will help you just as much as they will help our 2022 setup.

🎄Merry Shipmas everyone! 🎄

In 2014 Gerhard joined CloudCredo, a startup co-founded by Colin Humphreys, Paula Kennedy & Chris Hedley. They stuck together through two acquisitions: Pivotal & VMware. This year, Colin, Paula & Chris co-founded Syntasso, the Platform-as-a-Product startup.

Today they all get together to talk about about what it takes to build a platform team, why Team Topologies is a good conversation starter and why a curated blend of off-the-shelf, composed, and self-created services are required in any organisation operating at scale.

Your hunch is right, all of them used to share the same Pivotal London office with Tammer Saleh, our guest from episode 31. Chris used to win all table tennis matches without even breaking a sweat, and today Gerhard gets his comeback. Touché!

Tammer Saleh, founder of SuperOrbital and former VP of Engineering at Pivotal, is joining Gerhard to talk about table tennis, remote work, and challenges that teams have with K8s.

Some years ago, both Tammer & Gerhard used to work in the same London office on CloudFoundry, and nowadays they are both into Kubernetes. Tammer and the SuperOrbital team are deeply experienced in this topic, and they help teams at companies like Bloomberg, Shopify, and federal U.S. agencies tackle hard Kubernetes and DevOps problems through engineering and training.

Why do companies need Kubernetes in the first place? Which are the right reasons for choosing it? Is Kubernetes a platform? Gerhard’s favourite: we are doing Kubernetes wrong, but it works better than when we were doing it right, so what’s up with that? This last one was a lot of fun, and we left the entire minute of laughter in at your request. Enjoy!

This is our third Kaizen episode in which Adam, Jerod & Gerhard talk about GitOps the wrong way, ask questions with Honeycomb and realise that they must be holding the CDN wrong, and the effort that has been going into moving all changelog.com static files from regular volumes to an S3-like object store. If you like a good yak shake, listening to this one is a lot more fun than doing it.

Gerhard is most excited about the Ship It Christmas gifts that we have been preparing for you. While GitHub Codespaces is not going to be part of the upcoming Christmas special episode, today’s talk covers why investing in a Codespaces integration is worth it.
Changelog #459 and Backstage #20 are related to this topic.

Zac Smith, managing director Equinix Metal, is sharing how Equinix Metal runs the best hardware and networking in the industry, why pairing magical software with the right hardware is the future, and what Open19 means for sustainability in the data centre. Think modular components that slot in (including CPUs), liquid cooling that converts heat into energy, and a few other solutions that minimise the impact on the environment.

But first, Zac tells us about the transition from Packet to Equinix Metal, his reasons for doing what he does, as well as the things that he is really passionate about, such as the most efficient data centres in the world and building for the love of it.

This is a great follow-up to episode 18 because it goes deeper into the reasons that make Gerhard excited about the work that Equinix Metal is doing. This conversation with Zac puts it all into perspective.

By the way, did you know that Equinix stands for Equality in the Internet Exchange?

This week Gerhard is chatting with Romano Roth, Head of DevOps at Zühlke, a company founded by Gerhard Zühlke in 1968. Nowadays they help companies all over the world build, ship and run anything from factory robots, to AI assistants in complex regulatory environments, and even medical devices that perform autonomous robotic surgery.

When Romano is not leading a team of 30 software engineers that specialise in operations, infrastructure and cloud, he is one of the organisers of DevOps Days Zürich, and also the DevOps Meetup group, which is how Gerhard and Romano met in 2019.

Having started his career as a .Net developer back in 2002, Romano had his fair share of dev and ops challenges, and he always enjoys seeing real business value delivered continuously in an automated way. In recent years, Romano’s perspective broadened, and now he sees DevOps realities across many companies. If you are curious about what good DevOps looks like, and what are the real challenges, then Romano has some good insights for you.

In this episode, Gerhard is joined by Cyrille Le Clerc, Product Manager Lead on Observability at Elastic, and Oleg Nenashev, Principal Engineer at CloudBees.

It all started with Oleg’s tweet back in July, in which he was promoting Akihiro Kiuchi’s work on Jenkins monitoring with OpenTelemetry. This was done in the context of Google’s Summer of Code - a link to Akihiro’s demo is in the show notes.

As you may remember from episode 20, instrumenting our changelog.com pipeline is on Gerhard’s mind, and this conversation helped him clarify a few things. If you are thinking of instrumenting your CI/CD pipeline with OpenTelemetry, this episode is for you.

In the second set of interviews from KubeCon North America 2021, Gerhard and Liz Rice talk about eBPF superpowers - Cilium + Hubble - and what’s it like to work with Duffie Cooley.

Jared Watts shares the story behind Crossplane reaching incubating status, and Dan Mangum tells us what it was like to be at this KubeCon in person. Dan’s new COO role (read Click Ops Officer) comes up.

David Ansari from VMware speaks about his first KubeCon experience both as an attendee and as a speaker. The RabbitMQ Deep Dive talk that he gave will be a nice surprise if you watch it - link in the show notes.

Dan Lorenc brings his unique perspective on supply chain security, and tells us about the new company that he co-founded, Chainguard. How to secure container images gets covered, as well as one of the easter eggs that Scott Nichols put in chainguard.dev.

This is Gerhard’s first set of interviews from KubeCon North America 2021.

William Morgan shares with us some of the finer Linkerd details, such as the underlying security theme, why native Kubernetes objects are preferable to more CRDs, and the joy of meeting team members in person.

Frederic Branczyk speaks about Parca, a new continuous system profiling tool that uses eBPF to help you understand what is happening on your hosts.

Andrew Rynhard gives us a great Talos OS and Kubespan perspective, and shares some really good follow-up videos on these topics.

The last conversation is with David Flanagan - you know him as Rawkode - about new beginnings. It’s only been less than two months since we’ve had him in episode 18, and he kept really busy. Caleb, his 3 weeks old baby boy, was the youngest attendee at this conference, and some talks made him sleepy, so good job everyone.

This week Gerhard is talking with Arnaud Porterie, founder of EchoesHQ, a new utility that measures and communicates engineering activity.

They start by re-creating the 60 seconds Y Combinator pitch, and then shift focus to what it was like to get EchoesHQ off the ground. Next, they tackle something which is always on Gerhard’s mind: Why is it important to connect our daily engineering activity to intent?

Before EchoesHQ, Arnaud used to run the core team and the open source project at Docker, and combined with other engineering leadership roles that he held for over a decade, he kept encountering misalignment that was preventing organisations from making meaningful progress. Let’s hear why EchoesHQ might just be a great way of addressing this.

In today’s episode, Gerhard is talking to Sam Alba, Docker’s first employee, and Solomon Hykes, the Docker co-founder. Together with Andrea Luzzardi, they are the creators of Dagger, a universal deployment engine that trades YAML for CUE, and uses Buildkit as the runtime.

Why? Because we should stop rewriting the same application deployment logic in scripts, makefiles or continuous delivery configuration. That’s right, this is the YAML vaccine that we have all been waiting for.

Gerhard believes that one day, Dagger will become just as meaningful for application delivery, as Docker is today for application code.

Things go wrong all the time. We all make mistakes. And that is okay. What is not okay, is to think that it won’t happen, or that there will be someone else around when it does. In that moment, it doesn’t matter who wrote that module, package or microservice. But there is a better way to think about this, and there is an approach that makes people actually look forward to incidents.

It all starts with thinking of incidents as opportunities to learn, and then share those learnings with everyone, so that you can all improve. In this episode, Gerhard is joined by Stephen Whitworth and Chris Evans, incident.io co-founders, and former Staff Engineers at Monzo.

They get it, we get it, and now you can get it too.

This is our second Kaizen episode, where Adam, Jerod & Gerhard talk about changelog.com improvements since episode 10. OK, so Gerhard deleted the DNS API token. Not only did he take the time to understand how that happened, so that he could actually learn from his mistake, but now we have a system in place so that we can share learnings from incidents. By the way, these are publicly available in our #incidents Slack channel.

A great & unexpected thing that happened since we recorded this episode, is Jerod fixing 99% of all the errors that were happening in prod. The top error was the broken Twitter auth - sorry Matt - which was a result of us upgrading to OTP 24 a few months back. Episode 3 show notes include a YouTube stream which captures it all.

We wrap up this episode by each of us sharing the improvements that we would like to do until our next Kaizen. You heard it from Adam first: Ship It Driven Development

This week Emile Vauge, founder & CEO of Traefik, joins Gerhard to share a story that started as a solution to a 2000 microservices challenge, the real-world implications of shipping many times a day for years, and the difficulties of sustaining an inclusive and healthy open-source community while building a product company.

Working every day on keeping the open-source community in sync with the core team was an important lesson. The second learning was around big changes between major versions.

The journey from Travis CI to Circle CI, then to Semaphore CI and eventually GitHub Actions is an interesting one. The automation tools inspired by the Mymirca ant colony is a fascinating idea, executed well. There is more to discover in the episode.

In this episode, Gerhard talks to David and Marques from Equinix Metal about the importance of bare metal for steady workloads. Terraform, Kubernetes and Tinkerbell come up, as does Crossplane - this conversation is a partial follow-up to episode 15.

David Flanagan, a.k.a. Rawkode, needs no introduction. Some of you may remember Marques Johansson from The new changelog.com setup for 2019. Marques was behind the Linode Terraforming that we used at the time, and our infrastructure was simpler because of it!

This is not just a great conversation about bare metal and Kubernetes, there is also a Rawkode Live following up: Live Debugging Changelog’s Production Kubernetes 🙌🏻