A new chapter for Changelog podcasts

Adam and I have wanted our podcasts to have chapters for a looooong time.

Like, multiple years long.

They’re such a great way to navigate and explore an episode. You can skip directly to the subjects that interest you, skip past the ones that don’t, and even discover new parts of the show¹ that pique your interest.

I’m a big chapters user myself; have been ever since the feature was added to my podcast app of choice and some podcasts I listen to added them. But we never put in the work to chapter our podcasts because of various challenges in our way.

The challenges

We’ve self-hosted (and self-content-managed) our podcasts since 2016. To get that done, we built our own web app because, well, we’re software developers² and as the saying goes:

haters gonna hate, liars gonna lie, developers gonna develop

This app has served us very well, but one consequence of that decision is that we can’t sit around and wait for a (often VC-backed) hosting company to announce a feature like chapters and simply start using it. We have to roll it ourselves.

Two (and a half) factors made this challenging.

1️⃣ We shell out to (the extraordinary) FFmpeg to write ID3 tags on our mp3 files upon upload. FFmpeg, it just so happens, doesn’t support ID3v2’s chapter frame feature.

2️⃣ Our app is written in (the extraordinary) Elixir language. Elixir, it just so happens, didn’t have an ID3v2 package that we could use instead of FFmpeg to do that job.

And the half: I’m mostly an app developer. I write little libraries here and there, but I am not great at reading a spec (especially one that uses a binary file format) and turning that into a library. I know I could do it if I had to… but it’d take me a long time and I only get a few hours each week to work on this stuff.

Enter Sandman Wikman

I can’t recall what finally tipped the scale, but one day earlier this year it just felt like the right time to finally tackle chapters head on.

So, I figured that if we could hire our friend (and long-time listener) Lars Wikman to do the heavy lifting and create an ID3v2 package that supports chapters, I could find time to manage the rest.

Adam was down. Lars was down. The rest was all heavy metal code.

Lars created and released id3vx for us, an Elixir library for parsing and encoding ID3 tags. I put it to use in our codebase, and here we are.

Ok, it wasn’t all straight forward. But the details are only slightly interesting, and surely more interesting to us than they are to you. Lars joined us on our 7th Kaizen episode of Ship It! to discuss some of those details.

Listen to that if you’re super curious. What follows are a few bits that are worth a mention.

Our production workflow

Adding the technical means to attach chapters to an mp3 file is one thing, but being able to do that at scale (we publish 5ish episodes a week) and have them be good is a whole other matter.

We have a team of people producing these shows. From the host(s) that do the recording, to the content editors that cut out the bad parts, to the producers who put it all together and ship it out. We needed a way for the chapters to be added throughout the process by the people with the most context, but still editable by the producers and publishers. Here’s our flow:

1) Content editor leaves waypoints

Content editing is the process of taking the raw audio tracks and removing the bad bits, smoothing out the rough spots, and generally making it as listenable as possible. This work requires you to listen to the entire podcast, sometimes at a painstakingly slow pace. It’s tedious work, but it also means you have the most context to detect topic changes than anyone else in the process.

Here’s before and after shots of our recent Go Time episode on avoiding bloat:

2) Producer formalizes as markers

See all of those Topic Change markers in the panel on the left side of the project? Those are spots the editor thinks would make a good chapter break. They aren’t 100% usable, but they’re good enough for the producer to formalize during the final, “mastering” stage of production. Here’s what the chapter markers look like when the episode is finally ready to be mixed down and uploaded:

3) Publisher enters into our admin

After the mix down, we end up with a .wav file that has the markers and metadata in it and a .mp3 that needs ID3 tags. I decided early on that the source of truth for chapters would be in our admin, not the .wav file.

So, I built a little admin UI for manually adding/editing chapters (painful!) and a mechanism for “bootstrapping” an episode’s chapter by dropping the .wav file on the form (delightful!)

Now that we have the episode’s chapters in our database and a pure-Elixir library for writing them out as ID3 tags, we can output the chapters in all the right places!

Chapters, chapters, everywhere

Writing the chapter metadata to our mp3s was the hard part, but that’s not the only place they’re useful. You’ll find this feature manifesting in a few different places:

In your podcast app

This is the big one, of course. Any podcast app that parses chapter info from ID3 tags should have them! If yours does, but you don’t see any chapters. Please let us know in the comments!

In your RSS feed

Podcasting 2.0 is a very interesting movement that we discussed briefly on our latest RSS episode with Ben Ubois. They’ve created a new namespace and are adding a bunch of new tags. We’ve added a few of them to our feeds. Namely: funding, transcript, person, and now chapters.

On that episode linked above, we debated the pros/cons of putting chapters in the RSS feed instead of in the mp3 itself. I represented the 2.0 spec drafters’ decision pretty well on the podcast, but I’ve since found this blurb which explains their logic quite nicely:

Benefits with this approach are that chapters do not require altering audio files, and the chapters can be edited after publishing, since they are a separate file that can be requested on playback (or cached with download). JSON chapter information also allows chapters to be displayed by a wider range of playback tools, including web browsers (which typically have no access to ID3 tags), thus greatly simplifying chapter support; and images can be retrieved on playback, rather than bloating the filesize of the audio. The data held is compatible with normal ID3 tags, thus requiring no additional work for the publisher.

We’ll continue to put our chapters in both places for the foreseeable future, but hopefully more apps hop on the new hotness soon!

In your episode page

Why not also emit chapters on the episode details page and link up the timestamps to our onsite player? HTML for the win! (So much so that this was the first place the feature appeared. Too easy!)

The player itself hasn’t integrated chaptering yet, but since we already have the data as JSON, that should be pretty easy to wire up.

Enjoy!

Special thanks to Lars, Maqbool, Jason, and everyone who worked with us on this effort. Some have called the addition of chapters “The greatest technical improvement to Changelog podcasts since transcripts! And by some, I mean me. I’ve been calling it that. 😜

We’ll probably improve and refine from here. A few easy wins: adding more links, image support, onsite player support. But they’re pretty well baked for now, we hope you enjoy ’em!

If you crave more tasty ID3/Lars content, check out his blog posts and stay tuned for next week’s episode of The Changelog when we’ll be nerding out on the craziness of the spec! 💚

---

1. You know, like that one interesting bit near the end that you'd otherwise have missed because the hosts wasted twenty minutes of your life discussing HBO's Silicon Valley before getting themselves back on topic. 😉 ↩

2. We're also content creators. Having our own open source app to hack on, experiment with, and deploy to various infrastructures has been a brilliant source of content over the years.↩