Here we are, State of the 'Log 2022.

Holy moly...

Hard to believe it's been a year already, but here we are.

I feel you, man. I feel like this year, above all years - and this is something you say anyways; like, is it ever really a surprise that it goes fast? I feel like this year really has gone fast. The slowest year ever was 2020, of course, but 2022 seems to be the absolute roller coaster.

Yeah, things have been moving fast, and crazily... And here we are, in December, ready to close things down until the new year. This has become an annual tradition of ours. This is our fifth annual State of the 'Log episode, where we sit back, navel gaze, talk about some of our favorites, talk about some of the most popular episodes of the year, and something we started last year and it was a lot of fun, and we are doing again this year, is listener messages. So shout-out to all nine of you who recorded a little voice memo for us and sent them in. We'll be playing those throughout the episode. That's been a cool addition to the State of the 'Log, in my opinion.

Yeah, I agree. Something just about adding the listeners' voices into the mix just makes it the perfect way to end the year, and to do a true State of the 'Log. Because, Jerod, if we didn't have listeners, what would we be, man? Like, we would just be talking into the ether, bro. No one would hear us. \[laughter\] Obviously...

Yes. If a podcast ships in the middle of the forest and no one's there to hear it, it doesn't make a sound...

That's right. Jerod and I are master mp3 creators, basically, and we ship mp3's around the world, thanks to our friends at Fastly and our friends at Fly; two plugs for our favorites there.

Yeah.

That's a sponsor favorite, for me at least. Fastly and Fly - I really appreciate their support.

Absolutely. And I really appreciate ChatGPT naming us the dynamic duo...

Yes!

Because that strokes my ego very nicely.

Yes.

Of course, we were on Hacker News over the weekend, so if you wanted to get your ego unstroked, we got some solid criticisms there...

Oh, boy...

Always nice to see our friends and listeners at Hacker News... But let's not digress into that. We have a lot to do on this show. We have a bunch of favorites, we have a bunch of episodes, and a bunch of listener messages. So I have put out the call two weeks in a row on Changelog News to record your voice, send it in. Everybody who makes the show gets a free Changelog T-shirt, so sufficient motivation... And we got a bunch of cool messages. These first two, which I'll play somewhat back-to-back, didn't listen very closely, because we wanted to know what your favorite episode of the Changelog was, and they both submitted favorite episodes, but they weren't of our show. They were of our other shows, which of course we - we don't care. We think that's awesome.

True. Okay.

Nut the first one we're gonna listen to is Puneet. Here he comes.

Hey, this is Puneet from India. The Go Time episode where Mat goes to Berlin for GopherCon EU has to be one of my favorite ones. I loved his witty comments and banter.

Well, I don't think the terms "Mat Ryer" and "witty comments" are used in the same sentence very often, but... Puneet managed to splice them together. This was episode number 250 of Go Time, and it was all Mat's doing. So Mat went to GopherCon EU. He actually -- I think he was the MC of the event. We did a Gophers Say live on stage, which I helped out with... But Mat had this idea of he was going to just take a microphone - I think it was just his iPhone. I'm not sure what he was talking into... The audio quality was not our best ever, so I don't think it was a professional microphone... And he recorded his whole trip. So you know he would be on the airplane, on the train, walking through the airport...

Okay, just arrived at the airport... Yeah, the drive was okay. I wouldn't say the driver had good breath, but don't worry, it more than made up for it with his erratic driving. And I did suggest to, instead of bringing me to the actual airport, just drop me off at a nearby roundabout, but... We both agreed in the end that that was absolutely insane. But he wanted to avoid the charges, but I decided to cover them for him. So here we go... I'm going to now head into the airport... I'm on my way.

Getting on another train... He had a lot of stuff of him just getting there, and then some conversations while he was there, and then it was just over. So he just like took all of these audio clips and he handed them to me after he got back from GopherCon, and he's like, "Can you do something with this?" And I was like, "I'll do my best..." I appreciate the call-in, Puneet, because Mat asked me later - we put the show out, I did my best, I thought it was decent... But we had some new music, because Mat had written the theme song for GopherCon EU, so we used that music instead... \[sample 00:06:05.05\] \[06:05\] So it was kind of like the least Go Time episode Go Time ever. And Mat asked me later, he's like, "I listened to it, I liked it..." He's like, "Did anybody else like it?" And I was like, "I don't know. No one said a word." There was no feedback, there was nobody talking about it on Twitter, no one said it sucked, no one said it was great... We had zero feedback. So I'm like, "Sorry, man. I don't know. I don't know if they liked it or not." So I'm happy to hear somebody liked it.

Puneet closed the circle for us all. Thank you, Puneet.

Yes. And I have to -- I was giving Mat a hard time. There was definitely some witty banter. There was this common occurrence... He was going to Berlin, and they kept calling him a wanker, which is like a slur over there... I mean, it's not a nice thing to call people.

It's a slur here too isn't it?

Is it? I don't know.

It's not a good thing, I think...

No, it's not nice. I don't know if it's like an actual curse word, or anything. I didn't bleep it. I was like, "I don't know..." And Mat acted like they were saying "Danke", which is thank you.

\[laughs\]

So there's all these moments where he's like, "Oh, they're actually really nice here." And then there's this clip at the end where this woman goes, "Wanker!" and he goes, "Oh, you're welcome."

\[laughs\]

Well, that’s it from me, from Berlin, and from GopherCon EU. I had a great time. I’m off back to London now… \[wanker!\] You’re welcome!

It's classic.

It's hilarious. I've gotta go back and listen to that one.

So if for no other reason, go listen to that for that purpose. And it has chapters, so you can hop to the first one... Oh, you don't have to hop there; it's the first chapter. There's the first one, and then at the very end it's the last one. You could just check out those two moments. Of course, listen to the whole thing while you're there. But for me --

Oh, my gosh...

...that was worth the price of admission.

So thankful for chapters. So thankful.

Right?

Yes.

I wonder if we're the two people that like chapters the most...

I can't stop gushing about them, I'm sorry. It just helps me navigate our own shows better.

Yeah.

And maybe we're the anomaly, where we actually go back and listen to our own podcasts... Like, maybe other people don't go back and listen to their own stuff, to some degree... I'm not listening to me necessarily, or you necessarily, it's just more like the things we covered...

Quality control, yeah.

...because in the moment, it's hard to really grok everything, in the moment. Like a rewatch of a movie - you get more from the second watch if it's a really good movie that you really want to watch again.

Right.

I feel that way about our podcasts, almost every one of them.

Yeah. I love it because I'm always there the first time.

True.

A lot of people, when they experience a podcast, they just want to listen to the conversation. I get it; like, you're not really necessarily going to be hopping chapters, unless something gets way upstream, and then you want to get back on topic, and you're like, "I'll skip to the next chapter." So a lot of people don't care about chapters for that reason; they just want to listen to the conversation. But well, when you're there for the first time in the conversation, you actually want to go to specific points... And so I use our chapters feature all the time. And if you just want to hear Mat be called a wanker and then respond "You're welcome", I mean - chapters. Just hop right to that chapter.

So did he think they were for sure saying "danke"? Or was he just like-- it's a Mat thing?

No, no, no. It's just Mat's humor. Yeah.

Okay.

Yeah. He knew what they were saying. Totally.

Why in the world would they call him a wanker?

I don't know, maybe it's just something that happens in Berlin as you walk around. I don't know; I'm not sure why. Maybe he is one.

We'll have to ask Adam Wiggins next time we talk to him.

Yeah, totally. Alright, let's move on to our next one... So thanks, Puneet, for calling in. We'll get you a free Go Time T-shirt since you seem to like Go Time quite a bit. Next up, we have Eli.

Hey, Jerod and Adam. How are you guys doing? I wanted to share one of my favorite episodes from Changelog. I've used the Master feed to kind of browse through things... The news is certainly a good one. But I've gotta say that my highlight is the series - basically the Ship It series with Gerhard. And while Kaizen has been awesome, the one that comes to mind as like the tip of the top, so to speak, is "Fundamentals" with Kelsey Hightower. That episode was amazing, just in the sense of how much knowledge Kelsey threw out there, and I have to say, I have to re-listen and look at the transcripts, which is unusual for me... But it was really good. It was really good. And I really like how Gerhard focused in and came back over and over again to different things that Kelsey said, and tried really hard to crystallize in terms of a real example what Kelsey was saying... \[10:15\] But it was a super-dense episode, that was like really enjoyable... Like peanut butter -- or no, almond butter and dark chocolate mixed together, and slowly trying to eat that thing. It was just astounding. So well done, and certainly well done with all your other shows... But that was definitely a highlight. And thank you so much for that.

Well, it's a one-word podcast, which to me just screams amazing, right? Like, when you can ship a show with just one word as a title...

Fundamentals.

...that's the best, right? Fundamentals. Like, one, that's courageous to do. And two, probably something you can only do with Kelsey Hightower. And I would say three, he'd mentioned the show notes, and the transcript... Eli, by the way - it's my son's name. So thank you, Eli. And yeah, I think that's beautiful, because that means when we ship good shows like this, that can last for a while... Like, that's one that I think you can come back to a year from now, two years from now. It's almost like a book, in time, and it's a resource that can be there. And what I love most of what we do - and thank you, Eli, for highlighting this, is that we put so much work into the details. The show notes, the transcript, the chapters, as we just kind of gushed about, and we won't go on and on about... But to me, that gives people who really can appreciate the few episodes or the several episodes a year from every podcast we do, the ability to go back and dig deeper; to unravel the onion further and further and further. I think, to me, that's the detail that I really appreciate about what we deliver as podcasts, and just what Eli has said there about that show in particular; it just highlights all those things.

Yeah. So that was episode 44 of Ship It. Check it out, shipit.show/44. It also happened to be the most popular episode of the year, which is not a huge surprise, because Kelsey just spits hot fire everywhere he goes... And Gerhard really did a good job of drawing that out.

My role has always been “Document the manual process first. Always.”

Okay.

Because if you go and do everything in Puppet, now I’ve gotta read Puppet code to see what you’re doing. How can I suggest anything better? So if you write it down manually, and you say “First get a VM, install Changelog, then take this load balancer, put the certificate here, then get this credential, put it in this file, then connect to Postgres this version, with these extensions.” So now I can see the entire thing that you’re doing, and then the next thing I do is say “Okay, now that we understand all the things that are required to run this app, I wanna see the manual steps that you’re doing. All of them.” We build the app using this makefile; we create a binary. We take the binary and we put it where. You’re not storing the binaries into it? Oh no, we’re just making this assumption, that way we could just push the binary to the target environment. You need to fix that, that’s a bad assumption. You need to take the binary and preserve it, so that we can troubleshoot later in different environments, and we can use it to propagate. “Oh, okay Kelsey. Good idea.”

And we were only joking about liking Ship It and Go Time better than our podcast, because we love them as well. My favorite from 2022 from Ship It was "Operational simplicity is a gift to you" with Gary Bernhardt. That's episode 62. Gary's all like Kelsey, and he's a kind of a must-listen kind of guy. Everything he says is - if you don't agree with it, it's at least well reasoned, and interesting, and thought-provoking. So that was a surprise for me... I didn't expect to see Gary Bernhardt on an ops/infra Ship It episode, because I think of him so much in like the crafting software development world... But he did a really good job of highlighting really the way he tries to keep things simple, and how that's important with operations. I'll shout that one out as well. \[14:14\] Alright, listener number three... This is Jordi. Jordi hangs out on our Slack, so shout-out to Jordi; always happy to see you in our community.

As one of the hosts of the podcast Software Engineering Daily, I can only speak in awe, with a fair bit of envy about The Changelog. Its episodes are brilliantly researched, well-executed and thoroughly edited. Even swear words are edited out. I would be the editor's nightmare was I a suitable guest in the first place; which I am not. As an individual, the only reason I'm not a Changelog++ subscriber is because my creator budget is limited and already allocated. But this decision creeps up my moral judgment machine every now and then, sending regret waves that are increasingly difficult to hold against. In essence, The Changelog and all its subsidiaries is a bloody good podcast that anyone in the software industry should listen to.

We're gonna have to steal that copy, Jordi, because I like that - "regret shockwaves", is that right? We're in the midst of some thoughts around a new website, and I've gotta say that's some fine, good copy; fine, good things to say, of course, too.

Yeah, those are definitely some kind words... And Jordi was the one who was in our Slack, asking about our bleeping reasoning, which I explained there in Slack... And I posted it publicly as well, but I think we've already talked about our bleeping policy on the show.

No... But you can read it yourself if you like, Jerod.

I was gonna just summarize... But I probably --

Okay. Same thing.

Yeah. Let me summarize, because I don't have it pulled up... But the idea here with our bleeping is all about accessibility. It's all about reaching as many people, in as many circumstances as possible. And when it comes to the people that we want to reach, it's not merely adults, it's not merely mature; sometimes it's sensitive ears... And we want to be able to have people of all ages listen to our shows, without fear of coming across something that they don't want to come across. And then when it comes to circumstances - because most of our listeners are adults - there's a circumstance which is quite common when you listen to a lot of podcasts and you have kids, which is that you will listen with them in earshot. I do this all the time. And one common thing that happens, which is quite frustrating as a parent, is when you have a show which is generally speaking Safe For Work, safe for sensitive ears, and then all sudden here comes with like an F-bomb or something, like right in front of your kids. And you're like -- you feel a little bit betrayed, because you just didn't see that coming, and you would have paused it and listen to it later... And so we don't want to betray our listeners' trust, so that we keep the explicit tag off of our feed. And in order to do that truthfully, we have to then not have explicits. Now, things slip through here or here. You might find a wanker here, a wanker there... I'm not sure if that's explicit or not. Some things are judgment calls.

Yeah.

But definitely, with sensitive ears in earshot is something that we think about, and we want people to feel free and safe to just listen in their car with the kids around, and not worry about something inappropriate. So that's really the reasoning behind it, and... I don't know if there's any other reasons for it, Adam. We've done it forever.

Yeah. I mean, I think it's a version of what you said, but also just simply to be accessible; the accessibility of it. We don't want just anything possibly inappropriate... Even if you're just at a cafe, and you're just hanging out there. Like, for some reason, you're that weirdo who's playing it out loud; but that's cool too, if you want to share a show like that...

Yeah... \[laughs\]

You know what I mean? Just to sort of make this a safe place for everybody to hang their hat; you can listen to our show wherever you're at, by yourself, while you're running, or doing dishes, or in the car with your family, or even your mom. Like, what if you were in the car with your mom, she didn't mind you listening to your favorite podcast, and you're like "I'm listening..." Mom's like, "What are you listening to? They're talking about wankers here", or something like that. Hopefully that's not really a negative word, because we said it like 16 times, and we're gonna have to bleep ourselves if it's bad.

\[18:01\] We're gonna have to put a warning on this particular episode.

Yeah, geez. Anyways, as a side note here, I love whenever - and this is sort of a nod to you, Jerod. So this is more me talking to you, and also the audience hearing...

Okay.

...is I really appreciate the moments when I have something to say, and I don't have quite the words to say them yet, and I need to marinate a little bit more to figure out how best to say it, then I come back, and then you've said almost exactly what I would say, to some degree, potentially maybe better than I would have said it.

Oh, really?

Yeah. And so it's just like, one, I didn't have to do the work, and then two, what needed to be said was said. So in this case, that was an example of very similar thinking, and very similar reasoning. And I think I've told you this before, you're a pretty great diplomat, you do a great job of diffusing challenging situations with just - I don't know if this is a negative way to say it, but like lukewarm wording, where it's not like too hot, too cold, and shocking to anybody... It's just like perfect. So that's one thing I appreciate about you. And that's a good moment where that happened.

Yeah. Well, thank you. So all that being said, we totally understand it's dorky when you have somebody on the show who's like dropping truth bombs, and they throw an F in there, and you bleep it; you're like, "Ah, you ruined the moment." Worth it. Worth it for us to stay accessible.

One more thing on Jerod's copy, by the way, listeners... Something you don't get to hear - behind the scenes of every episode of the Changelog... Because I get to hear this at least 50 times a year, and over the course of my entire lifetime of all these podcasts we produce tons and tons more, is every time Jerod gets to tell a listener "You're not gonna offend us, but we may bleep you in post."

Right.

So he's letting the guest know, with a gracious heart, to say, "you're not gonna offend us necessarily with your words, but we want to make sure our listeners are protected from these certain things. And so you may not offend us in this moment, but know that you'll be bleeped in the post-production." So they're even warned and alerted beforehand, in a gracious way. So I think that's even a great moment of copy; it's like, "You're not gonna offend us, but we're going to do this in post, so be warned."

And the reason we say that is because we don't want to offend them by censoring them. And in fact, one point - I think we did talk about this with $STDOUT the rapper; that episode went out explicit, and I did not bleep anything, because that was his art, and I was featuring his art. And in that case, I made an exception, because I wanted his art to be represented as he created it. But the reason why we say that to folks is because of we don't want to surprise them, and all of a sudden have them be like -- because the listeners are sometimes surprised. They laugh at us, but at least the guests see it coming.

And you gave a warning for that one too before the show came out. You're like, "Hey, know that this one is different. There is some explicit in this episode, so if you're not in a safe place to listen, you might want to pause it and come back when you can be."

And I learned that from listening to other podcasts that I appreciate, who generally are Safe For Work, and they'll have an episode, and they'll warn it upfront, like "Hey, there's like two swears in this; just heads up." Like, I appreciate that as a listener, and so I did that on that episode as well. Well, let's get back to listener call-ins. Here's a guy that we know... Brett Cannon. Have you heard of him?

Brett? Brett Cannon? What...?

Yeah.

Okay...

Brett wrote to us. Brett talked to us -- he called in. I don't know what you call it. He sent us voicemail. Here it is.

Hey, Adam and Jerod, this is Brett Cannon. I wanted to say my favorite guest this year was actually a returning guest, and it was Richard Hpip in episode 454. Richard's dulcet tones and dedication to C were very interesting and fun to listen to... Although a shout out to episode 475 with Matt Aarons on the ZFS filesystem. For some reason, I just found that one really fascinating. And not to leave the Backstage out on any of this... Episode 23 with Parker Selbert and Oban Pro. It was really interesting to see how he's trying to turn his passion into something he could do full-time. And of course, I'd be remiss to not mention episode 18 for Tenet on Backstage. Congrats on five years.

Of course, he had to get the Tenet dropped. For those who didn't listen, Brett was on that episode, talking Tenet with heavy spoilers... That's episode 18 of our Backstage podcast, where we talk about pretty much whatever we like; it doesn't have to be necessarily on-topic. We don't do it very often, but we want to do it more...

Yeah.

\[21:58\] And one thing I thought was funny about this... So one thing you experience as a podcaster is that everybody listens at their own rate, at their own timing, at their own pace... And oftentimes, somebody will listen to an episode that you record a long time ago and then want to come and talk to you about it. And you're like, "Dude, I can't remember anything." Then you go read the transcript, or... Or they'll ask you a specific question, like "Why did you say this?" or "Why do you ask that?" and the context for us is just gone. We ship a show, we expect everybody to listen to it like that day.

That's right.

Like, "Hey, we want feedback." Brett is one of these guys, in my experience, who's always trailing behind. Like, he listens, he's a listener, I think he's even a Changelog++ member, but he's never up to date. The reason why I say all this is because that Richard Hipp episode was not this year. It was last year.

\[laughs\] Still a good show.

His favorite episode from this year was last year... But it was a good one. Of course, anytime you get Richard Hipp on the podcast, it's gonna be interesting.

We published it August 19th, 2021. And it was a phenomenal episode.

Yep.

We didn't just talk about SQLite. We talked about Althttpd, and also Fossil...

Fossil, yeah.

And then also last year, "Tenet with heavy spoilers." That was a fun show to kind of coordinate, because - I don't know, behind the scenes I had just been like marinating deeply on Tenet. There were just so many layers to it, and heavy spoilers... Paul from Heavy Spoilers on YouTube - he was just really doing a great job of like keeping me, I guess informed, so to speak. He was one of many, but I was like, "Let's get somebody who's 'an expert' at such and such film", in this case Tenet, and let's do a show about it."

Informed with the theories, right?

Right. Precisely.

With all of the different theories and interpretations of it.

And so Paul of course said yeah, and in good fashion, Bret Cannon came back on. Because when we do a Backstage about a film, I think at this point Brett would be offended if he wasn't part of that show. Right?

I think so.

And he'd need like a first right of refusal, at least.

If we do John Wick 4, just the two of us... I mean, we'll probably lose Brett as a friend.

Yeah. He'd be like, "Dude, come on. What's going on here?" So yeah, I'm glad he liked those, because those were good ones, for sure. And then Oban Pro with Parker Selbert, that was cool, too.

I think that was this year. So he got one out of three was in the calendar year 2022...

April. Yeah.

...but who's counting? Alright, here comes another familiar name, longtime listener of JS Party and one of the guys who is always listening, I think, like the day it drops... Because Rory O'Connor always has something to say about an episode right after we ship it, which as a publisher, you enjoy, because like I said, you move on in your head and then people wanna talk about it.

Instant feedback.

Yeah, you want that instant feedback. Here's Rory.

Hey, Changelog. Thanks for another year of excellent podcasts that keep me sharp and keep me informed, and keep me laughing, too. The two episodes that I enjoyed the most is your "The insane tech hiring market" and "The !insane tech hiring market" later in the year with Gergely Orosz. I've been a web developer for 25 years, basically doing the same job, if you can believe that... And I was wondering what it's like out there in the real world of the tech hiring market, and wondering what I'm missing out on, and wondering what I'm protected from... And these episodes really gave me a great insight into that, and helped me appreciate my own position... So thank you.

Pretty cool.

Yeah, very cool. There's a lot to say there, you know... Gergely coming back on twice... I think now it's become a staple. I think we'll have to do that. It has been annual, and I think we've kind of planned it somewhat annually, but he's been busy, so he almost didn't make it this year. But I'm like, "We cannot not have you on, considering the massive change." And then the subtle title change was just super-cool to pull off... But yeah, Rory, we appreciate listening to both of those episodes. And even for me too, just being informed about the ins and outs of what's happening in the hiring space... And I think Gergely does a great job of covering that. I believe I even gushed a little bit on that episode, because I just wanted to be, thankful to him in person; we were there talking to him in person. Because he does a great job of like covering the details. That's not a bone in me to do that.

Yeah.

\[26:05\] I appreciate the hard work, but that is a unique skill set that he has culminated from years in the trenches, at Uber and other places, and whatnot... And I just think that he's got that special knack for it; one, to enjoy the work, which I think sometimes that's very detailed work and challenging to many... But he really thrives at, one, producing it, and then really shedding the right kind of light in the right kind of areas to bring that knowledge to a lot of people. So he's done a great job doing that, and we're happy to have him back on another year, for sure.

Following up to Rory is another familiar name around our Slack... It's Jarvis Yang. He also --

Oh, yeah.

...appreciated Gergely's episode.

Hello, Changelog world. This is Jarvis from Minnesota. I wanted to say thanks for the great episode on "This !insane tech hiring market." I've been laid off as well recently, and this episode was very insightful, and just what I needed. I've usually been a part of the cost center side of things, but I'll definitely think twice when looking for new jobs. These layoffs also allow me to become more entrepreneurial, and I can see that there are lots of untapped, boring markets that could use some automation, or some software engineering love. Thanks again, Changelog, for an amazing year. You all rock.

Very cool.

Thank you, Jarvis.

Thank you for saying that. writing that intro, Jerod, for that show was challenging, because I knew obviously what we covered, but it was such a touchy subject... And I was just like, "You know what - we come to this conversation with great compassion and great understanding because of folks like Jarvis out there", that are just like, "I recently got laid off, and I'm listening to this episode." It's like, not only is that a relevant episode, but it hurts, right? Because you've got a circumstance that's not desirable. Who wants to be laid off? Nobody, right? And even as I was doing it, like, this is meant to be informative, but obviously, podcasts that we produce are meant to entertain to some degree, because otherwise you wouldn't come back. If you don't get that dopamine hit, what are you getting? And I was like, I can't just like do this intro and not talk about the, to be punny, insane amount of FUD out there. Like, there's just a lot of fear, uncertainty and doubt, so I'm like "Hopefully, this show gives you a lens into what's really going on." And hopefully, we covered that, because I didn't get a chance to like fully listen back... I was there obviously, as but I just wanted to express that great compassion and great understanding we came with that show with, because that's like a challenging show to produce when you know that there's people listening that you care about, and they care about you, and have been following you, and listening, and whatever, in the midst of that struggle and that challenge.

Yeah. Jarvis, I'm really sorry to hear that you got laid off, man. Jarvis has become somewhat of a friend via Wordle. So you may not know this, Adam, but Jarvis and myself and our editors, Jason and Brian, play Wordle every day.

Is that right?

There's a Wordle channel in our Slack...

What...?!

...and you just post your results. And I talked about it like months ago, when we first started doing it... I'm like, "Hey, is anybody still playing Wordle here? Here we are in this channel. Come play Wordle with us." And Jarvis was the only one who did. Like, there was nobody else that came to play with us... And pretty religiously, we all post our Wordle results daily in there, and see who gets the best one, or throw emoji on each other's responses... So yeah, Jarvis has been culminating in a relationship via Wordle for months now, and so I feel like I know the guy, even though Jarvis and I have never met... Although - Minnesota, not too far away.

Minnesota. Yeah.

Maybe we could do a meetup sometime.

That's so cool. I mean, this is like -- that's what I love about this show, and why I kind of come to it not unprepared, but like, in a surprised way... Because this is a hidden gem of Changelog Slack, in my opinion, and one of the reasons why we have this desire to have this free community for people to hang their hat. It's for things like this. It's used by internal folks, and one external, community member, but that to me is like -- maybe this show will kick off more in there, Jerod, because I can imagine there's more people playing Wordle...

If you're still playing Wordle... I know there's a lot of people that moved on. For me, it's a nice little thing to do in the morning while you're drinking your coffee, get your brain going...

Okay...

And yeah, if you want to have a Wordle community, come join the Wordle channel in our Slack and just post your results alongside us. It's a good time.

Will you be doing this through the Christmas holiday?

\[30:06\] I will definitely be playing, because I'm a completionist, and there's a streak, I've got a streak going... So I'll play -- I play daily. And will I be posting during there? Maybe I will; it just kind of depends... Like, do I want to hop in Slack or not? If somebody else is sharing, maybe I'll share mine. I'll definitely be playing through the holiday. Will I be sharing my results? Time will tell. It just depends on how much I want to disconnect from everything. But yeah, probably. Probably.

Now this - one more layer. Is there a syntax? Because I noticed, this is not an image, right? You just Wordle, and then the numbers you've done, and then Slack does it for you? How does this work?

So that's one of the cool things... I mean, gosh, Wordle's design was brilliant in many ways, and one of the things that's really cool about it, why it was so viral is he provides those as - and of course, it's The New York Times now, but the original creator did this... That's a copy paste. So when you say Share, and you copy your results, that's what it copies. And they're just emoji. It's like the green and white squares.

Oh, yes.

It's just -- it's not an image, it's text. And those are emoji to represent your results. So it shows like how many guesses it took, it shows where you got greens and yellows, and blanks... And so no one's typing that out. We just hit copy and paste.

That's so cool.

The cool thing about it is you can share your results without ruining the Wordle, because it doesn't actually share the answer, or anything. It's just shares like an image of how you did, but not the letters you guessed... Anyways, lots of brilliant stuff built into Wordle, the most brilliant of which is the daily cadence, where you can play once a day, and everybody plays the exact same puzzle... I mean, he didn't invent that, of course. The York Times Crossword Puzzle is famous for such things. Alright, let's move on... Here we have another return caller, Rusty Glue, who called in last year, back with another voicemail.

I just want to say big thanks to Annie Sexton from the episode 480. After listening to that episode, I have finally started using git rebase and git reset. Although I don't really follow her git flow strategy, I still find the interview useful to me, to get to the new level with git. And also, big thanks to you, Adam and Jerod, for doing all the shows. Last year I got a Practical AI T-shirt, and this year I'm hoping for a Changelog T-shirt.

Very cool. Hopefully, you get that T-shirt. And you will, because you're gonna get a coupon code (emailed, or something) to you, so... Too easy.

Yeah. We'll email them out a coupon code.

We ship globally.

Yeah. So that's episode 480, "Git your reset on" with Annie Sexton.

Yes.

That's another good episode title...

The titles are amazing. I mean, I have to appreciate our titles too, because we put so much nuance and detail into them as we produce these shows... And I laugh, because "Git your reset on" is cool. It's a cool thing.

That's a great one. People don't understand the pain of it when we just ship an average title, like how defeated we are when we just can't come up with something awesome, and we're like, "Yeah, good enough... Go ahead." Eventually, it has to ship, right?

It almost ruins the show in some ways, like "Ah, bad title..."

We can't delay shipping because the title is bad, but we would almost want to...

And it's the last thing we do. We may be marinating on a few titles along the way, but the very last thing we do is confirm the title is correct, and then we ship.

Right. Sometimes you'll have a working title that is pretty stinking good, like you came up with it as you thought about the show... But even then, it's not final. Like, we have to have a final decision on the title. And sometimes we'll ship a show... Let me give an example of something that we maybe haven't been quite as happy with lately... Well, the anthology episodes are just like listing out the topics, so that's always gonna be less than the coolest... "Linux mythbusting & retro gaming" - like, that's when you know we couldn't think of a title. Like, that just describes what the two topics were of the show. That one...

That's the classic challenge of a multi-focused podcast though, you know? I mean, when you have multiple topics...

It is tough.

\[33:55\] ...and they each are first-class citizens in terms of desired listen... Like, we want to listen to the Linux part of it, and then the retro gaming part is sort of Linux-ish, but it's not really, so it's its own topic.

Well, good for us, I guess... I can't find any other bad ones as I go down... So we've done pretty well this year. I guess "From WeWork to upskilling at Wilco" is okay; it's fine.

Yeah...

It's not -- like, "Build tiny multi-platform apps with Tauri and web tech" - descriptive, but not like it'll knock your socks off. A title should actually be like, "Oh, that's cool."

Yeah.

Anyways...

Just because I'm here, I really appreciate this one too, was "Building Reflect at sea." That's a cool title. It's simple, but it's also at sea. Like, Alex did this stuff at sea. It was cool.

That's right.

Continue, Jerod. We'll digress too far.

One quick comment on Rusty Glue before we move on to the next one... This for me, what he said, is so poignant and so why I listen to podcasts, and what I think a lot of the value provided is -- he even says, like the way Annie does her PRs, and her rebases... Like, he doesn't do it, and neither did I, if I remember that show. But he appreciated being exposed to somebody else's process, to somebody else's workflow, to their tooling, to the way they do things... Because it just expands your horizons as a developer, or as a builder, and allows you to have more informed decisions. Even if you're going to stick with your current workflow, right? Like, you spent an hour with us, hopefully it was somewhat entertaining; I can't remember if we were very good on that episode... But Annie was certainly good, and she had good things to say... But you don't have to like go do exactly what she does to get value. A lot of the stuff we come across, we don't necessarily use, or adopt... But just having that breadth of knowledge of what's going on, and what people are doing, and what they're thinking about - I think it's super-cool. So I'm happy that Rusty also appreciates that, even though he didn't necessarily adopt the style.

I think this brings up a question of “What’s the best way to collaborate at all?” Is it always that you’re on the same branch? Because in situations like this, it depends on if you’re touching the same files, then it can be a little bit tricky… But if you’re not, I think that there’s always a benefit in just having completely separate branches.

I’ve never done that before, so this is a new thought process to me. I don’t see why it wouldn’t be good, I just had never done it that way.

When you’re touching the same files it can be tricky, and I would probably reevaluate who’s doing what…

Your life…

\[laughs\] Try to reevaluate if this is better left to one person, or if it really needs to be collaborated on by two people… But if you’re touching relatively different files, then just creating different branches and merging them in separately - that also saves you a lot of the headache of “What happens if I force-push?” I know a lot of people have opinions about not force-pushing, and they’re welcome to those opinions.

My opinion is once you have more than one person pushing to the same branch, you should be done with your force-pushing.

Right, exactly.

Just because it’s causing them more work if you’re doing that. I have no problem with it if you’re on your own branch.

Yeah, same with me. I mean, I’ll force-push all day long to my own repos, because there ain’t anybody there to get upset. \[laughs\]

That’s right. Pushing nobody around.

I'm glad you said that too, because that was the challenge with that show, because it wasn't meant to be like "This is the way." It was more like Annie defined a way, shared it through the Render blog, it impacted a lot of people, and it made you think differently about the, I guess amount of effort you put into each individual commit message, you know? Because in her case, she mentioned she's got ADHD, and her thinking process - it actually distracts her from creating better, more useful code, because she's got to stop, and it cuts her flow... And she's learned this, and this is also an example of sharing what you learned too, despite the uncomfortability of expressing something about yourself that may be different or uncomfortable, or whatever. She shared this information, and I think it really impacted her and impacted others, too. You may not adopt the process, but it informs you on how to define your own, which I think is cool.

\[38:00\] Absolutely. Here comes our next listener voicemail.

Hey, everyone. I love The Changelog, and thanks for all the work you put into it. I listen to it all the time driving in to work. There were so many great episodes this year, from "Making the command line glamorous with Charm", to the future of building servers with Oxide and Bryan... That said, my favorite episode from this year has to be the lessons learned from auditing startups with Ken from PKC Security. There's so much there to apply within the startup that I'm helping to build... And it was awesome to learn from the patterns of successful and less successful startups that they worked with over the years. Thanks again.

That was listener Sean.

This episode almost made my top five.

Oh, really?

It was so close. It was so close.

I loved that episode. The one reason why I go back to it and I think about it and it doesn't make my top five is because it was so stinkin' long. Like, I was exhausted by the end of that episode... Which was really my doing, because I said we were going to go through all of these. Remember that? I said "We're gonna make it through all of them."

Yes...

And it turned out there was just a lot to say about a lot of them... I'm not sure how long the runtime is, but I remember just being like...

100 minutes.

Yeah, so that's pretty long. Longer than an hour and a half, an hour and 40... That's a long episode of the Changelog. I was exhausted, I can't talk that long and survive... And so that's what I think about -- even though the content I thought was pretty solid. And obviously, Sean, our listener who just called in, got value out of it, because he's working in a similar world, and learned a lot... So I'm happy to hear that.

Well, the sad part, Jerod, is this is pre-chapters. So maybe you'd like it better if you had some chapters to jump around to...

So inaccessible. We need accessibility.

When you're just like waiting in the minutiae, perceived minutiae; unless you actually listen to it and find value, it's minutiae until then... You're just waiting in this sea of podcast; you need some waypoints.

That'd be an easy one to waypoint, because we could go number by number through his posts. Wasn't it like 20 lessons, or 19, or something?

Something like that, yeah.

So there's a whole bunch of them, and we went like one through the next... So it'd be easy to chapter that, because you'd just chapter it based on the topic. Maybe somebody should go back and do that.

Open source those chapters...

Maybe you should go back and do... \[laughter\] Put that on your to-do list, Adam. Go back and chapter that for me.

Ah, I can't wait to do it, Jerod.

Alright, here's our last call-in... This is Tillman Jex.

For me, as somebody who's been starting to learn programming properly as of a little over a year ago, really the entire past year has been invaluable. I've learned so much of -- my learning has been accelerated so much by listening to the podcast, simply by hearing really great people talk about excellent places to learn, things to read, ways to think, technology to use, things to look out for... And if I would have to pick one thing that I think has had the biggest effect, it's definitely been the introduction to the initially mysterious word of Vim, and then to have been continuously tempted to look into it. Now every time I'm working in Neovim, which is my main editor now, I fondly think of all those introductions. So thank you so much, guys, for the work and the continual efforts. I'm still a student, but as soon as I start earning some money, I'm definitely becoming a Plus Plus member, 100%. Thanks so much, and all the best.

A lot to unpack there.

Yeah. I just want to say thanks for leaving that voicemail... Because that's the kind of impact that is super-meaningful for me, especially somebody just getting started. I sometimes wonder how valuable we are for people taking their first steps... Because so many of the conversations that we have are not beginner-oriented necessarily. It doesn't mean you can't come to them as a beginner... But having been in the industry for so long - I don't have beginner's eyes anymore, and so I wonder if it helps people who are at the beginning of their path... And so I just really appreciate hearing from somebody who's there, still student, still getting started, rockin' Neovim now, and benefited from our work. For me, that's just like the best.

\[42:05\] It makes it worth it. It really does. I mean, aside from this annual feedback loop for State of the 'Log, we don't get a lot of feedback. Now, that's not saying we necessarily are asking for it, but it is challenging when you feel alone on the road, in some cases; like, you don't see your impact until later. And that is challenging, because there are ups and downs throughout every person's life, in every year; there's always new challenges. This year I made a major move with my family; it's been a challenging year family-wise, just with that move, a lot of change in that. And so life is hard enough without major change. And whenever you don't have that feedback loop, it's easy to sort of let the "dark voice" take over to some degree, or have higher importance... And then you hear someone like this, like Tillman - thank you. And don't apologize for being a student and only being a student and not contributing in some way, shape or form. Like, we don't desire that. If anything, we want people who subscribe to Plus Plus, or buy T-shirts, or in some way, shape or form to support us because they truly want to, to get closer to that metal, to get those bonus clips, or whatever else we're going to do with Plus Plus. We don't want people to feel bad about not being able to do that, by any means. That's not why we created it. We created it because, time and time again, we would get asked by people like "Hey, how can we support you? How can we help make sure that you stick around and do more cool stuff?" And we thought, "Well, the only way we could do that is by giving you value." We don't want you to just give us -- in the majority of the way we survive as a business is through great sponsorships and great relationships with brands. And our relationship so far hasn't really been to invite folks like Tillman and others to support us directly. Changelog++ was a way for that, but it's not meant to make you feel bad, Tillman. So you'll have your time, and when that time comes around, join Plus Plus.

Well said. So that concludes our listener call-ins. Thank you to all y'all. We'll be hooking you up with sweet threads. For those who just want some sweet threads but don't want to write in, of course, we have merch.changelog.com. That Kaizen T is out there now...

It sure is.

And selling like hotcakes, so it probably won't be out there for very long... Of course, we can always just print more, but the print functions IRL are way slower than they are inside of our editors.

They are.

So should we move on now to our favs, or should we do the top episodes? Where do you want to head next? I think maybe our favs, because we can see how many intersect with our listeners faves.

Sure, yeah.

Alright. So mine are ordered by their published date, not by their place in my heart.

Okay. Well then read them then by the place in your heart.

\[laughs\]

Because mine are ordered by priority.

I didn't put them in my heart, I just put them on my list.

Okay. Then just read them down the list, and we'll go onesie-twosie.

I'll go publish date, you can go by the amount of love you have, and it'll be just like that.

Sure.

So one of my top five is from the very first month of the year... I tried to not have the recency bias, which we tend to have, which is that like we think about more recent episodes, and like them more because they're recent... So I went deep and started at the beginning, and the first one for me was, um, a solo show. Sorry, Adam; you weren't here for this one, but I still had fun anyways...

All good.

It was guest Paul Orlando from Unintended Consequences Blog, episode 474. It's called "Complex systems and second-order effects." I love this episode, I can go back and listen to it... These are just the kinds of things that I enjoy thinking about... And Paul is a guy who writes about it. And so he's already thought about these things way deeper than I have. I think about them at this shallow level, but to have somebody who thinks about the unintended consequences of complex systems, which are the systems that we work on, and things that happen despite our best efforts, or because we overlooked something, or as perhaps some sort of oddity in the world, what actually takes place. And it's so important for us, as software developers; we have so much leverage in our work, right? We can make small changes that have huge consequences, and... Well, those are just enjoyable conversations, to have the what-ifs, and the why's, and the how these things happen.

\[46:14\] the story behind the Cobra Effect is something that, as far as we know, never happened; but the story is during colonial India, so when the British were in India, some British administrator decided that they wanted to reduce or eliminate the number of cobras. Maybe this is in Delhi; I’m not sure where. So to try to achieve that goal, they put up a bounty, and they say “Okay, I’m gonna pay a bounty if you show up with a cobra skin.” And that’s gonna get rid of the cobras, right? Then the story, of course, is - well, people discovered “Oh, so I should just raise cobras, and turn them in for the bounty, and raise more cobras, and turn them in…” And then the British realized what’s happening, they eliminate the bounty, and everybody releases the cobras, and so you have a worse problem than you had before.

I thought that episode was really good. I think it would have been better with you there. I'm not sure why you weren't there; it was back in January.

I think I know why... I think this is recorded December 30. Thankfully, we record the recorded date, not just the published date.

Right.

And it's December 30th, 2021, and I think we got COVID over Christmas that year. So I think I may have still been recovering, potentially... I don't know.

Yeah.

I'm blaming COVID, as he's blaming COVID right?

Okay. Yeah. COVID is causing all our problems... Fair enough. So that would be one of my top five, was the Unintended Consequences, episode 474. Your turn.

Yeah, I enjoyed that show. Bummed to not be there, but these laws shows, I think they're reoccurring... I'd love to make them more frequent. I know that's a desire of yours as well. So I guess I'm suffering from the recency bias then, because this one's pretty recent... It's The Story of Heroku, with Adam Wiggins. I guess I was just a big fan, and have been - and you are too, Jerod, I'm sure, because we said that on the show, and we're not liars... We're big fans of Heroku, and by nature, big fans of Adam Wiggins, and his two co-founders that were a part of that with him... I'm trying to see if I can recall their names quickly. I know it was Orion and someone else, but they're not in my --

Orion Henry...

Yeah, bummer they're not my in show notes here. So whatever. Sorry about that. Orion Henry, and the other person's gapping me.

James...

But that was a great show, because -- James, yes.

James something.

James something.

Look it up now. We'll edit it; gotta get it right.

I'll come back in then... Let me see...

Or we'll leave this part in and just show people how much we care to get things right. \[laughs\]

Right.

Depends on who's editing...

Oh, yes. We'll keep it all in, because I had to look it up. It's a challenging last name - James Lindenbaum, and Orion Henry. So those are the two fellas he created Heroku with... And I just appreciate his relationship with them. I appreciate all the thought it seems they put into Heroku, and in many ways, the somewhat accident. Like, they obviously had a direction to go towards, but like Heroku was almost ahead of its time, and just hearing that story from Adam's perspective... And he also said he'd never done a podcast like this, and he said he would never come back to this topic, but he broke that rule for us, and gave us an exclusive, which I love... Not just the exclusive, but just the breaking that rule, because I think that there's a lot of people who care deeply about the story of Heroku. And I wanted to make sure we can get that out there. \[49:44\] Obviously, there's been some change with Heroku this year, with their free tier, and a lot of -- I don't know how many times for you, Jerod, but I've gone to like links that I've had of old, or from old blog posts, and I'm researching things, or whatever, and it links to a Heroku site that is now dead because of this change... Which I think is part of that unintended consequence, right? ...to go back to your favorite. That is an unintended consequence by Heroku. I'm sure there's a lot of resources that are out there that are not there anymore. And just that whole story arc of Heroku I think was something that I desired for many years to tell, and finally got to, so that's why it's -- recency bias, but also true bias.

True bias...

True bias, yeah.

Well, we have our first crossover episode, because that one also was in my top five favorite episodes of the year. Huge respect for Adam Wiggins. I think he's a super-smart guy, and the opportunity to pick his brain and ask him all my nagging questions was a blast.

Yeah.

Breaking from that - now, I have one that's a little bit selfish. I'll tell you why this is one my favorites. "Song Encoder: Forrest Brazeal." So I've done a couple of these Song Encoder episodes; I have two more in my mind. This is where we do a special feature of a specific person who is creating things at the intersection of music and code. Forrest Brazeal calls himself a code-bard -- or a cloud-bard, excuse me; not a code-bard. Because he's in the clouds. And he does just amazing, different kinds of music and all that, and so I featured in episode 477 on Forrest Brazeal. Here's why it's one of my favorites of the year - it's because there's very few Changelog episodes that I can unabashedly tell my mom to listen to, you know?

Yeah.

And, like, that's the one. If I'm going to like a normie, like "Here, listen to this. It's something that I created, that you can hear, and then be like, "Oh, that's what he does", even though it's not always what I do... There's just not very many -- like, the story of Heroku with Adam Wiggins is gonna be completely lost on most people. But a guy singing, and talking about the cloud, and the production value in all that is one that I can definitely just give to my mom and say, "Here's what your son does", and she can be proud of me. So that's why it made my list, is because it's just kind of a standout in that way, as it appeals to more people than what most of our shows appeal to.

It also goes a couple layers deeper than, I guess, just your expectation of a podcast like this... Because it showcases that - I wouldn't say true art, but like it's art beyond art; like, creating code and creating software and building companies around software tech and all that good stuff - that's a true art, for sure. But then to create art on top of that art, like singing like Forrest does - I think that's just like... That's meta, you know? Super-cool.

It is. And he's so talented. I mean, he's a classically-trained musician, and he has all these different styles that he can apply. And he's singing about software, and AWS, and Azure, and all of these things, and it's like... I love things that operate at this unique intersection, where it's like, there's probably nobody else that can do that. Right? Like, there's nobody else who has both of those skill sets and can bring them together... And so those people are pure gold.

Pure gold.

My day was going great; Just pushed a code update; But then the pagers started humming: Oops, did I just delete Half the production fleet? That sinking feeling’s coming, from deep within my plumbing Now my life is flashin’, Hope my boss will show compassion And I really, really need someone to say… Hey hey hey, it’s gonna be okay. You didn’t just set fire to your resume. This happens to the best, Try not to get too stressed, It’ll be an awesome story someday. I tweaked a small config, Turns out that it was big And now my app has been beheaded; When I do something wrong, I fear I don’t belong, How can the world forget it, we’re trending now on Reddit. Oh, mistakes will find you But you’ve got a team behind you. So fix the process - yes, but don’t dismay: Hey hey hey, it’s gonna be okay. We’ll do a full postmortem some other day. It was a swing and miss, But we will learn from this, And we’ll all be better engineers, like… Look, here’s what you need to know when something gets destroyed If there’s negligence or malice, then you shouldn’t be employed But if a human is assumin’ then the problem is the system; It’s gut-wrenching, butt-clenching, but you work with ’em. It can happen to a junior, or to a senior. Just because you caused a little pause doesn’t mean you’re incompetent, It means you’re doing work with real effect. It’s a scar of battle, baby. That’ll earn you some respect. And we all have been there Made a slip or two or ten there; So we try to do a blameless RCA… Hey hey hey, it’s gonna be okay. You will still be here when this blows away. We’ve all screwed something up, So welcome to the club. We would love to hear your story someday.

\[54:51\] What else you got?

Well, I'm going to concur with Brett Cannon, because... You know, I'm a fan of ZFS, I've been thinking a lot about my Linux story arc too, because I've dove deeper into Linux this year... I want to say, thanks to 45Drives, we have some more coming up in 2023 with them. I'm a big fan of 45drives.com and all the hardware they put out there; amazing, amazing. I mean, you can go build any one of their machines... That's what I love about their stuff, is that you can go build a 45Drive system on your own if you wanted to, but it's just best just to buy, because they built it. Anyways, that's what got me into ZFS, because it is the first time that I had a backplane that was suitable for an array of 15 drives... And I was like, "Okay, let's learn ZFS." And so I learned ZFS, and I got deeper into it... And then I kind of got bored, because ZFS is just so good, there's not much administration, really. I mean, you can scrub once a month or twice a month like I do, just to make sure that your data is truly good to go... But it's pretty easy to run. So I was like, "Let me talk to Matt." Matt Aarons is one of the co-creators of the original ZFS project. And not only am I gushing about ZFS, but also its history in Sun Microsystems. There's a lot of scrutiny around licensing, which I'm not really super-clear on, but as a technology, I love ZFS. I think it's super-awesome for managing RAID arrays etc. It's super-cool. So anyways, talking to Matt was awesome, and then digging into that with him... And one thing that I hope comes out of next year, 2023, is getting to work with Matt on expanding the RAID array we have a bit more, potentially doing a SEF cluster with a ZFS send and receive to have a backup, an onsite duplication backup of our ZFS array. And then some other fun stuff. So let's just tease that for a bit. So I love Matt, I love the work he put into it, and then I'm looking forward to some cool stuff coming out of this year with digging further into ZFS.

Definitely a solid episode; tons of listens on that episode. It was very popular. And the main thing that I remember from it, that I was like, "Okay, this is cool", was near the end, when you guys were talking about how they're trying to -- I mean, ZFS is old, right? It's like 20 years old.

Yeah.

And like keep up with the Joneses. So like SSDs was a big shift from spinning disks, and now like cloud storage, specifically object storage, and where does ZFS fit into an object storage world, and this big, new features written in Rust... Some of the conversation around Rust was really interesting. Maybe the perception or the danger of a show about something like that, that's old, is like, it's all old. What's new and interesting? It's like, well, it's been around a long time, but that doesn't mean it's not also still new and interesting, because things change, and either a project slowly bitrots and dies, or it also changes to keep up. And so Matt's been working with the team to do new things, as new things arrive. So that was interesting for me. Alright back to me... Let's go for another crossover here, because I'm now looking at both lists...

Come on now...

\[58:04\] I didn't have access to your list until you pasted it into the shared doc... But now I can see both of our lists, so I can see that we both have "Wisdom from 50+ years in software" with Brian Kernaghan, episode 484. And if I was absolutely forced to just pick one episode from this year, that's the one that I would have picked.

Hmm...

Guy's a living legend.

Yeah. Well, when you can get that kind of depth and wisdom, and I would say just history, in a single episode, and they're willing to talk to you and share all that as if they were seeing it for the first time... I'm sure he had to share that story a couple times, right?

Yeah.

He's writing a book on it... I bought his book on Unix systems... Yeah, I think Unix Linux is super-cool, and it's just taken me a long time to really fully grasp it to just the depths of it, you know?

Yeah.

...to the distros. It's easy to be overwhelmed by the world of Linux, because it's a lot of moving targets... But really, when you dump it down, or when you simplify it, so to speak, it's pretty simple... But you've got to have that wisdom - 50 plus years, where it came from, Bell Labs etc. I mean, that's super-cool history to go into.

Yeah. And so Brian - a couple of things about Brian. First of all, he's a very accessible guy. Usually, when you get up to that age - of course, he's still teaching. But when you get to that age, there's a certain point where you're just like not interested anymore. Somebody who we've tried to get on Go Time over and over in different ways is Ken Thompson, of course, and to no avail. Now, maybe he'll do a podcast, but he's not going to do it with Go Time anytime soon, unfortunately... Brian does podcasts. Like, he talks to people. He's still mixing it up with people in the industry, which is so cool. Now, having said that, he had been on a few other shows prior to ours. This was not a Changelog exclusive. And so what I like about our particular episode... I listened to a couple of his other episodes; CoRecursive he was on, and he was on Lex Friedman as well. And so I went out and listened, just because we don't want to create the exact same show that they create; like, we don't want to ask kind of tired questions, and stuff... So you want to kind of have your own spin, or your own angle on it. And I will say, this could have been like the day of the week, it could have been he just had a great meal... Who knows? Was it us? Was it him? He was genuinely enjoying himself talking to us. Like, he was lit up, eyes bright, he was laughing, he would have kept going, we talked to him for a long time... That was cool. Because that's kind of your fear with a guy who's talked to everybody, and had all the same questions before, and told the history of Unix, and he wrote it all in a book... And it's like, "Are you gonna ask me questions out of my book? Like, go read this." You don't want to bore somebody, you don't want them to be feeling like "Here I go again..." And that was my fear going into that show, was like, "Gosh, I really don't want him to be bored, or disappointed, or whatever." And so what I really liked about that show - and you can hear it in his voice throughout. I've listened to it back. You could really see it in the video clips. He was leaning in, he was laughing, he was smiling... I think he really enjoyed himself. And that for me made for a really, really good show.

I concur. Yeah, I think I agree with that. And then I think when you're with hosts who truly enjoy that conversation and the ability to sort of dig deep into certain topics like this in particular, that it's received well, and they get sort of hyped and vibed about it. Let me bring it. I do think we bring it. I'm not patting ourselves on the back too much, but I think we do bring it, and people enjoy having conversations with us, because --

I will say that we definitely try to.

Yeah.

I'll give us that, we try to bring it.

There are days...

Well, bring your next one. Bring your next favorite here. Let's keep rolling.

Okay, So for me, just Frank Krueger, "Practical ways to solve hard problems", that was awesome. Episode 486. I just really appreciate all of the history he has around creating software; it seems like he's hit all the hard problems, and he's found practical ways to solve them. I mean, it really was a good title for the show. I think he wrote a blog post very similarly, "Practical Guide to Solving Hard Problems." I think we just took it. We took away guide and said ways. We said ways.

Yeah, we munged it.

Yeah. That was just a good show. I mean, it was a good show in every single way, and I enjoyed Frank a lot. I kind of wish we can have him back on more often, honestly. Some sort of recurring show with him.

\[01:02:10.06\] We probably could. He has his own podcast. He's definitely a conversationalist, and that always makes it easier, when you have somebody who just naturally converses well...

Yeah.

Some of us are better at that than others, and Frank is a master of just conversation. So that one was definitely high on my list. I almost threw it in the top five. I paused and looked at it and kept going, but really, really fun show.

I brought it in for you.

And lots of laughs. I'm happy that you did.

Yeah. I mean, the Windows history there... It was just a lot of rich history in there that you just don't think about. And now working on iOS stuff, I just think like - it's the juxtaposition of where he began; and the security and the government stuff... I mean, he has been all over the map in terms of what his history was.

What's the biggest learning you've learned, or continue to learn, when it comes to reliability? And what are some of the cardinal sins, or cardinal rules?

You have to test the error path just as much as you test the functioning path. It’s always an edge case in the error handling where you mess things up. It’s so easy to write code that, you know, when it’s working correctly, it works correctly. That’s the easiest code to write. What happens if this line fails, what happens if that line fails, what happens if the machinery fails, what if the connection fails… Handling all those error conditions. And the best way to do that is to just assume anything can error at any time; and that sounds like a terrible way to program… But programming systems like Erlang have showed us the correct way to do that. So I fell in love with isolated processes, that were expected to fail, and you just handled those failure conditions. Every message pass, every function call can fail, and you’d better have a good, smart plan for how to handle that. So to answer your question in general, the answer is “Assume everything can fail, and make sure –” You know, just in my own code, I would just put random “Throw exception here.” Just to see what happens.

Hm… Like, while you’re working, just to see what happens… It’s like your own little Chaos Monkey, but inside your own local codebase. That’s a cool idea.

Yeah. Especially if you’re trying to solve a hard problem that you really don’t know how to work on… It’s much more fun to pop up an error dialogue; you’re like, “Oh, that took down the whole process. I wasn’t expecting that…"

Alright, last one for me - "Stable diffusion breaks the internet", with Simon Willison. This is one of those episodes that I was just jazzed afterwards; like, I just had so much fun... Simon's energy was contagious, his excitement level. The only bummer about that show was kind of his setup wasn't ideal...

Yeah...

He kept bumping his microphone, or there was something wrong with the audio, where it just didn't sound as good as it could have...

He had to have an earphone in, and it kind of rubbed his shirt I believe.

Oh, that's right, yeah. So we had a couple of different setups that might have worked, and we ended up with him with earbuds, air buds; like the wired ones.

Who knows what they're called...

Yeah, whatever they call that...

The wired Apple headphones...

The wired Apple headphones, with the microphone on it. And he was aware of this, but he couldn't help himself. And he has kind of -- he had like longer hair, so it was like rubbing against... And he managed it, for the most part. Those particular things, you have to hold it, otherwise it's gonna rub against your shirt, it's gonna rub against your chin, your beard if you have one, your long hair, if you have long hair... And we warned him, like "This the only problem", and he held it together for the most part. As it gets near the end, there's way more of those, where it's just like sound ruffles. And you're like "Doggone it!" This was like a perfect episode, if it weren't for that. So that sucks; but I can set that aside, because the information that he spat, the excitement level, the timing of the episode was really good. Just pure fun. And I just loved it. I would have him back on anytime.

\[01:05:50.21\] Simon's awesome. Met Simon years ago... And we didn't even talk about like what he's most known for; what he's most known for, at least back in the day, was his contribution to open source through the Django Web Framework, which he was one of the co-creators of the Django Web Framework, which is huge. We didn't even talk about that at all, really.

No. I don't think he's all that interested in that, compared to generative AI.

I think it was 2019, the summer of 2019, for ZEIT Day in San Francisco. I went out there solo; shot photos for ZEIT; was then Zeit, now Vercel. Covered their ZEIT Day, did a couple episodes from there... We have an episode with Simon from then, talking about Datasette. And if you know Simon, if you've seen him speak, or you've seen him on stage, it's no wonder why we had the audio ruffling, because he's very animated.

He's very animated.

When he's on stage, he's pointing, and moving... Like, you think he's doing gymnastics, or something like that. Like, are you giving a talk or are you doing gymnastics? Because you are on fire, man! But Simon is a lot of fun. And I've had a couple of conversation with Simon too, and he's really into SQLite. And I've encouraged him a couple of times -- I think we talked about it after the show, to talk to Richard Hipp, and I think I made an introduction, and... Anyways. He's just a deep thinker about this kind of tech. And I like his insights, because he thinks deeply about them, and he's also very committed to writing about them... So he's got this workflow of like learn, write, rinse, repeat etc. That's the kind of person you want to follow. And you said timing was right; I mean, timing was right, but that title is a banger.

\[laughs\]

"Stable Diffusion breaks the internet." That's a banger.

That is a great title. You know me, I do appreciate a pop culture reference here or there, so... I loved that.

Alright.

Last one.

Jack Dorsey.

Jack Dorsey.

This name may upset some folks, especially given current climates, right?

Right.

There's a lot of back and forth and controversy between him and Elon on Twitter, and just stuff, man... Like, that aside, which is not why I claim this is my fifth favorite in my list of favs... By the way, I have three must-listens as well. So Jerod, I have to cheat a little bit... Which gets me three extras on my list.

You always cheat.

Call it cheating. I just call it playing the system, okay?

Okay... \[laughs\]

So last year, Square Unboxed 2022 happened, and we've been working with Square for quite a while; they're a sponsor of ours. And behind the scenes, Shannon Skipper, whom you met years ago, as you know, at All Things Open... Not All Things Open.

OSCON.

It was OSCON. That's right. We met in 2017 way back there, and I got his card. I took a picture of it recently and I shared it with him... Like, "Did I get your card from like so many years ago?" It's just cool how things work out. Anyways... I even asked him like "You know what - we've gotta get jack on Founders Talk at some point. Let's make that happen." And obviously, Jack is an A-list type, if you want to A, B, C-list folks, which I don't necessarily do, but just to give... Jack is a major player in tech, from Twitter, to Square, to now Block, and cryptocurrency, and decentralized thinking... Like, he's a big thinker, and the moves he often makes are larger moves, with large waves. And so Jack Dorsey - sat down with him for the main stage at Square Unboxed, because my friends at Square helped make that happen. He's like, "Hey, you know what? We can make a fireside chat happen at Square Unboxed." And so we took that conversation knowing that that was meant to be for Square Unboxed, but also for Founders Talk... So this is like three layers later. Then it also appeared on the Changelog, which is where this list goes from, right? And got even more listens. I mean, it was great, so that's one of my favorites; just as a bucket list kind of thing. Like, it was a great conversation... I didn't get to ask Jack every single thing I wanted to, because it was kind of bridled, in the fact that it was meant for the main stage of Square Unboxed, so it had its guardrails, so to speak, of the direction of topic. If Jack is listening, or anybody else from Square Block, I want to get him back on Founders Talk, so we'll have a different conversation, a deeper conversation... Although the conversation did open up with his hacker heart, which I think was super-interesting. \[01:10:03.18\] Just because, you know, when you get to that level, you often get removed from the code, and get often removed from the things that matter. Maybe he shouldn't be writing production code, but at the same time, he still tinkers with Rust, and fun things around crypto wallets and whatnot, hardware wallets, and fun things like that. So that was one of my favorites, just because - Jack Dorsey.

Alright, so let's hear your must listens, which we all know is just another way of saying also your favorites...

I think I like these two. So you see the list, don't get mad... Okay, so the reason why I'm saying this is a must listen is because this first one is the "ANTHOLOGY — Advocating for and supporting open source." And it's less just about the content, but more about the ambiance of this episode. So listeners of this show, especially listeners of this particular episode, given how much you probably enjoy this podcast - hopefully you do - this is the kind of episode that's fun for Jerod and I to do. One, we get out there face to face with folks, and get to see people in the real. We did this at All Things Open 2022, just recently, but the setup we did there was a little bit different this year. We did the mic situation in a standing scenario, where we were sort of standing in a circle, where Jerod was to my right, the guest was to Jerod's right, and I was to the guest's right. So it was like a little triangle/circle sort of formation, but we were standing up, to keep the conversations short, hopefully. In some cases they went long, but I digress. But I think we got the sound right on this one. And so that's why I'm putting it in the must listen. So if you've listened to a show like this, when Jerod is in his home studio, and I'm in my office studio, or whatever you might call it, and we have a guest - it's a little bit different, because it's meant to be studio quality, sound good, that kind of thing. Whereas here, we got the mix of the hallway; it truly is the hallway track. And to me, I think this year we nailed it in terms of great sound, great hallway track, and great conversation. I think it was like all the things that come together to make that kind of show, an in-person IRL, at a conference hallway track show come together. That is an example -- that and it's counterpart, the other anthology. So I'd actually link them both up, but linking one up. So episode 515 is an example of how I want to sound when we go to conferences.

I do like the way you describe the difference between technical and non-technical skills, in a way that’s easy to understand. The technical skills are what we know, and the non-technical skills are who we are. We have tried and true methods for changing what we know. Right? Like, you put your head in a book and you read it, or you go get some experience… Changing who you are can be a more difficult matter. Do you have any advice on changing yourself, so that you improve your skill.

Yeah. And I think, unfortunately, over the last three-four years is where – there are courses coming up where they talk about these non-technical skills, and why they are critical… But there is not a whole lot of material over there. I would say my personality has changed, evolved over the last few years as I’ve started listening to these podcasts… So I would really encourage people to start reading about it. And sometimes you don’t realize how consciously or subconsciously it starts impacting you.

The second one, "A guided tour through ID3 esoterica" with Lars Wikman - you can say his name... I'm going to Americanize it. I'm from Texas. I can't be bothered with enunciations. I'm sorry, Lars.

Lars Wikman.

That was a good show. I mean, I thoroughly enjoyed it. I felt like -- I mean how entertaining and enjoyable can we make this show? Right? But thankfully, he wrote a lot about it, and he was just as encouraged by all the research he had done. And to me, if you care about this show in particular, State of the 'Log, then that's the show you should go back. It's a must listen for those reasons. It's a navel gaze, but it's a technological navel gaze.

Yes.

Because we just look at ID3s, and... Who else should care, besides people who create mp3's like we do? Almost nobody else. But if you enjoyed the Annie Sexton show, and you just enjoy the osmosis of the conversation that you get to hear, then you'll love that show. And then I think the last one for me just was the icing on the cake, "The legacy of CSS-Tricks" with Chris Coyier. That was awesome. Getting to go back and talk through the Digital Ocean acquisition, and creating CSS-Tricks, and his journey as a creator etc. It was super-cool. So that was it.

\[01:14:14.25\] Alright, I've got some must listens...

Oh, gosh...

If you can do it, I can do it.

Do it!

I'm gonna grab a must listen from our other podcasts... So one from each. I already mentioned Ship It; I think the show was Gary Bernhardt is a must listen, even if you don't listen to Ship It. From JS Party, episode 244. So we have -- one of our formats on JS Party is called Yep/Nope. Which is a nod to YepNope.js, an old feature detection library by one of our retired JS Party panelists... And it's a debate show. And we had a great one this year called "The spicy React debate show." And this was based on a blog post by Josh Collinsworth called "The self-fulfilling prophecy of React", in which he says "React isn't great at anything except being popular." See how spicy that is?

Very spicy.

And so we invited Josh on; we made the premise "Is react only great at being popular?" and we teamed him up with a few of our panelists, and had a good old-fashioned Yep/Nope debate. Hilarity ensues. A lot of interesting insight, and so that's a must listen for JS Party, episode 244.

Do you know why React is so high in satisfaction? Because it’s so complicated to use it makes you feel like you’re a total ninja when you’re using it…

I feel called out.

…when really all you’re doing is using ten lines to solve a two-line problem. It makes you jump through so many hoops that you feel like you’re on American Ninja Warrior, reaching the top of that giant thing, just for writing a little bit of code… When really, they made you do this; this was just solving a web problem.

On Go Time this year, just recently - maybe some recency bias by me... This one went out in November, an episode with Jon and Johnny hosting Akshay Shah, talking about gRPC and protocol buffers. Now, we've done shows over the years that touched on these technologies... I've largely ignored gRPC, because it just seems like it's not for me; it's for like big, important orgs, with lots of big, important things to do... And yet, listening to this one, I think Akshay has some of the best explainers of why protobuf is interesting, what it's useful for, when you'd want to use it, when you wouldn't, why gRPC is interesting... Similar things. It's like I finally grokked it listening to that episode. So if gRPC is on your radar, but you don't know much about it, or protobuf - of course, these are related technologies - definitely check out Go Time episode 256, with the lame name of "gRPC & protocol buffers." You can't always have an awesome name.

You can't win them all, Jerod. That's one of those ones where you're like, "Man, I missed the mark on that title."

Yep.

But you can't get clever sometimes. Sometimes you have to be on the point, right? You can't get clever every time.

Sometimes you just call it what it is.

That's right.

Yeah.

Well, "The spicy React debate show" - I enjoyed listening to that live, for the most part. I was listening live almost all the way through; I had to bail at some point. I think it was like mid part two. Nick had a hard time MC-ing that show. It was very, very challenging to MC, and sort of like moderate that, because it was just a challenge... It wasn't challenging to listen to, but it was challenging -- I could see, for him, like he struggled to maintain the peace and maintain the control, so to speak. But what do you expect when it's spicy...?

I think that was his first time moderating. I usually moderate, and Nick stepped in to moderate that one. It is not an easy task to moderate a debate, especially one that's like virtual, via the interwebs.

What is it that b0neskull says? What was the sound effect to say you were done?

"Wut?"

"Wut?" That's right.

b0neskull would just say "Wut?" Because we don't have an official sound for "The time is up." And so yes, what you often have is you say "Time's up", and then the people just keep on talking, and you're like... I mean, even the very first time that we did a debate show, I was the moderator, and I lost complete control of the panel. \[01:18:12.00\] At one point Feross was just reading quotes off of Hacker News, and I'm like, "What is going on right here?" So it gets crazy, and I think that's kind of part of the charm. Of course, the debate is kind of just upfront to talk about a thing and have some fun with it. We have three segments. The first two segments is the formal debate, and then on the third segment of the show, we actually talk about what they really think... Because you don't debate what you believe. You debate what we assign you to. So you have to represent the premise or be against the premise based on assignment, not based on your actual feelings... And so that's always just fun, to kind of debate, even if I disagree with it... But thankfully, the third segment's there for us to actually talk about what people really think... And that's usually a much more nuanced and interesting, but less funny and crazy conversation.

I appreciate the assignment from a listener's perspective, because I know it puts the person who's debating the opposite of how they feel out of their norm... And it almost shows their depth even more so for like what they know, and what they don't know, and what they know about their positives... Because it's almost -- it's hard, but also kind of easy in some ways to flip the script and say, "Okay, I know the reasons why this is not good. So just reverse it." Right?

There you go.

...into the debate, and take all the negatives you think and turn them into positives. But I really appreciate that assignment process. And then the third segment being sort of free-for-all, so to speak; say what you really feel. That's cool.

Right. We always have much more agreement and much more nuance in the third segment.

A lot of slaps, man; a lot of verbal slaps, too. It's like, a lot of mic drops, in some cases... Kevin, in that particular one, - you could tell that Kball kind of like does a lot of prep, or it seems like he does a lot of prep. Not that it's scripted, but you could tell he's prepared words... Because he doesn't want to like come to heavy-handed and offend, but he does wanna get his point across. And when he's right, he knows he feels like he's right, you know what I mean? That's how he is.

Yeah. When you show up for a Yep/Nope, you should have arguments. It doesn't have to be written out, or like super-deep, but like, you're gonna get two minutes on the clock to talk, on a podcast. And you better have something to say. And Kball always has arguments ready to go.

Yeah, he came prepared.

A lot of times you can just react to what other people have said as well. Pun not intended. Okay, let's move on. So those are my must listens. Let's get now to not just our favorites, but let's talk about popularity. So, a few numbers... First of all, we've published 274 episodes across our network this year. That's a pretty good number. That doesn't include this episode, so it'll definitely be at 275. If that includes Swyx's episode, we'd have 276. We've got -- actually, all of our podcasts have shows going out this week, so we'll have even more by the time this one goes live.

About 280(ish).

Call it 280. 69 episodes of the Changelog thus far. 45 of those are interview shows, 24 of those are news episodes. From those 45, here are the top five most listened to episodes of 2022. Number five, "Making the command line glamorous" with Toby Padilla from Charm. That was also mentioned by one of our listeners as a fav.

Wooh!

I liked that show, and I think we ended up rebroadcasting it into the Go Time feed...

We did, yes.

...which means it got probably way more even listens than this number. So good for it, I guess. I have tried out some of the tools from Charm, and they are cool. I've kind of put that whole command line Changelog client on the sideline, as I work on other things, but I do want to return to it at some point. Number four, "Principles for hiring engineers" with Jacob Kaplan-Moss. This one almost made my top five favs.

Me too,

I think this is the one that engineering leader -- the most people who are in management, leadership, those kinds of roles, have reached out to us and said "This is a must listen for me and for my peers." So that's cool. Number three, "Securing the open source supply chain."

Good. Feross.

\[01:22:04.27\] It's from our good friend Feross.

Yeah. Socket - high hopes. High hopes for Socket. High hopes for Feross, what they're doing here.

Yep.

And it's no surprise it's top three. And that one almost made my list, too. But it was so hard, Jerod... Like, in the pre-call we were -- we have to give a little nod to this, because in the pre-call Jerod and I were trying so hard to take all of the episodes we created this year and then turn it into five of our top fives. It's just so challenging. It's like choosing your favorite child. It's just impossible.

Mm-hm. Number two, no surprise here, "Wisdom from 50+ years in software" with Brian Kernighan. This one reached 50,000 listens, excluding Spotify... So once you add Spotify in there, it's probably closer to 60,000; maybe 55,000-60,000 listens. So yeah, that one's not a surprise. Number one though - this might be number one two years in a row.

I think it is. I think it is.

Was Jessica Kerr our number one episode of 2021?

I'm pretty sure she was. I don't have -- I was actually just wishing we did a slightly better job of the State of the 'Log behind the scenes notes, because then we can have like comparing top five this year to last year's... Maybe a bit more organization. Maybe this year can be the year we do that.

Maybe next year can be the year we do that.

I know we've documented it, but is it comparatively the same?

Yeah, I don't know about that. I mean, she definitely was in the top five. Anyways, this is called "One more thing every dev should know", which is a funny name, considering our last episode was called "One thing every dev should know." And so we got her back on to do one more thing. I think we're probably gonna do Jessica annually, at this point; hey, she's such a big draw. And of course, well-deserved. She's entertaining, exciting, controversial, smart, always brings interesting things to the show... Over 53,000 listens on our platform. Not sure about Spotify on that one. So all of our stats are excluding Spotify and excluding Google Podcasts, because they re-host, and they then have their own stats. And we're kind of big on Spotify. We're not big on Google Podcasts. I'm not sure how many people listen on Google Podcasts. But Spotify has a substantial enough audience that all of our numbers would be better if we were able to suck them in. But regardless of all that, Jessica Kerr is the number one most listened-to guest of the year. So congrats to her. That's episode 483, called "One more thing every dev should know."

Our job as developers - I don’t wanna think of it as writing software; I think of it as changing software. Because that extends forever into the future. So step one, get it out there, step two, change it. Step three through infinity - change it.

So I went back into our history a little bit, and we have our wires crossed. So State of the 'Log 2020, "The one thing every dev should know" in 2020 with her was second-most popular.

Okay, so I overshot a little bit.

And I think we didn't have her on last year, so that's why she wasn't on the list. So she was not on the most popular list last year because she wasn't on the show last year. It was in 2020.

Right. So what was last year's number one?

Well, "Why we love Vim" was top.

Oh, yeah.

Second one after that was Nick Janetakis, "Modern Unix tools." That was fun.

That was fun.

"OAuth is complicated" was number three. "Why Neovim?" trailing at number four, and then "Lessons from 10,000 hours of programming" was number five.

Yeah, that's a good one. That's a good top five right there, if you ask me...

That is.

Now we're not just navel gazing, we're just complimenting ourselves.

What a fun year, Jerod, 2022.

It's been a good one.

It has been a good one. Very, very excited. Got a lot of fun stuff happening for 2023. Very excited about this next year... Wow. So excited.

Well, we can wrap up by saying thanks to everybody for hanging out with us, for listening to our shows, for hanging in our Slack and talking to us... Everybody who comes on the shows, everybody who talks to us, we appreciate.

Yes...

And we hope everybody has an awesome end to their 2022, regardless where you are, and we hope that you have an awesome start to your '23. We'll see y'all on the flip side. We'll see you on the flippity flop. That's Michael Scott... \[sample 01:26:06.06\]

And if you don't know, changelog.com/community. Too easy for you to join, hang with us in Slack, say hello, lots of people in there... You're welcome. And you're welcome.

And thank you. And you're welcome. Kaizen! Oh, no, that's a different show. That's not how we end this one.

\[laughs\] That's it.

SOTL.

Yeah.

Bye, y'all.