Changelog & Friends – Episode #61

Reverse rug pull, so cool?

with Adam & Jerod

Featuring

All Episodes

Jerod & Adam share our Zulip first impressions, react to Elasticsearch going open source (again), discuss Christian Hollinger’s blog post on why he still self-hosts & answer a listener question: how do we produce podcasts?

Featuring

Sponsors

SentryCode breaks, fix it faster. Don’t just observe. Take action. Sentry is the only app monitoring platform built for developers that gets to the root cause for every issue. 100,000+ growing teams use sentry to find problems fast. Use the code CHANGELOG when you sign up to get $100 OFF the team plan.

Paragon – Ship native integrations to production in days with more than 130 pre-built connectors, or configure your own custom integrations. Built for product and engineering. Learn more at useparagon.com/changelog

Coder.com – Instantly launch fully configured cloud development environments (CDE) and make your first commit in minutes. No need to traverse README files or await onboarding queues. Learn more at Coder.com

Test Double – Find out more about Test Double’s software investment problem solvers at testdouble.com.

Notes & Links

📝 Edit Notes

Chapters

1 00:00 Let's talk! 00:38
2 00:38 Sponsor: Sentry 01:42
3 02:20 Robots & Friends 01:51
4 04:11 Stranger than (non) Fiction 01:15
5 05:26 All Things Open 2024 01:37
6 07:03 How to hug Adam 00:52
7 07:56 Zulip first impressions 10:34
8 18:30 The reverse rug pull 07:19
9 25:49 Sponsor: Paragon 03:26
10 29:15 Why I still self-host my servers 15:24
11 44:38 How we podcast (Adam flex version) 06:15
12 50:53 Sponsor: Coder.com 02:18
13 53:11 Sponsor: Test Double 03:05
14 56:16 How we podcast (Jerod direct version) 03:07
15 59:24 How we Audition 03:00
16 1:02:23 How we chapter 06:44
17 1:09:07 How we chapter (Jerod flex version) 03:45
18 1:12:52 Adam's Overcast chapter plea 01:09
19 1:14:01 A JS Party chapter story, Nick 01:31
20 1:15:32 Random access FTW 01:57
21 1:17:28 A little Kaizen session 02:39
22 1:20:08 Thank you for caring! 02:23
23 1:22:30 Coming soon 01:29
24 1:23:59 Next week on The Changelog 01:51

Transcript

📝 Edit Transcript

Changelog

Play the audio to listen along while you enjoy the transcript. 🎧

The only Bob Seger song I could name is “Old time rock ’n roll.”

That’s like the worst one.

Ah, well, I don’t know the man’s –

All Bob fans kind of hate that song.

I don’t know the man’s work, obviously.

You’re missing out, man… I’ll have to give you his greatest hits. It’s on Spotify.

I’m sure it is. I don’t use Spotify, though. I only program against it. I’ve been coding against Spotify this week, so… I’ve finally cracked it, man. I’ve finally cracked it. We’re gonna get our data out of there.

Is that right?

I’ve got it, dude. I’m pulling data out of their unofficial API. It’s cool.

Is it legal? [laughter]

I don’t know, honestly. I’m sure it’s not illegal. Well, I’m not sure it’s not illegal. It’s one of the two.

Well, they have terms of service.

It’s either legal or illegal. I don’t know.

It’s one or the other.

I mean, listen, they let you log into a dashboard and look at it, so that’s all I’m doing. Only it’s a robot.

I love it. I love bots.

A robot doing it for us, so…

I used to like bots more before there was artificial intelligence, and now they’re kind of scary. Bots were controlled by us, and now they’re kind of like maybe not controlled by us. So when I said bots, I kind of cringed internally; maybe I don’t. I don’t know.

I feel like we still control them, do we not? I mean…

So in the Bobiverse book five…

That’s the deep cut that I don’t have any access to…

That’s okay, though. That’s okay. I think it’s in the synopsis of the book where it’s not plot ruining, but there’s a premise of artificial intelligence not being nice.

Let’s just say. Trying to escape. Trying to be nefarious.

And it’s a little scary. It’s just a book, though.

That’s just made up.

For now… Until five years from now when science fiction becomes fiction.

I’m skeptical.

Alright, I should say, sorry, nonfiction.

Yeah, I was going to correct you, but then I was like “I know what he means.”

You know what I mean, yeah.

Yeah, totally.

And like an idiot, I still can’t [unintelligible 00:04:25.04] “Is it nonfiction or fiction?”, every time in the moment.

It is kind of backwards, isn’t it?

Because one’s the negation of the other.

And you think you’d start with the real thing, and then negate it for fiction.

And then the movie ruined it for me, Stranger Than Fiction.

Stranger Than Fiction. That’s Will Ferrell?

Will Ferrell.

I never saw that one.

It was a strange movie.

Was it stranger than – but it was fiction.

It was good. It was a good breakout role for him. It was different than all his preceding roles. It was like the first time you went out of the straight up comedy role.

Right. And he did it, he accomplished it?

He nailed it, in my opinion. He was solid.

Really?

Like, I was rooting for him. He worked for the IRS. He was an auditor.

Who’s the best comedian crossover to drama? I mean, short list, right? You’ve got Adam Sandler, you’ve got Jim Carrey, you’ve got…

Adam Sandler’s the GOAT, bro.

Yeah, he probably is.

He’s the GOAT.

Did Mike Myers ever cross over? I think he tried to. Oh, Steve Carell’s actually done pretty well.

Anyways. Did you book your flight? I booked my flight.

I booked my flight.

We’re booked, baby.

All Things Open…

All Things Open.

…2024…

…2024…

Raleigh, North Carolina.

Raleigh, North Carolina. I’m gonna repeat what you say.

We’ll be there.

I think we’ll be there. I can’t see why we wouldn’t be there.

We’re planning on being there.

We’re just not sure where our booth is at yet.

We don’t know where our booth is at, but we’ll have a booth, right?

We will have a booth. We will have options, I believe. I think we have an option of in the main area near the ballrooms, where we normally are… But where we have normally been is highly sought after, and they had a swell of sponsorships in the end, and so money trumps non-money. And I guess we’re in the non-money bucket.

We’re non-money people.

[00:06:11.19] Yeah. I would say we bring the money, but we’re not given the money.

I mean, we’re so money that we don’t even know it.

That’s right.

But we don’t actually have the money, so it’s different.

I get it. We’ll be there, though.

It’s cool. We’re flex.

We’ll have our microphones. We’ll have stickers. Who knows what else we’ll have.

I’m hoping we have at least one TV with clips.

Yeah, we’ll have our clip gallery behind us, hopefully.

So if you’re going to be at All Things Open 2024, which is at the end of October, 27th through the 30th, I believe, is our official trip dates…

Yes, Sunday through Wednesday.

Find us, talk to us, high-five us. Adam gives hugs, if you’re into that kind of thing.

Sometimes. I’ve pulled back from the hugs.

Oh, you have?

Yeah. Certain individuals, depending upon how they look at me, it’s hug time.

How should one approach if they’re interested in an Adam hug?

Great question. Geez… I would say longingly, and with a slight smile.

Arms wide open, or is that too much?

I would say come with a handshake and convert to hug.

Right. Well, that’s the easy version. You shake the hand, and you lean in, you can do the back pat, and then you can be like – you can whisper in his ear, “I want the whole thing.” Something like that.

“Give me the full hug, Adam.”

[laughs]

There you go.

That’s getting weird. But hey, if you do that - well, then you listen to the show, and you follow orders, or at least entertain our silliness.

Who knows?

So there we go. All Things Open 2024. Also, our show with Alia Abbott has actually borne fruit, which is rare for us. We’ve actually – we’ve followed up. Zulip. We’re trying out Zulip, in earnest, meaning we’re actually using it, and we’ve invited people into it, and we’ve got people in there, and we are checking it out. Now, I thought we’d talk about this on Kaizen, which is coming up just next week… But our Kaizen has busted at the seam, so why not just do quick first impressions here? What we think about Zulip after using it for two weeks-ish, maybe 10 days in earnest.

You know, I did not expect me to like it as much as I like it.

Really?

I really was only trying it out of seemingly force. Like, no one was forcing me, but I felt like we really had to.

Well, we said we would.

Well, yes, we did.

So yes.

But there was no force there. You can always make empty promises, or at least –

No, man. No.

…empty words, to some degree. Well, I don’t mean to like not do it, but I mean like, I was begrudgingly trying it out, let’s just say.

I know you were. I felt it. And I felt your begrudge for a minute.

Because you’re like, you set it up, you invited me, and then you’re like – I can tell you’re like “And I did my thing. I did what I was supposed to do.”

Yeah. But yeah, well, I felt like I had at least done that. But what clinched it for me - and this is where we were chatting in our own DMs in there - was I was like “I really just feel like we need more realness to it.” I can’t just DM with you and be like “Yeah, I like it. Let’s do it.” That’s not going to be it. We’ve got to have some real stuff in there. And wow, in addition to my mind changing, or at least my judgment, my prejudgment…

Sentiment.

…yeah, changing, there’s people in there. And I would say there’s more deep conversation in Zulip than has ever been in Slack. Now, there’s been lots of conversation, but it seems like it’s deeper and longer because the thread is there, the topic is there. I don’t know, something uniquely different is in Zulip. And now when I go back to Slack, I really just feel like I have no idea where the conversation’s at.

[00:09:53.17] Right. Yeah, a couple of things that have struck me… The first one is required topics makes you think a little more before you do something or say something. And so there’s a little more intentionality to what you have to do, which is friction, which sometimes stops you. It’s like - I think you said it - this feels more like real-time email. And because when you start a new email thread with somebody, you have to put a subject in that email. And that’s what a topic is, really. And so it does feel a little bit more like that compared to Slack.

The second thing is - and our community folks have figured this out quickly - it feels like it’s built by nerds more so than Slack does, even though Slack has some nerdy things for sure in there. Nerds built this, and that’s really cool. You can feel that love.

And then the third thing is, we’ve already had a success story with one particular community member who already interacted with the Zulip team and their dev channel, and had influence on the way the product was being built. And that’s the open source ethos that we love. We would never have that kind of access with Slack. We’ve known folks inside of Slack for years, off and on, engineers, and we’re like the smallest fish in that huge ocean of users. So that’s pretty cool. It has its problems, it has its warts… But yeah, kudos to the Zulip team for putting together something pretty cool.

Yeah. And I think when you look at – like, the first thing I start to think about when I evaluate something is the experience and the user interface, which is not simply just design, it’s also experience. I thought because “Well, they’re not venture backed…”, not that they’re less talented, they’ve got less resources to apply to eke out all the UX permutations. And I really just thought it would be less polished. And that’s not true. Like, the “built by nerds” thing, the keyboard shortcuts is what’s selling me. It’s hard-selling me. Like, being able to navigate around…

Yeah, it’s completely keyboard-driven. Like, you can drive it without your mouse 100% if you want to, which is really cool.

Yes. Which is why when I go back to Slack, I feel like I’m clicking everywhere, and it’s so fatiguing in comparison, one-to-one, going back and forth. Because we are straddling the line, where we’ve got a micro version of our community in our Zulip, and there’s conversations happening, there’s topics… I feel like it’s pretty fleshed out, in a way. It’s on its way to being fully fleshed out. And then we’re going back to Slack for other conversations, because we haven’t made that cut over yet. And I’m not sure if we will; maybe we can talk about that here on this conversation, but…

I don’t know the answer right now.

Yeah, I mean, I kind of feel like I know the answer.

What is it?

I feel like it’s yeah. Like, Zulip is the future.

That’s how I feel. I feel like – so I didn’t share this with you yet, because I’ve been… You know me, I’m reserved, I suppose. I haven’t been talking a lot in there, but I’ve been observing people conversating, and participating through observation, and here and there jabbing in with some stuff… But the web public stuff is super-interesting. I think it could be cool to find ways to like make the Changelog community - not so much bigger to be bigger, but this idea that I’ve always shared and I think you’ve adopted it too, is this idea of “No imposters here. Everyone’s welcome.” And that same idea where maybe Zulip, because we have this full history and we can self-host it, and all the things… They were not locked into their cloud. We can begin there, and move to our own self-hosted version of it, I feel like we have a lot more room to like really lean even heavier into community, where we could have before, but we would have had to pay for it, and it was a significant cost, in the Slack world… Where maybe we didn’t put that kind of pressure on ourselves to do so because of our known limitations… Whereas now I feel like unfettered. Now we can literally do what we want. And I don’t know this to be true, but I can hypothesize that it would be, that we can get pretty good buy-in and support from Zulip, in whatever ways. Like, if we have ideas for how to make this even more community-focused, then I think that there’s an opportunity there that was definitely not there with Slack.

[00:14:17.13] I agree with all of that, and I think that there’s definitely ways where we could increase the amount of community discussion around the shows by deeply integrating into Zulip, versus what we have been doing previously, which is some of the ideas being tossed around in our channel about sharing certain channels publicly, and allowing people to lurk… Yes, allowing people to lurk. Sign up, integrating the actual comments of our episodes into that directly, the discussion… So yeah, there’s a lot of opportunities for nerdery and tomfoolery, that previously were kind of just - they were non-starters, because you were just deepening a relationship with an entity that you really couldn’t go deeper in any sort of communal way. It was like – what are those barnacles on top of the whale? We like just a little bit bigger barnacle, living off this whale.

Yeah… Well, we’re not their customer either. We’re not who they’re optimizing for.

So I suppose now that this thought has come into my mind in this moment, the only hesitation or reservation I have with Zulip is the uncertainty of their future… Which is not to say that I think their future is uncertain. Whereas Slack, maybe it’s the same. I mean, geez, products die every day, and they get deprecated.

Yeah, who knows.

Yeah, exactly, who knows. Even though they’ve got maybe deeper pockets, or…

Way deeper.

Yeah. Whatever. I think my only concern with Zulip is that reservation of “Will they persevere?” I know the team is capable of doing it…

Sure. But even if they don’t, then it’sn a open source project that we self-host, and maybe we just don’t go deeper than that. We just continue to use it as is. I mean, as is, it’s got completely serviceable community chat; completely serviceable, without any changes.

Yeah. Great point.

Obviously, improvements always welcome.

Well, I guess my reservation came from the fact that while I know it’s open source, I’m used to things like we’re using in Slack not being open source, and so my brain hasn’t said “Okay, this is open source.” So that reservation can be degraded down to 50%, versus 100% reservation.

Sure. Yeah.

Because that’s super-cool, and that’s – I don’t know if you listened to the show yet, but I monologued a little bit… I’ve been doing this a little bit lately in the end of our interview shows… Because I have thoughts, and I just started calling it like closing thoughts, and stuff. Because I’ve just got thoughts afterwards that I just like reveal… And this one in particular, I talked about potential. And I don’t know if you’ve ever heard me describe potential as kinetic energy stored, waiting to be released… Because I feel like that’s my reservation with Zulip, is they have so much potential. And it’s not that they haven’t achieved greatness, by any means. I’m not trying to degrade or downplay the greatness they’ve done, by any means. But I believe they have so much more potential to potentially unseat; not so much fully unseat the giants like Slack or Teams… But when you look at the feature set that people really want, I feel like Zulip’s got it. What they don’t have - and we talked about this on the show - is the awareness.

And something’s got to change there. And that’s where my uncertainty for them comes, is like “You’ve got to figure out how to market.” You can’t be the best unknown tool in the marketplace. Something’s got to change there. And I don’t know if it’s dollars spent, but something’s got to change with their, their GTM, their go-to-market strategy, how they talk about them, their awareness. Firefox, famously, “Get Firefox”, we’ve lamented on this in positive and negative ways over the years. Like, they did all that grassroots stuff with a very shoestring budget. Zulip can do something similar, I believe. And that’s where my reservation comes, is like, they’ve got so much kinetic energy stored, waiting to be released. This potential. And I want to see them do that.

[00:18:21.01] Well, perhaps in the spirit of open source we can also help them do that. I think our adoption would be useful in that sense. Let’s talk about some other things going on. Of course, with open source companies, you’re always afraid of a rug pull, as we’ve experienced many a pulls, especially of late… How about this? I’m calling it “the reverse rug pull”, in which they, I don’t know, put the rug back. Elastic. Elasticsearch. Open source once again.

How about this? “Reverse rug pull, so cool.”

That might be going a bit far, because it does imply that you have rug-pulled in the past… So you are already not cool, and now you’re putting the rug back… So I don’t know if it’s so cool, because the better thing is just like leave the rug where it was. It really held the room together.

Sure.

[00:19:12.15] That rug really tied the room together, did it not?

But yes, much cooler than the rug pull. We don’t want to talk too much about this, because we are talking with Shay Bannon, CTO of Elastic, soon. This month. He’s coming on Interviews for the deep dive. But yeah, Elasticsearch opened back up again, went full OSI-approved AGPL license. And Shay’s pretty excited, as you can just read into his announcement post from August the 29th… And we’re excited too, right? I mean, you called it so cool, so you must be for this move.

Well, it was just a saying. I don’t know if I have that belief yet. The jury is still out for me. So when I read his words, I can read between the lines and hear the sentiment of positivity. Like, he seems to be very excited.

His roots in terms of how Elastic was born was hacker. And I think I was reading over on InfoWorld from our friend of the show, I suppose, Matt Asay… Is that how you say his last name?

It’s either Asay or Asay.

Asay, Matt Asay. And he - I’m going to paraphrase, but he talked about Shay Bannon’s initiation of Elastic was he personally paid for the trademark for Elastic to protect his work. He was the developer, making it in an apartment, I think in New York City. And so very much like the stories we all hear about, which is why I’m kind of leaning back towards this “so cool” aspect… Because we all have dreams when we produce works in the world, our art; sometimes it’s code, sometimes it’s pixels, sometimes it’s both. And I believe that in a position which we may not have fully agreed with - and we can argue and have argued, and have done shows on, with the Elastic versus AWS scenario - that you’ve got to do sometimes what you’ve got to do. And we may not agree that “Hey, let’s now go closed source to protect”, but I can at least respect their choice to do so. I may not agree with it, but I can at least respect that they’ve had the fortitude to do so.

And now, to be in a position – the same hacker that made it initially, when you read his post on it, you can tell he’s excited. You can tell that he’s got joy writing the words, and revealing this, and going back to an OSI-approved AGPL license. And at least there is now a trajectory for, as he says, more open source in the world.

[00:21:59.26] Yeah, I agree. Obviously, we have questions, which is why we invited him on the show to talk… One of the main things I want to ask him about is he asserts in his post that their move was successful. Like, that was a successful thing they did, despite him not wanting to do it. And now it’s provided this opportunity to go back. And I would love to get that all out in the open - how they define success, how do they know it’s successful… Because we recently covered RedMonk’s analysis of open source rug pulls and whether or not they’ve been worth it. And to the best of her ability, I think it was Rachel Stephens over there at RedMonk, did the analysis on a bunch of at least publicly-traded rug pulls, so that you can get the data. And she couldn’t find any correlation between actually any sort of meteoric rise after the license change, especially ones that have been out there for a while, including Elastic. So that’s an interesting thing. And we’ll be talking with him soon. To our listener, if you have specific questions, lines of thought, conversation that you’d like us to broach with Elastic’s CTO, Shay Bannon, definitely let us know… In Slack, in Zulip, I don’t know; email editors [at] Changelog.com. It’s probably the safest place right now… Unless you’re already in our Slack or already in our Zulip. Then just go ahead and use what you’re already doing. That works.

I was just thinking, as you were suggesting that, if they were going to pile on or start anew in Zulip, where would it happen at? That is the challenge. Not to go back there, but that’s the challenge of “Where does that live? Does it live in general under a topic called shows? Does it live under Changelog Podcast”, that isn’t a channel yet, that will be a channel soon? Or interviews?

Interviews. I think we are going to create one channel per podcast, as I’ve already started to do, where we publish new episode notifications for discussion… Which I haven’t created one for this show yet, because we haven’t shipped an episode yet… Although today or tomorrow, I assume, we’ll have last week’s going out.

Yes, sir.

With Erez Zukerman. In that case, I think it would just go to The Changelog, or Interviews, I guess, and be like “Questions for Shay Bannon.” And you’d just start a topic called that. But you don’t have to post into an existing topic. You can start a new topic, and it could be ephemeral. It doesn’t have to be a long-lasting thing. Just post your conversation there, and then Bob’s your uncle.

Bob is your uncle.

That’s my thought on the matter, at least. But yeah, you do have to stop and think before you just start talking, don’t you?

And so that’s the other challenge with Zulip, is - my reservation came from “Now everything has to be structured.” And then you have to be the person who says “You’re off topic” or “That thread”, like they do in forums. Remember that, when you’d be like slapped around basically in forums, like “You’re hijacking…” You know what I mean?

Yeah, hijacking. Well, the cool thing about Zulip - now we’re turning into a walking advertisement. I’ve already done this once… You can just take an entire topic thread and move it to a different channel, or messages that are on a topic - you can just take that entire set of messages and move them to a different topic. So instead of saying “You’re hijacking”, you just move them to where they belong, and people are happy to be like “Oh, okay, it’s over here now.”

Yeah, cool.

So that makes it somewhat less of a one-way door, into more of a two-way door.

How about this? “Reverse rug pull, we’ll see if it’s cool.”

[laughs] There you go.

And I guess we’ll find out when we talk to Shay himself, and release that episode and see how we feel about that… We’ll see.

Break: [00:25:48.25]

Well, let’s talk about another Changelog News item that I did not feature in the audio… I actually think I just put it in the list of links at the bottom, so I didn’t write anything about it, but it’s a really nice post by Christian Hollinger, or Hollinger, perhaps… “Why I still self-host my servers, and what I’ve learned recently.” I thought this one would hit home with you, Adam, as a home labber, but maybe not a self-hoster… But maybe a self-hoster… This article is about two things, Christian says. “Why I still bother, and what it has recently taught me. Think of it as a brief retrospective and an encouragement for readers to go down the same rabbit hole.” So Christian likes self-hosting and he thinks you should, too. It’s a long post, but to summarize the two reasons at least - independence, reason number one, and “Learning is good”, reason number two. Are you convinced, Adam?

I concur, it is a rabbit hole. I am convinced that you should at least attempt to do this. I don’t think you should feel bad if you decide that it’s not for you, because it is a rabbit hole. It does require, a certain level of responsibility over time, especially if your family or others around you begin to depend on those services, and you can no longer dedicate the time necessary to keep going.

I suppose as long as you’ve got the time and the desire, then self-hosting is certainly fruitful. You will learn a lot. I learned a lot about many things that I just never had to really consider before. And I think that I’m a better conversationalist around technology because of it, and I think that I’m just generally more wise to the tech in the world. Whereas before I had a once removed relationship with it, where I would work with someone who was deeper in the knowledge, and now I have firsthand knowledge.

Yeah, 100% agree. I have self-hosted many things throughout my life, and I learned so much doing so. Both production, small business, and friends and family stuff, whether it’s self-hosted on machines in my house, which I have done for a long time… I had a Linux server that ran in my home, and I ran all kinds of services off of that. And then also VPS, self-hosting, shared hosting, self-hosting… You know, just managing your own things. And yeah, the amount of learning is really not to be compared with. You can’t fake it, it’s just real.

But the learning comes through trial, and it comes through things going wrong. And this post that he writes is like “Here are the –” One of the sections is like “Things that broke in the last six months”, you know? And it’s like, yeah, you’re going to have that kind of stuff. And it’s similar to any sort of maintenance. I don’t know how many large appliances and vehicles you own, Adam, but you and I are both old enough to know… A house, a car, a laundry machine, anything that’s significantly expensive and significantly complicated breaks. And when you own it, you own the break, you know? And you’re going to either pay to fix it, or learn to fix it. And for me as a guy in my 40s with a pretty large family, I don’t have the patience or the time to self-host anymore, but I think everybody else should. [laughs] Not everybody else, but given your circumstance, I think everyone should try it, and some people should do it. And there’s certainly things that I’ve thought about self-hosting, even though I don’t have the time, because of privacy and autonomy, and the independence that Christian speaks of. For me more importantly than the learning, because I’ve already learned enough, I think, in the category, is there are things where I want the independence and the privacy. And so that’s why I consider it, even though I don’t want to do it, because I know how much of a headache it can be.

Yeah. I’m definitely not in the - for me specifically - “Adam, you must self-host all the things” camp. I’m not in that camp.

Well, I asked you to self-host Zulip and you were like [unintelligible 00:33:36.24] on the show, remember that? I put you on the spot…

No, I was not excited. Well, it’s just too critical. I do self-hosting for the things that are – that I don’t mind being down. If somebody’s like “Adam, why is Zulip not working?”, that’s the worst world for me ever. I don’t want to live in that world. I don’t want to be responsible to that degree. I don’t mind assisting, it’s just not something I want to personally be responsible for. Nor do I think I have the expertise currently to do so.

[00:34:05.15] Well, you probably would earn it over time.

I would earn it over time, but I’m like…

Battle scars.

Yeah, I mean…

And you get better at it, and it can become faster, because you’re like “Oh, I know what probably went wrong…”

Yeah. Confidence is there. I’m sure I could do it. It’s not about lack of confidence. It’s about lack of desire to hold the responsibility. I would prefer Zulip to run as Zulip should run, and be up for everyone all the time, and it not be my problem.

So what are some things you’re okay with hosting?

Pi-Hole. Stupid stuff.

Pi-Hole sounds like it’s critical, though.

It is, but it’s so easy to run.

Well, that’s what you find out about a lot of these things. They’re very easy to run. And then you’ve just got to make sure that they’re updated, or fix them. Getting it set up is the hard part. They run while they run. Things go wrong here or there… A network connection goes down… A time database gets out of sync… An update fails…

Usually for me it’s like “Oh, critical vulnerability in this thing that you don’t care about anymore.” And it’s like “Oh, I have to go update right now”, even though three people are using this. Like, that kind of stuff. But most of the learning and most of the pain comes through that initial setup with a new technology.

For sure. This person’s – what is his name? Christian? Is that right?

Christian goes deep. He is self-hosting – and he gives a list of things that he is self-hosting. I’m trying to find the list quickly.

It’s at the top.

My Services, yes. Pi-Hole’s in there. RouterOS… I won’t read them all. UniFi controller’s in there, TrueNAS is in there… I think he even mentioned he does it via Proxmox, which I would never do, after trying it once… I don’t like to virtualize a NAS and virtualize the access to the disks. It’s just too critical. So I feel like a dedicated box to your storage device is proper. Even if it’s overkill or a waste. You know it’s done right, and you can pull the disk immediately if you need to and swap it, and do some stuff with TrueNAS to ZFS, or straight to ZFS if that’s all you have. VS Code, I thought that was kind of interesting…

Yeah, VS Code… I guess it’s like a web-based version of it, or how do you self-host VS Code?

Slight plug… I think they might be sponsoring this episode, too. And I’m not saying this because they sponsored it. I truly like their technology. Coder.com is so cool. I think it might be like this because Coder.com is a cloud development environment. So we’ve heard of this with like Codespaces, right? And Gitpod. Those are all primarily Docker container-based, where you’re not running in a VM, you’re running in a container. And Coder.com does both. But what they do uniquely is when you – I actually have a Coder.com instance in Proxmox, so I dedicated a brand new VM I created from Ubuntu, made that the Coder box, and now I can turn that into a cloud development environment. I can have code a new project, a new instance, like a new provision of a thing for me and my developers - which is just me, because I was just tinkering with it… And inside of Coder - the reason I bring it up is not to plug them. It’s because whenever you launch the code, you launch VS Code - you can, at least. You can launch Vim as well. So it’s for all the hackers. You can launch VS Code in the browser, and it will connect to the code on the VM in the browser. Like, you’re literally editing it like that. I think that’s what he’s doing here, is he’s self-hosting similar to that, but like it’s the dedicated layer of just VS Code, where it’s remote-connecting to somehow the code on your local machine via the browser. Probably through the LAN.

That’s hardcore.

[00:37:51.26] That is so hardcore. And I don’t know why you would do that, but you know… In the Coder instance you do it because of resources. You want to take all those resources - CPU, RAM, GPU, whatever you need to dedicate to your environment - into that CDE, that cloud development environment. Versus your local. One more layer. This is something I think is kind of cool… I have my Jekyll blog still yet, so adamstakoviak.com is a Jekyll blog, basically. I do not run that locally. I have set that up to run inside of Docker. Because I don’t want to fiddle with Ruby and any of the gems and stuff like that locally. Like, it’s all inside the Docker container. Which I think is so cool. And I never have to mess with local stuff.

That is cool. I’m sure a lot of these services are dockerized, which makes it a lot easier.

Jellyfin for sure is.

Yeah. But I mean, he’s got MariaDB running locally, Redis, InfluxDB, Jellyfin, as you mentioned… Gitea, so his local Git server… Local wiki… Of course, Nginx, because his website and his blog are both hosted at - I assume this is his house.

This is like a homesteading version of software. He’s a homesteader.

You know, I’m glad you brought that up, because I thought that whenever I saw his photo of his garden. I was like “Self-hosting and gardening go hand in hand.”

A hundred percent.

Yeah. Autonomy, man. Like, leave me alone. I’m going to run my life.

But that’s like – if you’re gardening, then you’re like “I want to be self-sustaining”, but then you’re also tethered to technology, because you’re still self-hosting… So you’re not like ejecting from the world, right? You’re not remote in Denver, on a cliff. Or not Denver. Colorado is a proper state. On a cliff, in the mountains, remote. You’re still tethered to technology.

Right. You’re not living out in the boondocks of Montana.

Right. Yeah. I like Montana, too. Yeah. What this looks like to me - and I don’t want to be negative Nancy, or negative Tim, or pick your name, by any means… Is if you’re trying to be productive, this seems like a lot of yaks that could be shaved, on the daily. Like, if you’re self-hosting your local Git server, provided this is all routinely easily maintained… Your VS Code is in the browser, you’re maintaining whatever that is, and whatever can go wrong could go wrong, and you can fix it… But when it goes wrong, you’ve got to fix it. Do you think that this is a list of shiny objects that can be distractions? Or just simply things that need to be shaved like yaks?

I don’t know, because nerds are going to nerd out. And I feel like some of this stuff is probably him nerding out, which is totally fine to do. So in the case of that, like, define productive, if this is what you enjoy. He obviously enjoys it.

So it’s almost like similar to gardening, where if you don’t love gardening, don’t go plant a big garden, because it’s a whole bunch of work. And the people who continue and persist in that, they love the work, they love the process. And so for them, that’s what it’s all about. And so yeah, I mean, there’s certainly some yak shaving going on, but you know, if you enjoy shaving yaks, you could yak, don’t talk back… I don’t know, I just got stuck on yak there. But go ahead.

Christian closes with this, which might be a good end cap to our discussion. He says: “If you’re a software engineer, I recommend self-hosting things. You learn a whole bunch of things through forced exposure to problems that you’ll be less likely to encounter in your day job, which in itself is a benefit. Even better, I do believe you’ll wind up using at least some of these things in your day job eventually, provided you work on something vaguely backend-related. By hosting stuff yourself, you also get a reasonable level of autonomy, or at the very least some hedging against the corporate dream of your entire life being a perpetually rented subscription.” I think that’s nice.

Good ender. I mean, it’s certainly romantic. That part gets me.

That does get me.

Not enough to do it, but enough to appreciate it.

That’s where I’m at. Because he mentions Nextcloud in one of his services… I don’t want to be – I mean maybe I do at some point, but at least right now… I don’t want to be responsible for calendars being up, photos being up, file services being up… I mean it just seems like “Oof…” But maybe Nextcloud makes it easy. I just feel like, in the moment, without having tried it, it feels like a lot. And I speak to that because of the rented subscription. The easiest place you spend money is like Dropbox. Calendar I guess comes with our Google Suite for our business… And in other cases you’re getting it for –

For iCloud, if you have an iCloud account.

Yeah. So you are on this “You will own nothing and be happy” World Economic Forum prediction several years back, where someone famously said, and maybe even infamously said, “You will own nothing and be happy.” I just don’t know about that. Like, they’re perpetual. It feels like a long-term debt that society puts on you. Like, to own an iPhone, you kind of inherit a level of perceived debt, necessary cost to maintain the service. Not just the phone service itself, but the things that come with it to use it. Like photos.

I don’t know if that’s the right word, but I’m with you in spirit. I think that –

Well, it’s debt if you… The reason why I say that –

You have to pay it back.

…is that you commit to a payment.

Well, you’re renting.

Right. You’re indebted to pay for the service if you use the service.

Yes, you have to pay for the service if you use the service, and if you stop paying for the service, then the service goes away.

And I think renting is totally reasonable in certain cases. Like, I don’t want to own a calendar service, I want to rent one. Because the calendar is only important over the next month, weeks, and maybe years… But have you ever considered about your historic calendar? I couldn’t care less what was on there. Every once in a while you go back and like “What day was that?” And you check an event. And you’re like “Oh, I didn’t put it in the calendar”, and you’re like Dang it!“But if it’s in there, it’s kind of useful. But your past calendar events… I couldn’t possibly care less. Rent that service. If it goes away, it goes away. Other things… Photos? Opposite. The history is what it’s all about with photos. That’s the kind of stuff that I’m afraid to self-host.

Availability and history. And reliability that it’s there forever. Deletion of photos is very, very bad.

Yeah. Alright, let’s move on to our final topic. So you know how we take episode requests…

Yeah, man.

Oh, that’s so cool.

It just takes a while sometimes.

Is it is it at Changelog.com/request?

That’s right.

Oh, gosh. That’s a great URL. I love that.

That is really nice. Well, we read every request, and we don’t fulfill every request, but every once in a while we dig one out of the ashes, and we would fulfill it three years later. So shout-out to listener Alex, if you’re still out there. If you’re still listening, Alex, three years later, you’re a trooper man. We appreciate you. Because that’s a long time to listen to –

And you should be in Zulip.

…anything. And you should finally be happy that we’ve answered your request. This one’s navel-gazy and self-serving to a certain degree, which is why we didn’t do it for a long time… But I thought it’d be a nice end cap to a Friends episode with just the two of us, where we can take at least one listener question/request. Alex wants us to talk about how we podcast. How we produce podcasts. He said he’s heard of some really cool workflows from both the Linux people, and you guys - which I guess means we’re not Linux people - about different ways you record and upload scripts. He says “You guys, if I recall, have some type of encoded timestamps to the mp3s. Destination Linux timestamp the episode while they record. Jupyter Broadcasting uses some type of all-in-one container for recording”, if he recalls… And we have master feeds, they also have master feeds… So just some of the inside baseball on how we produce our shows. Maybe some of the nerdier parts. I don’t know. Some of it’s boring, at least to us, because we do it… But maybe interesting to other people.

[00:46:25.06] I think it’s super-interesting to other people, because I’ve been having some conversations with a future branded podcast we’re producing…

Oh, yeah.

And they have questions, and I have answers, and they sit back and listen as if we have pulled up a glass of favorite drink near a campfire, and it’s just so fun.

[laughs]

Seriously.

Really?

Yeah. I didn’t think what we’d done is – you don’t know what you know until you try and teach other people, basically.

Right.

And then they’re like “Wow…” I mean, to us it’s logical and simple because we’ve repeated it so many times that –

And refined it.

Right. We could probably do a lot of what we do, to some degree, in our sleep.

Right.

You know, figurative speech, of course…

Figurative speech… [laughs] I swear I’m not sleeping right now.

Not literally in my sleep, but… For example. I’ll give you an example. And this is not necessarily answering Alex’s question, except for maybe to flex a little bit, and I’m not a flexer… I wrapped up this conversation yesterday for this future brand of podcast, that I’m not sure I can mention yet, which is why I’m being vague… And the idea was “Okay, we’ve got the music components together”, and they can’t hear/see the future vision that I can see coming, because I’ve got the experience, and we’ve got the experience, and I know where we’re trying to take them. They haven’t done that yet, and so they have questions and reservations and apprehensions that need to be resolved… And the easiest way to do that is to give them a version, a listenable version of it.

And so yesterday, within the span of an hour and a half, I think, I sat down and produced a fake episode that uses the intro music, and the timing, I [unintelligible 00:48:05.27] the intro and how I think it should work, how it blends into the show, how the show ends, and how the music comes back in, how the show could end… And I used all the components that we’ve produced for them, and gave them three different versions because we have three different outros they’re considering. The intro is solid and it’s not being scrutinized in any regard. But the outro is like, they’ve got questions of how it should – should they be the same, should they match up? How similar should they be?

And you know, doneness is the enemy of perfection. If you can’t just get it out there, it’s going to – well, perfection is the enemy of doneness. I said that backwards. You can’t strive for perfection; you can obviously – you know, don’t wait until it’s perfect, or seemingly perfect, or all the answers are answered, to get momentum and move.

The reason I’m saying this story is that in the span of an hour, I solidified and created what has been in my mind for many months, waiting to take action on when the time is right… And it’s done. And I think when you listen to it, you’ll be like “Yeah, that’s a pretty good version of what it’s going to be.” It’s obviously four minutes, not 40 minutes, like it might be, or will be… It’s a micro version, listenable, that’s actually kind of compelling. And when I listened back to it, I was like “I kind of want to listen the whole thing now.” And it’s just a fake; it’s just episode zero.

Well, it’s not a teaser. There’s nothing –

There’s something in there.

Well, but not – not yet.

If I told you, I would reveal too much.

But I was like “Yeah, I would listen to this. This is cool.” And the reason why I say that is because we’ve done it so much that, in the span of an hour-ish, I created what is likely to be the future of that. Not two days, or several days, and you know, had to go back and forth… It was just done.

You’re just inventing the future, right in front of their eyes and their ears.

That’s right.

Alright. Solid flex. Stay tuned for branded podcast upcoming, of Adam’s design, with a partner, which will be the second time we’ve accomplished such… We do produce Big Tent, Grafana’s Big Tent with them, which is the first one…

And selective, but open to that process with others. Very selective. Strategically, because we’re a small team. But to more directly answer Alex’s question… Maybe he’s interested in tools, techniques, like how we do what we do, and what we use.

He’s not interested in my flex. He’s like “Adam, just be quiet. Let Jerod tell the true story here.”

We’re going to chapter that one “Adam flexes”, and then we’ll have the real answer…

“Jerod answers Alex directly” is the next chapter…

[laughs]

Break: [00:50:53.08]

I do have a propensity to answer questions directly, which some people appreciate, and others think is boring… So your mileage may vary. But in brief - okay, so tools and techniques. We record everything on riverside.fm, which is a proprietary in-browser technology that we pay for as a service. Happy to rent it. Don’t want to self-host it. I’m sure it’s very complex.

And very cool use of web tech. I mean, and it’s gotten better, it continues to get better… We’ve been on it for at least a couple of years now; very few incidents, so we’re happy with the software. We own a separate account for each of our podcasts, and everybody logs into their own account, they record in there… And that handles the majority of the actual video and audio recording and syncing. And then we export out of there into Adobe Audition for editing.

We have a kind of a three-step process in our editing. One’s called Prepped. So we prep the audio; this has to do with making sure all the tracks line up, all the boring stuff that people don’t ever think about. Sometimes prepping the sounds, trying to get a different version in case the audio of a specific track isn’t great, and then we edit it, content-edit. This is usually our editors; shout-out to Jason and Brian, who content-edit our shows… And their job is to basically cut out all the bad parts, which is just awkwardness, weird pauses etc. They do their work… Uhms and ahs… At that point they leave markers.

So Alex asked about timestamps and chapters and all that… We don’t do any time stamping or markering while we record. It seems like the ideal time to do it, but actually when you’re in the moment, in the groove, you’re not actually thinking about “Oh, this is a great –” Except for earlier, when I said “Here’s a chapter.” You know, every once in a while I’ll think about it, but… You just want to be able to just be free from all that, and just enjoy the conversation. So we aren’t doing anything – I think Riverside has… Oh yeah, they have a “Mark clip” button right there that we could use to like set a marker… But we don’t do that.

Sometimes there are things that we know about, and so we’ll just tell our editors afterwards, like “Hey, look out for this…” You can drop a marker here, marker there in our locals as we go… And I’ve done less of that than I used to. I used to do it more. But jason and Brian take good care of us, so they handle the edit, then they pass it back to us for mastering… Which is all the final stuff. Chapters, ads, voiceovers, music, final edit decisions if there’s any content editing to do… All this is in just this big old Dropbox folder, basically. It’s organized by show and by episode, and separate Adobe Audition sessions for each phase, so very much a manual version control system that works just fine, just by copying files and renaming them… And then we mix down a wav file and an mp3 file, and we upload them to our website… Hit Publish, baby. Now, that’s both detailed and glossing a bunch of stuff, right?

Yes. Yeah, the cool thing about the way that Adobe Audition works is that you have a file type called .sesx, and those files point to a local file set, essentially. It’s like a timeline… I imagine it’s probably – I’ve never actually read the file type. I thought it was proprietary maybe…

It was like an xml file, I think…

[00:59:46.14] It’s very xml-like, yeah. And so when you make these copies of it, the file is like a couple megs, at most. For an example, Jerod mentioned the Prepped version of this. We have a show coming up today on these ergonomic, really awesome keyboards from ZSA… And the prepped file is 130 kilobytes. The edited version is 2.2 megabytes, and because the master version only has a couple things added to it, it’s the same file size. Now, once those changes go in, I’m going to that right after the show, or after this recording that we’re literally talking into right now; I’m going to go do that work in the master file, and then that file size will grow probably to, at most, 3 megabytes. So these session files are very small, and are supported by the local file system, which has recorded sessions in their imported files… All these different things that are sort of like file system stuff that Adobe Audition uses.

Adobe Audition has been really good to use, I would say, over the years, because it moves from person to person pretty easily as an independent file system, or an independent directory that you can copy, and do whatever you need to do with it… And so we’ve been very happy on that front. And even chaptering within, in the wav file; when we do that, there’s a span ,that you can put like a marker, and another marker, and join those together and make them a chapter, or a two-point marker. I’m not even sure what they call those things, honestly. Merged markers, something like that… And those turn into chapters.

Range markers, I think…

Yeah. It’s been very easy to do that, and I think it’s important probably, Jerod, for you to talk about some of the stuff you’re doing with the wav file. Can I interview you a little bit on this process?

Sure, yeah.

I feel like maybe they might get more mileage on an interview version, versus a monologue version of it.

Absolutely.

So I know what we do… We mix down a wav file, and then we mix down through a process called match loudness, to get to the mp3… Because there’s some things that happen in this match loudness process that make it broadcast-worthy. It kind of pulls levels up inside the mp3 file to make the levels normalized and stuff like that, for production audio out in the world.

And so the wav file has chapters in it; it’s larger than the mp3. We then drag that wav file into match loudness, push Go essentially, or Run I think is the word for the process… And then a minute or two later, depending upon the machine you’re using it on, out comes this mp3. And so we upload the mp3 into our CMS, our website, our application, and then before that though, we also sort of drag and drop this wav file. Can you talk about what you do to introspect that wav file to pull out the chapters, to pull it into the CMS? And then some things that happen with the mp3 when it comes to like date changes, or title changes, or slug changes, how those things permeate into the final CDN that actually goes out to the world.

Right. So there’s really two sources of truth for any piece of information about an episode, and those two sources are the RSS feed - or feeds, in our case - in which the episode lives, and then the ID3 tags inside of the mp3, the final artifact. And those two sources of truth should be synchronized and match. And so the obvious place to do that work is in our admin, because our admin is where we author a lot of that information, including the title, the date published, the duration. We pull that out of the mp3. But all information is stored in our in our CMS.

And so the way that the chaptering works - and really the way that all metadata works - is every time you save that episode in our admin, it’s taking the result of the episode information that’s in our database, it’s rewriting a brand new mp3 that has that exact same match data. So it’s always the same.

The mp3 does not contain the chapter information until our admin contains a chapter information for that same reason. The chapters need to be outside of the mp3, because they also have to be in the RSS feed. We also use them on the website; we use them in multiple places. And so the data has to be outside of that mp3. And so the way that we get that is out of the wav file.

[01:04:15.02] So the mp3 gets uploaded into our admin; the wav file is huge in comparison. Mp3s range from 50 megs – well, Changelog News is like seven megabytes. You know, anywhere from 10 megabytes to 100 megabytes, sometimes bigger for Adam’s epic interviews… But the wav files - that’s raw audio, uncompressed, and so we’re talking gigabytes. So we don’t actually want to upload the wav file to our admin; we just want the information out of it. And so every episode post audition has four artifacts. Two wav files, two mp3 files. One for regular Changelog people, and then the “It’s better” folks get their own file.

It’s better!

That’s right. For Plus Plus. When you drag the wav file into our admin, it’s just dropping it into the local browser session; it’s not uploading it to the server. And the browser, via our good friend JavaScript, uses – I can’t remember what library we’re using. wave.js, or something… Something somebody else wrote to basically introspect the wav file, which can get at the markers, pull them out, match the timestamps, and basically create for us a bunch of chapter objects in the form. And that’s how that works.

So the wav file is then discarded. When you hit Save, the wav file is not going anywhere; it just disappears. The mp3 gets uploaded, along with the chapter information.

Yeah, the chapter information can live in the admin episode before the mp3 does.

Because it’s its own dataset, basically… Which informs the RSS feed…

That’s right. And you can go back and edit it later, and it will reflect into the mp3 and into the RSS feeds… And so that’s why I want that to be the source of truth, versus the wav file. The way file is kind of like a starter for your chapters, basically.

Right it’s like the… Starter’s a good word, I suppose. I was trying to come up with a different word… [laughter]

The kindling…

The initial vehicle to get it there, I don’t know; the transport layer.

Yeah, it’s just like the baseline dataset.

It’s actually the conduit between audition and web.

Yeah. And you could completely ignore it. You could author all your chapters by hand, in our admin, by hitting “Add chapter”, start time, end time… You know, you could do that, if you’re a fool. But we’re not fools.

Nah. And so the cool thing too is once the – to kind of go technically a couple layers into the details… The wav file does not get uploaded, as you mentioned. It infers and informs the admin of all the chapter data. And the cool thing I think of is afterwards if there’s a typo, which I will go and save, and then preview the web page… Because sometimes in Audition it’s not easy to see all those typos, because the interface of audition has small type. And I’m in my 40s, as you’ve alluded to you being in your 40s… It’s not that my vision is bad, it’s just, it’s small. And so i’ll see things differently when previewing it as the future episode page of this episode, for example. And I’ll notice that I fat-fingered something, or whatever may have happened, and I will change it in the admin, versus having to re-upload a new wav file.

Now, that does mean that the wav file is no longer the source of truth… Which is obvious, because it’s not meant to be. But the change happens in the web, not in the wav file… Which can take minutes, sometimes 15 minutes if you’re on a non-Apple silicon Mac…

Yeah. Too long.

…like I’ve learned. It could take 10 minutes sometimes to mix down from the session into the wav file. And so that could be too long; like running tests, just too long. Forget it.

Right.

Just do it in the browser.

[01:07:57.24] And sometimes you’ll catch a typo on an episode that you’re listening to a week and a half later, maybe Adam mastered it, so it’s not even on my machine, like Dropbox hasn’t synced it… I don’t wanna go sync his session down, remix it down etc, so I just go into our admin, make the change, and we’re good to go. So that’s really cool.

All of the mp3 chaptering abilities - shout-out to our friend, Lars Wickman, who we hired a couple of years ago to write an Elixir library which allows us to write ID3 v2.3 tags, which we couldn’t previously do with our FFmpeg-based solution - which is why we didn’t do chapters - as quickly as we wanted to. All of that you can find in old episodes; there’s a Kaizen with Lars, there’s also an episode that I thought was really fun, called “A guided tour through ID3 esoterica”, where we talk about all the cool stuff Lars learned along the way as he wrote that library, which we now maintain, although it’s had very few changes, because it works pretty much as advertised over the years. It allows us to embed images and links as well, super-cool… And you know, not to flex or anything, but I feel like we have the best chaptering game in the biz. That is a flex, isn’t it?

Flex as you like. I concur and I agree.

Our chapters are pretty on point.

I’m such a chapter snob now, man, really… I feel like anybody who’s an avid podcast listener, listening to podcasts that don’t have chapters, or haven’t appreciated or come to appreciate chapters in podcasts are missing out so deeply. And maybe it’s just because we’re professionals, and we quality-assurance our stuff, that I want to jump around more than the other listeners… Because there’s a lot of listeners like “I don’t jump around the podcast, so I have no need for it.”

They listen to it straight through, and then they move on to their next episode, and some other show. It’s fine.

Yeah. But I’m like “Nah, I kind of want to just jump to that one spot.” Especially when the chapters are so well named. So much care and love put into –

Well, for me that’s the fun part, is we get to inject a lot of fun… Really – I wouldn’t call them Easter eggs, but the fun is in the chapter names, and sometimes the chapter metadata, that I know seven people are going to notice, or less. But someone’s going to notice and get a giggle, hopefully. Or even just roll their eyes. But yeah, chaptering is something that we really take seriously, and put a lot into… And I guess don’t care so much that a lot of people get a lot out of it.

We’d love more people to. But it’s kind of like that thing with Steve Jobs and Jony Ive, where they were designing the inside of a thing, or the back… And they didn’t care if nobody else saw how cool it looked on the inside, because they knew; they knew how cool it looked. So that’s how we do chapters. It’s probably the most technical and interesting part of our flow. Everything else is relatively bog standard.

Yes. I think the chaptering is the is the bee’s knees, man. It’s the cat’s pajamas. It’s the good stuff. And I think our workflow affords us the ability to do that with such care. Because in the – you know, going back to when we marker… Taking the time in that post-production process took a bit to get used to, because it’s a time slot, and you’ve got to dedicate to it. You know, you may dedicate more, I may dedicate less… You may dedicate less, I may dedicate more. Who knows? But doing that pass is like the final, I would just say like chef’s kiss moment opportunity on a show… And maybe we take it too seriously, and others take it less seriously, like you just mentioned with Steve and [unintelligible 01:11:42.13] It’s like designing the inside of this thing that nobody gets to see, but they know how cool it looks. I feel like that’s the cool stuff for us.

There was a couple chapters… I can’t get to them, that I can think of… I don’t know. I was going to bring out one from a recent show, but it’s so contextually adjacent from this conversation it won’t really translate well. But if you go, dear listener, and you go through - even like LadyBird. That episode, episode 604, has 42 chapters.

That’s a lot.

[01:12:15.14] And the final chapter – 42 is an awesome number too, by the way.

That’s true.

The final chapter is “Cozy lofi from Katalin Kult”, because Andreas’es wife produces LoFi music, that you probably could listen to when you’re coding, on YouTube.

Good stuff, too. I’ve been listening to it.

Have you?

Mm-hm.

And that was his plug. And so we decided to put – I think it’s actually called “Cozy LoFi”, is the name of that track… Hence we put that in there as its own chapter, dedicated. You could just jump to that chapter right now and listen to it. To me, that’s the cool stuff. It’s the details in podcasts, and it really – and I would just say, Marco, if you’re listening, I love your software for the most part, but this latest update to Overcast… I am not super-happy with it. I want to take this moment to tell you that chapters are so cool, and the way chapters work now is not so cool in Overcast. I liked it so much better when it didn’t – when I clicked on a chapter, the page didn’t go away. And now you have to go back to it again, and find where you were at before you can use it as a jumper. Like a table of contents that did not move when you moved around.

The audio would move, because it’s audio… But the interface did not hide. The chapters are [unintelligible 01:13:29.10] and as soon as you click on one, it goes away. And then when you click on it again, it doesn’t even take you to where you’re at in the chapter list, it takes you to the top. So you’ve got to scroll, scroll, scroll to where you’re trying to be at. And so the experience of chapters has drastically changed, in my opinion… And I’m super sad. Please change it, Marco, if you’re listening.

A plea from one particular user…

And if not, we’ll just chapter this in the audio and link directly to this chapter and send it to you.

There you go. So one quick chaptering story before we move on, as we’re just stuck on chapters - this chapter will never end - is a recent JS Party episode is called “A Nick-level emergency.” So Nick Nisi, whom you may know from JS Party, and also from Changelog & Friends, and other Changeloggy goodness, is a big TypeScript fan. And the Node.js people recently added some TypeScript-related features, and Nick wanted to have an emergency pod about it. It was not worthy of an emergency, and so we did it anyways, we made fun of him along the way, and we called “A Nick-level emergency.” That’s JS Party number 333, which is a fun ride of itself. But then I got this idea for the chapters. Since this whole thing was Nick’s emergency, I’m going to have him referenced in every single chapter. And so there’s 22 chapters on that episode, and if you read them, every single chapter has the word Nick in it somewhere. Chapter three, “Note adds TypeScript stripping for Nick.” Chapter four, “Nick would absolutely love this.” Chapter five, “Nick, I’d rather be TypeScripting.” You get the point. That’s the kind of nerdery I’m talking about. Why not, right? Have some fun. Probably less than seven people realized this.

Less than seven percent, huh? You think so?

Well, how do you know? How do you know how many people will look at your chapters? We need chapter pixel-tracking technology ASAP.

Yeah… I do appreciate that level of detail that we can put into it, which I believe the hardest core of hardcore listeners do appreciate. Just imagine the experience of hearing about us for the second or third time. And maybe you’re not a full-time listener, but you’ve been linked up to an episode page. And you land there, and somebody says they talked about X, and pick your variable, right? They talked about Nick-level emergencies on this thing… And he was mentioning his desire for what’s new in ECMAScript 24. You can jump right to that chapter. And that chapter is also a linked chapter that goes out to an entire article on the new stack about it, because it’s probably what Nick did in the show notes, was like “Hey, this is what my reference point is.”

So they go right to minute 18 and 26 seconds, and listen to what’s new. So when they come there, they kind of get this version of instant gratification. Whereas podcasts generally require you to dedicate, in this case 51 minutes potentially, to hunt for the goodness. Chapters point you directly there. Now, I mentioned initially, chapter snob. That’s me. Even on YouTube, if I’m watching anything on YouTube, like a recipe… I’ve been like all in this chef stuff… If I find a video that’s like next-level chicken parmesan, and I want to like examine their recipe, I don’t need to know… Like, if I’ve experienced, I could – chapters let me bypass the things that are not for me, versus having to listen to the whole thing, or watch the whole thing, or whatever it might be. Like, if you are on YouTube or you’re podcasting, like we are, and you’re not chaptering, and you have anything that’s more than five minutes… Sad, man. Like, I will just bail on it. I’m just not going to give your content time, because you have not respected my time.

So here’s a feature for YouTube I’ve been thinking about… Because we do now allow YouTube to slurp in our audio episodes. And you can subscribe to JS Party, Go Time, what have you, on YouTube, via the playlist at youtube.com/changelog. So this is YouTube’s official way of adding audio podcasts as a feature to their platform. And they rehost, like Spotify does. But they do not respect chapters in your feed, or in your mp3, or anything like that. And so we do not have chapters on YouTube, whereas we did go out of our way and implement Spotify’s specific requirements for chapters on their platform. We do not have them on YouTube.

What I’m thinking about doing is creating a second feed for every one of our podcast propers, that includes the chapters as timestamps in the show notes, which is how YouTube works as a creator. The way you add chapters in YouTube is super-lame and manual. It’s probably actually a good solution for non-technical people, because you basically just put a section of your description, a list of timestamps with a title, and it will turn those into the chapters inside YouTube. That’s how it officially works. There’s no chaptering functionality in their YouTube studio. You just put it in the description and it turns those into chapters.

And so I might start writing for every one of our podcasts a second feed file, that’s just for YouTube, and point YouTube at that one… And everything is going to be completely identical, except where we put the chapters in the show notes as timestamps. And that would give us chapters on YouTube.

I don’t want to do that in our feeds generally, because it bloats them quite a bit, and our feeds are already pretty bloated due to being complete episode catalogs, versus like the last 100 episodes… So our feed files are already pretty large, but that’s something I’ve been thinking about doing for YouTube just to get the chapters information out to YouTube. However, there’s just not that many people listening on YouTube yet, or maybe ever, because we’re a non-video podcast.

A good episode – I think the most recent Changelog News outperformed, and it was only a couple days ago. 500 people listened to that on YouTube. Typically we get 100-ish per episode over there. But I think having chapters over there would be appreciated. So I’m thinking about doing that, haven’t done it yet. Now we’re basically kaizening, so maybe we should stop.

Well, there’s a little kaizen in all the conversations… You know, you’re always – part of kaizen is always be improving. That’s always, right?

That’s right.

[01:20:06.09] Improve without ceasing.

Alright, what’s left?

I would say what is left is just to thank everyone who is paying attention to this enough to care about this last “How we produce podcasts” section. Maybe you don’t care, maybe you do care; maybe you care about how we – maybe you’ve got some ideas and you’re like “Man, have you tried this?” Or “I like that workflow, but what about that?” Our stack is open source, and I would say open to contributions. There is a Zulip for that, or a Slack for that, potentially, where maybe you can hop in there, into a dev channel, and if you’ve got some ideas… Obviously, it’s open source; you can fork it and do what you want. But you can contribute, you can leverage the codebase to learn… I was going to ask you some questions about the RSS feed, because I think over the years you’ve iterated to what I would probably say is potentially the best RSS feed alive… Like, with all the necessary features; you’re really good at being on the tip of the RSS feature list, where what should be and could be supported is supported. You know, almost – I wouldn’t say instantaneously, but pretty close to instant, whenever it makes sense. And so I think RSS feeds are pretty solid, so much so that I actually pulled down the master feed and I have it opened up in Zed just to look at a raw RSS feed from the master feed, which is just humongous.

It’s like 12 megs. [unintelligible 01:21:27.22]

I’ll tell you. I curled it down, it is 11.7 megs.

Oh, [unintelligible 01:21:30.29]

And it took me three seconds to curl it down. That’s a pretty long time for an XML file.

12 megabytes.

I mean, it should be almost instant, right?

Not for 12 megs…

I suppose… I mean, an XML file is not usually that big, though. That’s a pretty big XML file.

It is. It’s got over a thousand episodes in there.

But it’s got a lot of cool stuff in there. So all that to say is our code base is open source. If you’re curious on how these things are implemented, look at that. Obviously, we are still in the process of potentially fully adopting Zulip, so there may be a place for you to have a conversation if you want to contribute, so you can hop in there and share some of your ideas, of course… Or just fork and PR away. There you go. But yeah, it’s been fun to talk about self-hosting, and the non-cool/cool rug pools that are reversed, and stuff… I’m excited about that stuff.

Alright. That’s all we’ve got for this week. Thanks for hanging out with us. Coming soon to a Friends near you, Kaizen… Coming soon to a Friends near you, Natalie Pistunovich, talking AI code editors… Coming soon to the Changelog, Elasticsearch… What else is coming soon on the Changelog? Oh, coming soon on the Changelog, Jimmy Miller talking about the best/worst codebase.

That should be good.

Yeah. Yeah. Looking forward to that one. So stay tuned right here, and if you’re a nerd and you like pretty things, go curl down our RSS files, and just appreciate that formatting, because it’s a lot of hard work put into those suckers, and nobody reads them but computers.

Yeah. Chapter data is in there… All the stuff. It’s a lot of stuff in there. Very cool. It is hard to appreciate that stuff. And I would say one more back in your list, because this came out after that, is ergonomic keyboards from ZSA. Cool stuff.

That’s right. Scroll up, hit Play.

That’s right. Or down. Depending upon where –

I’m looking forward to that episode, although I’m looking back at it as we publish…

The time travel of out of sync podcast recordings. Alright, let’s call it a show. See you all on the next one. Bye, friends.

Bye friends.

Outro: [01:23:56.22]

So Jerod, is there anything else we could talk about with this RSS feed? It’s – let me count the lines…

Let me count the ways…

88,724 lines is the length of the [01:25:29.21] on this file.

Yeah. Well, that’s our entire episode history in a single file… So that’s kind of interesting to think about. That is The Changelog, in a nutshell.

It literally is.

It is. Everything that we’ve done is in there.

Changelog

Our transcripts are open source on GitHub. Improvements are welcome. 💚

Player art
  0:00 / 0:00