Jerod Santo: Do you have any letters of recommendation you can provide? [laughter] No, this has been fun. Very interesting stuff. It was awesome talking. Thanks for sharing with us.

Nick Nisi: [58:23] Yeah. My recommendation, if anybody asks, which nobody ever does, but my recommendation is to go with Astro, until you’re absolutely positive that Astro can’t solve your problems.

Richard Moot: …too many cooks in the kitchen. Yeah. Just too many cooks in the kitchen, everybody wants to go in a different direction… And I think that really, just start doing it, even if – so even when we first started doing the podcast here, I think one thing that helped a lot… And I’m nearly positive this was Adam’s recommendation - we actually just interviewed, brought in somebody internally to run a kind of test episode or two… And I think it’s just, if you go and you do that, you can just build at least the tiny bit of confidence that you need that this can work. And you can hammer out all those little things that you might be worried about ahead of time.

The other thing that I would just say - it’s actually a lot easier than anybody might be making it out to be. You’ve just got to start, just sit down, have a conversation and record it. It might not be great the first time, but when you just sit down, you have the conversation, you record it, you can have a sense of what went well and what didn’t go well… And it’s weird that I’m saying this now, because we ran into this same exact thing when we launched our YouTube channel…

[00:42:05.16] When we first recorded episodes of the YouTube [channel], we overproduced way too much, and we realized “Okay, we can throw out half of the work here.” We were way overthinking it. And I think even before doing this podcast, we were way overthinking it. So the thing that I just can’t help but continually think is “We should have just listened to Adam and trusted him more about it that - just start it. The format will start to make sense.”

And I think the only part that I think I still got hung up on was what the content is going to look like. And I think in retrospect, you just need to start inviting a few guests, and start recording… Because the content will start to make more sense as you start making it. Because you might think “Oh, this is what I want the content to be about.” You start having the conversations and the interviews with people, and then you realize it keeps going in this other direction. And this other direction is actually not what I thought originally, but it’s working.

I always think of this analogy a lot of the time; I’ve said it to my team… I don’t know if it works well, but I always say “The map is not the territory.” So the map is an abstraction of where it is – so there’s a very big difference between reading on a map of where you’re going, and actually walking the path. And so in actually walking the path, you can see “Hey, there’s a tree falling here.” Or “Hey, there’s not a lot of cover through here.” Maps don’t tell you these things. They can kind of just tell you an idea of “Oh, I can get from here to here.” But it’s very different actually going and walking that path. And I think when you actually go and make the podcast, that it’s very different than when somebody tells you how to make a podcast.

Jerod Santo: Yes, I think you could call it like that. It is under the hood still an event-driven system, as Flink is, but it is built for completely different [unintelligible 00:25:16.02] Just to give you an example, the core of Flink is the way it is this exactly one stateful processing, so it basically has these data streams that it keeps moving, operations that do stateful stuff with the events, count, joined, and so on… And then there’s this asynchronous process running in the background that takes these consistent snapshots. So if something goes down, you can restore the state from the snapshot and just start the flow from there. And it has this clever way to do this in a way that maintains consistency across all the parallel machines, and it does that efficiently, incrementally, frequently, and so on. But it’s a very throughput-optimized thing. So it stays off the critical path, if you wish; it runs in the background. So yeah, it’s really good for throughput, but it does this persistence operation once every couple of seconds. If you tune it to be very frequent – most people actually run it more in the order of minutes. So when something goes down in a pipeline, you just replay the last minute of data, which typically doesn’t quite take a minute, because it replays faster than the original, the data rate at which the events are produced… But still, it takes you back a certain amount in time, which is usually okay for analytics. The worst thing that happens is, okay, this feature here in that vector that goes into that traffic model or that recommendation is maybe a few seconds older than it would have otherwise been. It’s not such a big deal, typically.

On the transactional processing side, imagine you have this multi-step process. You want a really fast checkout process, and you say “I do want to just start the next step after I know the payment has gone through. Before that, I’m not updating inventory. I’m not kicking off any of the other processes.” Then you really need actually a persistent step to be recorded, ideally in milliseconds. Maybe it’s not that critical for the processing, but we’re building this for even more low latency use cases, like payment processing and settlement and so on. And there, you really just want to kick off the next step after – you know, the previous step is persisted, possibly in a multi-data center replication way… And only then do I start the next thing. So you have to really design this completely differently. It’s completely optimized for low latency, transactional durability, rather than analytical throughput. So yeah, it’s a completely different design, even though both ultimately are event-driven architectures.

Adam Stacoviak: Well, let me make a recommendation on these pecans - because I say pecans - and then we’ll get into some home lab stuff, and maybe some AI home lab stuff… We’ll see, if we get a little crazy around here. The brand I want to recommend is a Texas brand. It’s a family-operated brand called Berdoll. B-E-R-D-O-L-L.

Adam Stacoviak: Really good people. Usually pretty knowledgeable staff. Even if they’re new, they’re pretty knowledgeable. And so I benchmarked this advice from ChatGPT against the advice I would get from, you know, the real deal. The real deal Holyfield, as they would say. And pegged it on the Michelin Defender LX M/S, which means mostly super-awesome. I don’t know. [laughter] So I went with a road-ish tire. The road-ish tire. Anyways… Great recommendation from LLMs, honestly, in my opinion.

Jerod Santo: Alright. We move onward and upward… Here’s Brett Cannon.

[00:43:13.26]

Hi, Adam and Jerod. Congratulations again on another banger of a year for the Changelog. For my highlights of 2024, to kind of breakdown the themes… Probably the first theme was hardware. Episode 608 for interviews, with “Building customizable ergonomic keyboards” with Erez Zuckerman of ZSA. I thought that was really cool, to hear their ethos and approach to making keyboards that last, and can last for a long time.

Interviews episode number 592, “From Sun to Oxide with Bryan Cantrill” was great just for the stories alone; also with what Oxide is trying to do with hardware. And then finally for hardware was interviews episode number 582, “We Have a Right to Repair” Kyle Wiens. I’ll also say that’s the most expensive episode for me personally, because it led to me buying an iFixit repair kit, and it has actually been very helpful. So thank you, Adam, for that recommendation.

The next theme is languages. No shock coming from me. Interviews episode number 611, “Free-threaded Python” with my friends Pablo and Łukasz from the Core.py podcast. It was obviously a lot of fun to hear someone else interview them for a change… And then also Changelog & Friends episode number 28, “Gradually Typing Elixir.” It was kind of cool to hear Jose talk about how Elixir is trying to bring in typing after having seen how Python tried to pull it off.

The third theme was operating systems. Actually, in Ship It episode number 122 with Linux Distros, with Jorge Castro, it was kind of cool to hear how Universal Blue is trying to use containers to make operating systems a bit easier to work with from a Linux perspective. And then it was great to hear “Let’s talk FreeBSD (finally)” from Changelog interviews number 574, with Allan Jude, because FreeBSD I don’t think gets enough play in the world.

Theme number four was apps, with Changelog & Friends episode 35 with Oban Pros. It was cool to hear Shannon & Parker Selbert talk about how they make Oban Pro work as a business. And also, personally, it was kind of a fun episode because it was the first time I was out for an extended walk with my son by myself, and trying to keep him calm with mom not around…

And then there was also “Why We Need Ladybird”, Changelog interviews number 604, with Andreas Kling and Chris Wanstrath, and how trying to make a browser is extremely hard.

And then finally, the fifth theme is people, and that was from Changelog interviews episode number 595, with Kelsey Hightower, talking about being retired but not tired, and just hearing Kelsey seemingly having a great time, no longer being constrained by the corporate world and getting to do what he truly wants to do.

Once again, congrats again for a wonderful 2024, and I look forward to 2025. Bye.

Matt Colyer: So today we have a very extensive API through GraphQL, that we expose. The data we just talked about, the categorization. So whatever the merchant decides - we make a recommendation, they say, “Yes, that’s correct. This dress is actually a dress.” Once they save that change, that information is then available through that product description API.

Jerod Santo: Storing times for human events

Simon Willison dishes out some hard-earned wisdom he acquired by working at Lanyrd & Eventbrite, two websites that have to deal with storing the time that an event is happening.

An event happens on a date, at a time. The precise details of that time are very important: if you tell people to show up to your event at 7pm and it turns out they should have arrived at 6pm they’ll miss an hour of the event!

Some of the worst bugs an events website can have are the ones that result in human beings traveling to a place at a time and finding that the event they came for is not happening at the time they expected.

So how do you store the time of an event?

Simon points out that the “best practice” of storing events in UTC breaks down for future events because timezones and locations and user errors and international political shenanigans. His suggestion:

My strong recommendation here is that the most important thing to record is the original user’s intent. If they said the event is happening at 6pm, store that! Make sure that when they go to edit their event later they see the same editable time that they entered when they first created it.

Justin Garrison: WebRings weren’t like a way to communicate between sites, but it was a way to like list out like sites you’re also reading. It was like a recommendation engine for people that had websites. They’re like “I have a blog. If you like my blog, you’ll probably like these other people’s blogs”, and they would link to other people. So it was like a discovery engine before things had discovery engines. It was all like personally curated, and then no one ever updated them.

Johnny Boursiquot: [laughs] Alright. Yeah, so before we transition to unpopular opinions, is there any parting words, a recommendation that you’d give to someone or perhaps a team that is looking to embark on a similar journey?

Autumn Nash: It’s like, you know when your friend always posts the best food pictures on Instagram, and then you ask them for a taco place recommendation, but you feel like you can trust them? It’s people in those communities making starter packs. So it’s not a corporation pushing it on you, or pushing the people that they hired to be in this community, but it’s like you’re – the other community member’s showcasing who they are interested in following, and I really like that aspect.

I think like Blue Sky is being very – the growth is from the community. It’s from people, and it’s getting back to the part that we really enjoyed about Twitter. Like, the people in the community that was there; that was the cool part of Twitter. It wasn’t Twitter, you know? So I think if we can get everybody in one place, this is what we’ve been looking for for so long.

Elham Tabassi: The first thing I will say is that you don’t need to implement all of the recommendations in the AI-RMF to have a complete risk management. So our recommendation is start by looking at and reading the AI-RMF. It’s not a very long document. I forgot, I think it’s about between 30 to 35 pages. So get kind of a holistic understanding of this. And then check out the playbook in the AI Resource Center, where for each of the recommendations - AI-RMF is in high level four functions. Each function is divided into categories, and then subcategories, so in sort of a granular approach. We give recommendations on what to do for the govern, and then for each of those recommendations get into a little bit more granular recommendations.

The playbook for each of the subcategories, which is about, I think, 70 subcategories in the AI-RMF, provides recommendations on suggested actions, and informative documents that you can go read and get more information, and also suggestions about transparency and documentations for implementation of that subcategory.

So we often suggest to get a better understanding of the AI-RMF, spend some time in the playbook to get a better understanding of the type of the things that can be done. And then, based on the use case, based on exactly what you want to do, start by simple, small number of recommendations in the AI-RMF, and start implementing that. Govern or map functions are useful starting points.

[00:38:09.08] Govern provides recommendations about the setup that you need for a successful risk management, so it can give you ideas or an organization’s ideas about the resources that are needed, the teams that need to do this, so they can align it with their own resources and the teams that they have. And the map functions, as we discussed, gives recommendations of a better understanding of the context, getting answers to what needs to be measured.

I will also add that the functions, govern, map, measure, manage - there is no order on doing that. It depends on the use case, it depends on what needs to be done. The starting point can be recommendations of any of the functions. We usually recommend to start with govern and map. And then start with as few number of the subcategories or recommendations that the resources and the expertise of the entity allows for their implementations. Of course, prioritize in terms of their own risk management.

And then the last thing I’ll also add is also be mindful that the risk management is not a one-time practice that we just do at once and you say “Okay, I’m done with my risk management.” AI systems - there’s data drift, model drift, these newer models can change based on the interactions with the users, with the environment… So we suggest a continual monitoring and risk management. I think one of the recommendations in the map or govern is to come up with a cadence of repeating the assessments of the risks.

So those would be my recommendations. Another thing that I would say is that - I mentioned AI RC, I mentioned the playbook… We also in the AI-RMF talk about profile. So I keep emphasizing the context of the use, and mentioning the importance of the context in AI system deployment, development, and the risk management… At the same time, AI-RMF by design is trying to be sector-agnostic and technology-agnostic. We try to kind of come up with the foundations, the common set of the practices that’s needed to be aware of, and are suggested for risk management. But we also have a section on AI profile, and recommendations on building verticals. These profiles are instantiations of the AI-RMF for a particular use case or domain of use, or technology domain, so that each of the subcategories can be slanted or be aligned with that use case. So there can be a profile of AI-RMF for the example that I used, medical image recognition. There you can imagine a profile of AI-RMF for financial sectors. That’s something that we have been asked to work with the community on.

That was a very long intro to say that there are a couple of profiles posted on the AI Resource Center. One is the one that the Department of Labor did for inclusive hiring. Another one that the Department of State did for human rights in AI… So that can give some sort of a window to, or idea about where the organizations can start.

In addition to the profile, we have also posted a few use cases, and we will post more use cases, and that is how different organizations are using AI-RMF that can hopefully be more practical examples of how to use AI-RMF.

Shay Banon: Yeah. I think details matter, as you said, and for people that are interested in it, it matters. I think the end result of having more open source in the world, it’s much easier to just all agree on. And if the arc of it is where we’re heading, I think it’s goodness. And to be honest, I actually think that – like, if you build enterprise software, that’s my recommendation. You can pick AGPL if you want to go open source, and not worry about the cloud stuff. As we discussed, I’m more worried about the large language models and AI area, which we need to figure out. That’s an area that we also need to invest in. And hopefully, this podcast also helps push it in the right direction, if that makes sense.

Chris Benson: Quick follow-up on that. It’s just something I wanted to draw out, because we’ve drawn it out across some other episodes a bit… You were just making a recommendation, kind of go for the smaller model versus the larger model. For people trying to follow - and there’s the divergent mindsets - could you take just a second and say why you would advocate for that, what the benefit, what the virtue is, in the context of everything else?

Pablo Galindo: [00:39:54.11] It depends. If they are paid to make sure that the thing is stable, the recommendation is no. Because - I mean, try it out? Sure. You can get very excited, by the way. One of the things that we have seen… Me myself, for instance, at the company I work, Bloomberg, we have tried to see what will happen. Even if it crashes once every three months, or whatever it is, but just to see a sense of what is improving here. Because if then, down the line, you’re going to justify any effort on your side to adapt your thing to the free-threaded version, it’s really good that you know what you’re going to gain. So that, for sure.

Trying - we really want people to try it, and tell us what they see, and what doesn’t work. Is this a production-ready workload? Absolutely not. Not because we know it’s going to crash or anything, but because we don’t know the real status of the thing. We take stability very, very seriously in the core team, so we are not comfortable right now. That’s why the thing is called experimental, because we are not comfortable right now recommending people – especially when you not only have the interpreter, but all these other things. Still, we need to figure out good APIs for those extensions to hook into the interpreter, and things like that, and that is not there right now.

So the chances that we can look at these things and say “Yeah, this can run in production, no problem” is almost zero right now. But as we release Python 3.14 and 3.15, we will move this experimental into supported, which means that you can run it in production, but it still will be two versions. And eventually, when we gain enough confidence that this is the future we all wanted, it will be only one Python, which will be Python without the free-threaded version. But until we call it stable, and supported - that’s the keyword, supported - I wouldn’t recommend people to actually use it in production environments.

Prashanth Rao: The stuff I said last year, I think I was still spot on, in the sense that I did talk about graphs and vectors last year too, and it’s still relevant today. So yeah, I’ll take my chances this time around. [laughs] So I think - yeah, you’re absolutely right. Things change way too fast. And my personal take here is that graph RAG is at a point in time, and we don’t even know in the next three to five-year timeframe whether RAG is going to be as hot as it is today. I don’t think LLMs are going away anywhere. Let’s face it… I mean, they’re here to stay. And they’re going to continue to evolve. We’ve just seen with the 01 model that came out from OpenAI they’re able to do reasoning… They’re able to do so much more than what people give them credit for.

So as this capability keeps evolving, and LLMs keep morphing into whatever else they become, there’s no guarantee that the kinds of tasks being done today using custom models, machine learning models, won’t be done by an LLM. Now, some of the wild takes that I’ve seen involve the fact that you don’t need to index anything in a database. You could potentially have your LLM’s parameters store the index in a very fuzzy way, and you could retrieve from that index, assuming that research goes in a certain way. But obviously, that’s a long way away. I don’t see that happening in the next few years.

What I’m particularly excited about in the next few years is everyone’s been talking about agentic systems. And if you look at the pivots that all these framework companies have had in the last year, Langchain and LLaMA Index, they’re heavily trying to push this field forward in terms of how agents can help build, I guess, more capable systems. Now, RAG is just one small subset of the things agents are orchestrating. They can actually do many other things related to recommendation, and say like search and retrieval, and a host of other activities.

So the conventional way or the standard way of doing agents has been using this framework called React, which is - you could say it’s sort of like a graph-based framework, but it’s sort of static, where you kind of decide the behavior of the system by programming in these paths, and the agent acts on it, and then you send it back after the reasoning step is over, and so on. But where I think actually graphs are going to be very interestingly used in the future - and I’ve seen some examples of frameworks that are already doing this - is can you use an underlying graph representation of action spaces to guide your agents? That is, can the agent actually actively update the action space via a graph structure? And once that happens, you can essentially have sort of more powerful agents, that are not constrained by a rigid, static sort of framework. You kind of have a dynamically updating agentic system, but it’s not as open-ended as having this recursive agent calling that we have today. You still have a framework. The graph kind of acts as like a base structure for the agent to take its actions. So that’s definitely one aspect of where agents are going.

But that being said, the field of graph, or you could say knowledge graphs, and their role in symbolic systems - they’ve been around. It’s been around for so long. And in the future, they could potentially be symbolic systems that utilize graphs in conjunction with these statistical models that are based on LLMs, and a hybrid sort of symbolic statistical system that combines these tools together. That may not technically be called an agent, but it’s still a hybrid sort of AI system, in the sense that you might have leveraged the power of an LLM for the language capabilities, but then used symbolic systems to do the other tasks.

All this is to say that I think graphs are way broader of an entity than what we are seeing today in terms of how they’re used. There’s a lot of other use cases that we couldn’t go into in this discussion. And Graph RAG obviously is a very, very small part of it.

So I would say I’m currently excited about Graph RAG, but I don’t see this being a topic that is going to be as talked about in the long term… But graphs as an entity and graph databases - these are systems that are going to actually be a core component of many systems of the future.

Jerod Santo: Plus, if you’re doing long-term thinking, the way you all are doing it builds value over the long run… Because the price of you all providing these standard plans for at this point 1,500 organizations which are community-focused, non-profits, open source, research, Academia, etc. these are people who will use and love your product, and it will help generate a network effect, you would hope, that would eventually bring their business to Zulip, their friend’s business when they go to ask them for a recommendation to Zulip, who becomes a paying customer. And that stuff doesn’t pay off in the quarterly or sometimes even the yearly, because you’re actually losing money by giving this away to more people… But on the measuring 10 years, 15 years, 20 years down the road, that stuff compounds and becomes massive. And it’s something that Slack I think currently has, to a certain extent, is some network effects, where it’s like, people already have a Slack app on their phone, and so it’s easy to add another Slack. In fact, yet another Slack is kind of a fatigue at this point. It’s like “Oh, I have so many Slacks. I don’t want to have another Slack.” But it’s a big advantage when it comes to getting people to use the tool if they’ve already used it, if they already have it on their phone or on their laptop. And so what you’re doing is you’re getting Zulip out there for these people, and you’re doing good at the same time. So I applaud that strategy.

Justin Garrison: Question four. This database is specifically designed to handle hierarchical or graph-like data structures, making it ideal for applications such as social networks, organizational charts, and recommendation engines.