Adam Stacoviak: When I’m poking around the UI, I see the option to add formulas and variables. Is that what you’re talking about when you say flowcharts, and stuff like that? That’s pretty interesting, because you’ve got different types, and integers, and data coming from different places that connect via nodes. Is that what you’re talking about?

Derek Collison: Yeah, so - I mean, NATS was built specifically to power those subsystems in Cloud Foundry. And I had no desire to do anything more than that. So it was, for at least probably the first 18 months of its lifetime, specifically geared toward just supporting what we needed in Cloud Foundry; from command and control, telemetry, eventing, things like that. When we started the company Apcera, which was trying to do a redo of Cloud Foundry on some of the things I felt I didn’t spend enough time and energy on - security, governance, federation, all these different types of things - we rewrote NATS in Go. NATS was originally written in Ruby. That’s what Cloud Foundry was written in. I still love the language… Not great to deploy things into production that are Ruby-based, although I know DHH has done an amazing job at trying to simplify that. But back in the day there was a lot of moving pieces to deploy the same…

Adam Stacoviak: Yeah. It has to be media, though. Content for - not a sales content. Content to show off who you are, to tell your story, to tell your customer story, to tell the bottleneck, the breakage story, the “It’s broken” story kind of thing. Not this “Let’s buy our thing, here’s how it works” only story. I feel like that’s where maybe folks will hear us talk about this and go and explore and examine themselves, and come back and say “Oh, we should do this. Let’s just sell our stuff on YouTube.” I think that’s not the way to do it. I think you need to talk about your world and your ecosystem, but not “Here’s how you buy our thing.” There may be a channel for that, but I feel like that’s a specific layer of the funnel that you address, and that’s more like sales content, literally. Can it live in YouTube? Sure. But I wouldn’t overly saturate that channel with two different types of content, because now you’ve got this “Let’s capture some people, let’s get some attention”, kind of distribution or become more exposed to certain folks in the world… And then you have “Hey, I want to buy your thing. Help me buy your thing.” That’s a whole different content slice, but do they belong on the same channel? I don’t know about that.

John Long: Oh, yeah, man. I’m deep on certain areas. I’m pretty proud of… So Zest comes in three flavors. You can have social, which is what you need for a website. And it’s got stuff like SaaS in there, and other things that you might want. But it’s got Instagram, and all the different types of logos that you want for a website. And those are pretty juicy. I haven’t seen – and I’m a nerd, but I haven’t seen a set that has as many, and feels current, and all that. So there’s that. And then there’s free, which is 200 free icons. And I actually tried to put my best icons in Zest Free. But when you’re ready to pay for it, you’ll get all the extensions of the best icons, and that’s what Zest Pro is. So there you go.

Adam Stacoviak: …you want it to use different services to enable that application developer to have the autonomy to deliver their application and not have to go through an intermediary… Which is really the DevOps world, right? You’re cutting that out, not so much because you don’t like it, but because you’re trying to serve a different sliver of the developer marketplace, so to speak. As you said before, was that the cloud is big, there’s room for all, basically, and you’re going to have AWS winning, you’re going to have Azure winning, you’re going to have GCP winning, but you’re not going to win – they’re winning in different areas. They’re winning different types of teams. Maybe larger enterprises that have uniquely different scenarios where they literally have to have a DevOps team because their business requires them to operate a different way. I don’t know, that’s kind of how I look at it.

One thing you said, actually, in your acquisition – sorry, not in your acquisition; in your announcement, I should say, of this $80 million funding that you just announced back in January… Congrats, by the way.

Jerod Santo: There’s kind of two angles into open source as an enterprise. You have “Let’s support, embrace, extend”, not extinguish, but extend, “the stuff that we currently are already using”, right? Like “We already depend on this thing. It’s core to our business. We’re going to actually put some money into it, or put some engineering effort into it.” And then the other one is like “Let’s take something that we have built, that’s internal and proprietary, and let’s open-source that.” And I think that there’s different concerns and different pushbacks to those two different types of embracing. On the actual “Let’s open-source some stuff front”, certainly there’s people inside of large enterprises that say “Yo, we can’t do that. This is our competitive advantage.” What do you say when people say “This is what we sell. Let’s not give it away”, for instance?

Elecia White: It came out of a – and to be fair, a lot of people started doing IMUs and Kalman filters in the early to mid ’90s. And it exploded as to the applications… But a bunch of PhDs were done at that time. And so they became very, very popular because they solved the inertial measurement unit problem pretty well. And now they solve a lot of other problems that involve a bunch of sensors.

Usually, when we talk about self-driving cars, the GPS has one set of errors, mostly; it’s really slow. The LiDAR or camera can have another set of errors, and it provides a different set of information, and the different types of Kalmans can put all of these together. And the idea is just you put the errors here and you put the data here, and you try to make the best out of the bad situation you have.

Chris Benson: So Jeff, as we kind of jump fully into the AI aspects here, could you describe - without giving away any secret sauce, obviously, anything like that, but kind of describe which AI technologies have been important to help get you go, and some of the things that you’re most interested in. Because we’ve been talking so much about generative AI, and large language models in recent years, and I’m sure that’s part of the mix… But one of the things that we’re starting to do at this point is kind of understand how people are using those, and also what other things they might be interested in that are not necessarily straight [unintelligible 00:37:46.03] another LLM. I’m curious if you can kind of share how you’ve built the business around different types of technologies a bit.

Angelica Hill: But all jokes aside – and like would give me a chance to be on a podcast… And I actively have learned over the… It’s made me a better software engineer, it’s made me a better gopher… Because on most of these episodes I’ve had the privilege of like learning actively in that moment from these brilliant people, that otherwise I might never have been able to interact with. But being able to say “Hey, I’m a host on Go Time. I want to learn about, I want to dig into X, Y, Z topic.” It has enabled me personally to interact with so many interesting, awesome people… But also given me so many other opportunities that I never would have otherwise got, by taking a chance on people. I think that’s the beauty of Go Time. We bring on so many different types of people… There’s not this high barrier to entry where you have to be some staff engineer, written a book… Like, we really do host episodes that are coming from all different walks of life, people, experiences… And we’re able to give kind – I can’t remember who said it, but give these people… I think it might have been you, Mat… Who might otherwise have not had a forum to be able to have their voices, their opinions heard. And we go, “Yeah, sounds good. You have something interesting to say. It doesn’t matter if you have one, ten, fifty, six months, one week of Go experience. If this is a topic you feel passionately about, let’s get you on. Let’s talk about it. Let’s expose people to all the different levels and experiences.”

And so personally I felt that, but also, I think that’s the beauty of Go Time, and the role that I feel like and I’ve heard it’s played in a lot of people’s lives, and their Go lives, is that it’s a place to feel heard, to feel listened to, to feel like you’re accepted, in a way that, bluntly, I think a lot of other software engineering areas - there is a higher barrier to entry. I certainly felt this. There are spaces you go into where you just don’t feel – you feel embarrassed if you’re not like a 10-year veteran of the language, and you know 17 languages.

Gerhard Lazu: So it feels like I never really left Changelog, because first of all, the Kaizen episodes, of all the infrastructure improvements that we are still driving, they’re still very much present… Trying all the things that we did over the years, and taking it to a place where it is now, and continuing the journey - that has been a long-term, very satisfying journey.

That’s the way I would put it. I’m very happy that that is continuing, and we’ve figured out a way to make that work with Adam and Jerod. So that is personally a very satisfying thing, and also professionally a very satisfying thing.

After Ship It, and the reason why for me, even like back then, when I was on the last episode, 90, “Embracing Change”, it was a priorities – I had to reshuffle a bunch of priorities, basically, and I had to give more time to my main job, which at the time was Dagger… And it still is at Dagger, even to this day. And I think that was one of the big changes that happened between me starting Ship It, and then having to part ways at that point.

[00:16:25.22] 2021 was a very interesting year, and it was not because of COVID, but that obviously did make it interesting for everybody… But for me personally, I was transitioning into a startup again. I went from a large enterprise at the time that was VMware… And at VMware, I had been working on RabbitMQ for, I think, six or seven years. And I went through like different types of teams, until eventually I ended up on the core RabbitMQ team… So you get all the Erlang, you get all the Make… And there’s a story there, because it connects to Dagger… And you just get to see a lot of really important systems, distributed systems, distributed systems problems… And you realize how important the kernel is, even when you’re not using containers. So little differences between the different kernels can have a huge impact on how something like the Erlang VM behaves. And these are really important applications, like things for banks, financial institutions, GPS trackers… And you may be thinking food deliveries, but there’s also some other GPS trackers, which are really important they work correctly. Tails, payment systems… It’s all over the place.

At some point - we didn’t realize this - some cars, the doors wouldn’t open. And RabbitMQ was in that stack. I was not expecting that. You would honestly not expect that.

Tom Occhino: [00:12:06.11] Yeah, I think it was sort of an implementation detail leaking out. I think we wanted to do one thing really well. But more importantly, we just had all this other stuff that this was sitting on top of. We had a way of doing data fetching at Facebook, we had a way of doing, you know, basically everything. Our router was like a server-based router… We really didn’t need to solve all of the concerns. But over time, what happened is the community ended up filling in all the gaps. We needed a router, and then React Router came around. React needed server-side rendering. And then I think Pinterest was maybe the first company to build a server-side renderer that would work. Because Facebook didn’t run JavaScript on the server at the time, and so we weren’t investing in server-side rendering. But the architecture was very much designed with server-side rendering in mind. So you needed all of these things, state management libraries and different types of integrations, and the community really filled in the gaps.

And then I think the goal for the core team over time was to make it so that all of these things that were built on top of React could get thinner and thinner. They didn’t have to be so sophisticated because the framework would take care of more of the atomic, low-level complexities for you, so that your framework that sits on top of this library that’s growing in size could be thinner and thinner, but just as expressive.

So yeah, I mean, there was no sort of intentional “Oh, let’s not worry about any of that, because the community will fill it in.” It was like “Well, that’s not what we need to solve, so we’ll kind of like create an opportunity for the world to fill in the gaps.” And that’s when I think, React, we introduced it very clearly as a library. Is this a library or a framework? It’s not a framework; it doesn’t do everything for you. Ember was batteries included, and that was beautiful. Angular was the same way. Here’s how you do testing. Here’s how you do data fetching. Here’s how you do routing. We were just like “Nah, this is just your views.”

And a lot of the early adoption of React actually used it in conjunction with like Backbone models. So there was even a strong relationship between – I mean, Instagram did this in the early days, where we had Backbone feeding into React Views, and it was just an easier way to do your view layer.

Hazel Weakly: Yeah. So the distinction there between the store bot implementation and the home bot abstraction is, if you think of an abstraction as something that’s useful for you, your team, your organization, your company to operate, then the abstraction is really for “How does this company get better at doing the company thing?”

So the most effective abstractions are always going to be so deeply specific to your understanding that they don’t necessarily make sense to anybody else. It’s not that other people couldn’t use them, but it really just does not necessarily make sense.

For example, if you work at a company that deals with a whole bunch of distributed databases, you’re probably going to develop a lot of really nuanced concepts about consistency models. And if you’re a company that just has a generic website and doesn’t do a whole lot of things with it, you probably don’t have a lot of abstractions and concepts and understanding around consistency models. They might have ones around like different types of environments, or A/B testing, or things that are useful for websites. And the implementation, the reason why that’s store bought, is because if you build the abstraction on top of your own implementation, you’re going to run into the not invented here syndrome kind of thing, and you’re going to isolate the entire company from the context of the rest of the development world so much… And it’s actually going to hurt you a lot more in the long term.

Imagine if everybody saw Kubernetes and then said “Oh, this is a great idea, but I’m just going to write it from scratch for my own, and build these nice little concepts on top of it.” You can build the abstractions, and you can build the concept and understanding, but if you rebuild Kubernetes, that’s just a waste of time. It’s not meaningfully helping you understand the problem better… It’s actually hindering you from understanding the problem better, because you can’t work with other people at the implementation level.

Yann Lechelle: And then by design, the open source team that is full-time on Scikit-learn and other open source libraries, means it’s a cost center to the company. So that cost center is by design, and we know that’s a cost we have to cover. So we will cover it through different types of activities… For instance - and this was something that was done in the past, where brands were sponsors. So either they hire someone that becomes a core developer, and they’re naturally sponsoring someone to build up this technology, or they were giving money as a donation to the research center. But now that the team is with us, we are translating this into a contractual sponsorship framework. So brands who want to contribute to Segatern and help us compensate for salaries will get something in return - exposure. And if they actually put more money into it, then we’ll have conversations around the roadmap. Find a way to make it converge in a win/win kind of way. Because Guillaume, for instance, can say “This brand wants us to do something, but it makes no sense for the community.” Then we won’t take their money for a sponsorship type of business. However, if companies want to pay us to do a certain type of paid for software, we’ll look at it. But that’s a different branch of the company.

So we’ve really clearly separated, and by design we know there’s a cost to it… And that cost is actually - if we are doing well, it’s compensated by the fact that we have done good by the brand. In other words, hopefully the community will actually resonate with what we’re doing, and so they’ll pay us back by actually appreciating what we’re doing, which will carry the message further.

So we think that there is a self-fulfilling prophecy if we actually keep adding value to the whole scheme, as opposed to removing value. And I will not name certain projects that have chosen a different way. But on the other hand, going back to the governance of the company, when a company flips and becomes VC-funded, or only VC-funded, VCs require a sort of return on investment that is too radical. And so that sort of forces a change of posture vis-a-vis the community and the licensing scheme. In our case, we’ve actually created a structure that is balanced in terms of shareholding groups, and so we will ultimately have - or that’s the goal of the structure and the architecture - as much money from public support than from private support. So it’s, again, sort of balanced.

Break: [00:35:19.19]

Kevin Ball: With the Ollama stuff I think you can run more and more things locally. And I think we’re moving in that direction. I’m actually really interested to see what happens with Apple, and Apple Intelligence, and those things… Because I’ve heard the first version kind of panned; I’ve heard it didn’t come out that well. But I think that model of like “Hey, we’re going to separate the different types of things we [unintelligible 00:49:38.09] we’re going to do local things, we’re going to provide utilities for that…” Plus they own their own hardware, which they can then tune to running these things… I think a couple generations down the road in terms of local development environments we’re going to be running most of these models locally.

Qi Xiao: Right, so not really, I would say. So I don’t think of a – I can’t really point to one feature in Elvish and say “It’s all Go’s fault that I have to write three times more code compared to implementing it in Rust.” There is not a single feature that’s like that. And I think it’s because Go is actually a very good language when you think about applications. It’s a very good language. It’s very geared towards implementing applications.

[00:43:57.00] I mean, Elvish is obviously an application. You can consume it as a library, but I don’t think of it as a library that much. I don’t worry about it as a library that much. However, there are things that I think does make it slightly more painful, more ugly in Go. Again, it goes back to - I don’t think Go is actually a very good language for implementing a library, because… I would mention two things. One is that Go doesn’t have true enums, and it doesn’t have true tagged unions. And if you’re thinking about implementing an application, you don’t really worry about that too much. But if you’re thinking about implementing a library, a lot of times the API can be much more cleanly modeled if you have true enums, or true tagged unions.

For instance Elvish’s parser - there are maybe 20 different types of nodes in the syntax tree, and it will be very handy if I actually have a Rust-style enum that [unintelligible 00:45:00.19] is one of those 20 different types of nodes. That actually can be quite handy. It makes some code a bit cleaner.

So even though I don’t think of – Elvish itself is not a library, it’s an application, but it’s made up of a small… It’s built as small modules, which themselves are kind of libraries. And internally, it does create a little bit of internal friction, but it doesn’t really show up on the overall application level.

Another example is Go doesn’t have keyword argument, default arguments… And the way people work around that is maybe like have a struct, and whenever you want two keyword arguments, you just make them fields of the struct. It is fine if you’re building a library that’s consumed by a lot of people, but if you’re just building a kind of internal library, an internal API, it does feel like this is a bit too much boilerplate. So it doesn’t make for the – Go doesn’t make for the cleanest API, is my take. So a lot of time I just keep looking, staring at a piece of code and think “I wish this looks a little bit cleaner. I wish this looks a little bit more beautiful.” But you have to accept the limit of the language.

Chris Benson: So that was very useful for me in terms of trying to frame and understand what you’re relaying here in terms of govern, map, measure, manage… And you talked about something a moment ago that was really interesting, in the sense of you have these characteristics that you’re trying to measure toward trustworthy, but it’s not just one, and it’s not just a black or white issue. You have a collection of them, and they vary across different types of use cases, it sounds like. So you kind of have characteristic profiles, in a sense. How do you think about – if you were out there as a consumer of the guidance that you’re providing from NIST, maybe in a small company that’s doing some work in AI, and you’re trying to implement the guidance from NIST, and you’re kind of evaluating your own profile of characteristics through that govern, map, measure, manage process. How does one frame that? If you’re kind of just getting into this and trying to implement the guidance, could you talk a little bit about how an organization that maybe had not done this before might go about implementing a particular, whatever their use case is, and – how do they get started in the process? What’s your recommendation there?

Chris Benson: I’m curious - again, selfish question - as you’re doing that and you have the local version and the remote version, the connection between the two there… What does that look like? Is it something that if they’re widely separated, if MotherDuck’s in the cloud and I’m out on a device that’s not cloud-based - is that efficient communication? How do you all handle those different types of use cases?

Shay Banon: Yes, and until you understand the complexity of going through the legal process, if that makes sense. There’s two aspects to it, I think. The first part is, if you go back to open source - and I’m relatively old school open source - a big part of building an open source software was registering a trademark and owning it. Red Hat did it, and others. And the first thing that I did even before I released Elasticsearch was take a loan and register Elasticsearch trademark. By the way, I don’t know if you know, but there’s many different types of trademark… But I actually registered two types of trademark: one for downloadable software and one for software as a service. So even back then, I wanted to make sure that when we build or someone builds Elasticsearch as a service, it doesn’t get confused.

And at least for a while, people respected trademarks. You know, Apache, and Red Hat… And that was a wonderful protection, because you could go and fork the project, you could go and do all of these things, but it’s just a different – from that point on, it’s a different project. I think cloud changed the game. And I think because Elasticsearch was so popular and Elasticsearch was one of the first services that Amazon decided to take and provide - I don’t know, I’d like to think the best of people, and afterwards try to figure out why not… I think they just took Elasticsearch and said “We’ll call it Amazon Elasticsearch.” Maybe they didn’t have experience with open source, they didn’t have experience with these types of things… You know, naively, some product manager takes a product, provides it as a service, and that’s the name. And I think that you can see that they learn after that, because when they took other open source projects, they called them differently. They call it ElastiCache, and they call it Amazon Managed Service for Kafka, versus Amazon Kafka, and things along those lines. But Elasticsearch was one of the first ones, and that name stuck. And that part, I don’t know, for whatever reason, Amazon just refused to change the name, regardless of how much we try to talk to them and try to process through it. And you’ll have to ask them why… Basically, it was kind of like the answer is “I’ll see you in court.” And people think, “Oh, you’re a big company, and you have lawyers, and you can pay for it, and go through the process” and whatever, but it’s exhausting. You have to give testimonials, and you have to do this… And it’s at least a few years of process. And maybe it gets resolved - and we ended up resolving the trademark infringement. By the way, thanks to the fork, because it made it really easy. But it just takes a long time, and it takes a lot of attention. And it never felt right to me. I want to be focused on building great products, not doing lawyer-y stuff, if that makes sense.

Adam Azzam: I stumbled into workflow orchestration by accident. It was a total necessity for a previous startup that I was working on… I was working on basically a career co-pilot or like a job search co-pilot in early 2023. And there we were using medium to large language models to basically help job seekers conduct job searches in the background, and then we would let them have conversations with jobs, and we would also be able to explain and deduce why they were good fits for particular roles that we matched them with. And what that meant from an engineering point of view is I had to take your routine callout to a large language model, and then now I had to do this millions of times a week. And when you do this at the scale of millions of times, for different types of pipelines, whether or not it’s extracting schemas from particular applicant tracking systems or job postings, whether it’s conducting a conversation with a job seeker, you just start seeing exactly how these things fail and break down at scale…

And so when I went to my brain trust and was like “Look, I’m having a hard time dealing with resiliency issues at scale. What should I use for something like this?” I heard the usual player of like Airflow. So I got that spun up, and it felt like I had to learn a completely new language… And when I got stuck debugging a particularly nasty Airflow job, I turned to a good friend of mine who was running the conversational AI team at Square, and he was like “Oh, cool. I’ll help you debug this”, and then he pip-installed Prefect.

And so we spent that entire afternoon just rewriting all my pipelines in Prefect, and that’s really what kind of – you know, you speak about sort of Broccoli AI… That is what turned a very well-meaning, well-designed consumer AI app into something that, from an engineering perspective, was durable and resilient. I don’t want to presume that everybody knows what workflow orchestration is. It was a thing that – I didn’t really know what it was until I needed it.

Kevin Ball: So, I mean, it’s a mess, obviously, and I think it’s done a lot of damage to the WordPress community. I think, honestly, it gets into the same thing that we’re a little worried about for Evan You, which is when you have a for-profit business, you have multiple different types of incentives. And it’s easy to be generous when times are good. So the business is growing, everybody’s welcome… “Sure, use the WordPress name. Use this, whatever. We’re still making plenty of business.” You get into tighter money, other challenges, maybe the company’s staring down layoffs or something because their finances aren’t looking good, and you start to say “Okay, what can we do here?” And sometimes that’s shift focus to a paying product if you don’t have a paying product… Sometimes that’s look at ways that you can scrape off some freeloaders… In this case, I think Matt views WP Engine as freeloaders. I’m not going to judge on the merits of the case, because that’s already all over the place… But that’s a target. I mean, maybe you do a layoff. This buyout offer is the most brand-happy way to do a layoff I’ve ever seen.

Jerod Santo: Yeah. I do think that’s the best model so far for the style of software that you’re building; of course, different types of open source I think lend themselves to different models.

Jerod Santo: Yeah. I do think that’s the best model so far for the style of software that you’re building; of course, different types of open source I think lend themselves to different models.

Daniel Whitenack: Yeah. I’ve taught a lot of workshops over the past couple of years, I’ve done trainings with our customers, I’ve built things with our customers, and always, when it gets down to vector databases, I’m like “Hey, anything I say, you can maybe learn some things from me, but if you really want to learn about vector databases and get the right sort of intuition, you really need to go to this series of blog posts on the Data Quarry, which is your blog, which was the first thing that I saw, which brought you originally on the show. We talked through vector databases, and trade-offs between different types of vector databases, and all of those things.

Since that time, of course, you’ve moved on to a company, a data company, working in a different type of data, which we’ll talk about today, but also is related to some of those things… So yeah, how has the year been? Lots of updates and fun things it sounds like you’re working on.

Chris Benson: Yeah. Nor do many of the authors that we’ve been talking about. So I guess my impression has been – it’s a little bit startling. The latency thing kicks in after you’ve been using the 4.0 model for a while. And it forces you to start realizing that there are definitely different use cases for using the 4.0 and the 4.01 preview. And I think that that is notable, because it’s really the first time that OpenAI has offered a new model that everyone didn’t just instantly switch to that as the thing to use.

It was kind of like - they said when we went from 4.0 to 4.0, “Well, there may be cases where you go to either one.” But in practice, I saw people just going to 4.0 pretty much nonstop at that point. Whereas this one, the latency kind of forces you to change your ways. And they also have given guidance on the prompt engineering, that the way you prompt the 01 preview is not the same anymore, because what they’re doing behind the scenes, and with the notion of possibly multiple concurrent addressing of your prompt in different ways, and they verify them against each other - all things that I’ve read that are unverified at this point in time - that because they’ve changed the way they’re processing on the back and the latency is now there, that there might be different types of things. They tend to highlight coding, they tend to highlight math and other critical reasoning skills where you’d want to go to the 01 preview, rather than back to the 4.0… And they’re claiming that it’s more accurate. I’ve seen numbers like 15% increase in complex reasoning tasks to support that. And there’s some drawbacks, which we can talk about in a moment.

But I think I am still trying to figure out the tasks that I’m going to assign to each model in my own mind. And I’ll find myself stumbling a little bit on that. I probably, at this point, over the next week, trying this stuff out when I’m coding - I don’t have a reason to do a lot of math in my day-to-day stuff, just on a constant basis… But for coding, I’ll probably be spending more time on the 01 preview, whereas I was using the 0.0 itself before that. So it’ll be interesting to see.

So with that difference, as you look back to the two between the 0.1 preview and just the 4.0 we’ve been using, I just want to throw out into the mix, at some point in the not too distant future, we’ve been led to believe that GPT-5, which is a much larger model, according to OpenAI, will be coming out. So it kind of leaves you wondering a little bit, which model that we’re on now is that going to be closer to? Are there two different sets of prompt engineering approaches that we have going forward at this point? There’s a lot of unanswered questions.