Alex Gallego: Yeah. You know, strategically, one of the best things that cloud has allowed us to do and explore is this idea of data sovereignty, which is this totally new idea for cloud products, that in my experience only started to become popular really in the last six months. And with cloud – okay, so part of the reason why this was late, just for a full context for the audience listening, is that we did launch a cloud, and we actually onboarded public companies in the cloud, so it actually did work, but it wasn’t the architecture that I had in mind. I was like “This is not the future of cloud.” And so we threw it away, and then we wrote a new cloud. [laughter] So maybe the context is that we did end up writing it, but we just wrote two clouds before we called it GA.

And so on the second version that we just launched, what it allows us to do is this idea of “Bring your own cloud.” Still fully managed. But the hard drives, the data that Redpanda writes - those hard drives belong in the users of VPC, but the control plane is in our VPC. Now, from a user, it’s still a fully-managed SaaS, just like Mongo Atlas, or something; you go in, you click-click, deploy a new cluster, in your region, in your cloud etc. That all happens exactly the same. But the user owns that data. And what’s interesting in that is that at least the computational model that we define is unified for both dedicated, and this idea of BYOC.

I think that privacy is easier to achieve the data sovereignty, where for some industries like insurance, or marketing, where if they add a new vendor, they have to send an email to like 10,000 customers. They’re “Hey, we’re adding a new vendor. These are the risks” etc. And so we de-risk all of that, because they get to run their code, Redpanda data plane, in their VPC, and we simply do all of the management. It’s all automated. And so that cloud was really what took us two years to build. It’s like, a new version that gives us the flexibility to either run the data in your cloud, or in our cloud. I think that’s probably why it took so long for us.

Alexis Richardson: Yes. For example, we wanna see great projects. I’m personally involved in projects like Flux and Argo now, which have joined together. Flagger… Weaveworks have other projects as well that are either in or hovering around the CNCF, like Cortex and Scope. Now, we want the CNCF to be a great place for fantastic open source projects, a really appealing place to put your project if you wanna be a real leader in the space, and we want to bring in the ones that we think are the best. Not necessarily individually the best; sometimes there’ll be several projects that collectively cover different aspects of one space [unintelligible 01:11:02.08]

Some people have said “I am confused. There are components. Where is my platform?” If we were to pick one stack or one platform, we would be forced to focus on just one use case. So instead what we’ve done is we’ve created an environment where vendors and big end users and systems integrators can choose how to assemble platforms and create economic value around that and create products. So we have a separation of project components and commercial products that the CNCF encourages.

Now, with other foundations, there’s been things where even naming any commercial interest at all in the room gets you sent out. And I used to work at VMware and we were responsible for Tomcat, and if we used Tomcat in the wrong way, we’d get letters from the Apache Foundation.

Then there was the Eclipse Foundation, which moved a long way towards the commercial interest, but unfortunately, you know what’s happened to that; it struggled a bit. I think with OpenStack one of the things that wasn’t quite right was they tried to create a single stack-like organism at the beginning, which meant that the more it grew, the harder it became to change. And it was only suited to one use case. And of course, if they got the use case wrong, because they thought it might be a public cloud, then it was private cloud, then it was an automated data center… Then the stack wouldn’t necessarily adjust with that. We don’t wanna make that mistake, so we want there to be great projects that you can build into platforms and products, but we see vendors doing that.

Daniel Whitenack: I think what you wanna be thinking about is - yeah, you have that period of experimentation and model development, and that may be local, that may be in the cloud, but ultimately, you’re gonna want to run an automated, non-interactive pipeline that takes in some amount of training data, trains your model using some framework (maybe TensorFlow or PyTorch), on a workstation or on a cluster (like a Kubernetes cluster) and then outputs a serialized version of that to some resource, like maybe an object store like S3, or something like that… And then this may be plugged into CI/CD in some way, where maybe I push new model training code to GitHub, and a CI/CD tool like Jenkins is listening on that GitHub repo, knows that my code has changed, automatically runs my pipeline to retrain my model. That model is retrained; as soon as it’s retrained, then there’s a call made to update the model that’s being used in maybe TensorFlow serving, or another serving framework like MXNet or Seldon or something like that, or maybe your own custom service that’s using that model, something that other software engineers have written, or something like that. Or it could even be a JavaScript app or a mobile app that is updated based on that.

I think the more pieces of that that you can automate, the better, in my experience… Assuming that you’re gonna be wanting to update those things fairly regularly.

Alex Suraci: It’s a good question, because there was already a lot of like CI/CD systems out there. And there’s even more every day, it seems like… But what really drove it was, we were trying to use Jenkins, and we were trying to use that to automate all of Cloud Foundry, which was like a massive pipeline. We were gluing together plugins and trying to keep that thing running, and it was kind of its own separate job, just keeping Jenkins in check. And meanwhile, we were building out this platform that was driven by like declarative YAML. You’d just say what you want, and then you’d tell it to go, the system figures it out… And the thing that we were using to drive that was very much the opposite. So I wanted to try and take what we learned from that and apply it to CI/CD.

[05:59] I think the main motivation was having just clarity in how the whole system works, and being able to trust it, and not worrying about if that VM gets struck by lightning and dies, we have to like spend another week just getting everything – like, clicking the right buttons, getting it up once again. So that’s what brought Concourse to the world, really… Myself and Chris Brown raging at Jenkins.

Johann Gyger: Yeah, I’m mainly working in the finance industries, and also public sector. I can tell that. And what I’m doing right now - I already mentioned it, for one company, which is in the finance sector. They have a public cloud strategy; they want to move everything into the cloud, from on-premises solutions. They’re’ facing a couple of challenges, of course, and they also have huge security requirements. And one thing is, as I already mentioned, the whole Salsa thing, where we try to rethink the whole supply chain security part. So that’s one of the major projects I’m working on right now.

[01:06:13.08] There’s also quite some TerraForm stuff involved in the public sector. So let’s say - I specialized in consulting customers that are trying to adopt public clouds, or cloud-native concepts. It’s more in the public cloud sector right now - how do you get into the public cloud safely? So I’m working in the public sector there as well, with like “How do we automate everything, like provisioning, cloud accounts, and so on?” What controls do we add? And there’s a plethora of solutions that you can choose from. Policy-as-code, for instance, is such a thing, which is really great. You can use OPA, you have different [unintelligible 01:06:56.09] languages, like Rego, or CUE… We haven’t leveraged everything of that yet, but we are starting to dig into those solutions, and they are pretty powerful. OPA, for instance - it helped us to add guardrails to our infrastructure pipeline, so that we don’t delete resources, for instance.

Zac Smith: So yeah, I was close to all that. And then just recruiting, that initial stage of not only the people who were willing to take a gamble on this very polar opposite thing called “Build an automated infrastructure company in the time of the public clouds who were dominating.” You had to really believe in this. So doing that, and getting the first few people going, and really built off of that… And then, like I said, doing that people analyzer, so you could build leadership really quickly. And so to answer your question - yes, I was involved. Usually, I would interview every person. Not interview, because we had a pretty strong value system that we had to read. We had to read our mission statement to you before you wanted – just to make sure, like, “We’re out here to build a better internet, and what that means for us is this. And if this is not how you feel about it, and these are your core values, then maybe this isn’t the right thing.” Because you know, often it’s really hard to interview people, and whatnot, so you’ve gotta align on values, and then of course, you can move into skill sets and whatever.

So having that done allowed us to be in a point – my job in the process was usually not to hire - I wasn’t out there recruiting, it was to get that alignment at the end of the process… Like, “Alright, do you want it? Do you believe in it? Do you follow the same core values that we do? If so, what questions can I answer?”

Steven Wu: If it does come, I think there are definitely – if you’re automating a query, for instance, there’s actually things. So there’s this notion of a thing called like an automated detection rule for like XDR type stuff, and basically, the idea is you have some query… So the way that Splunk or like Datadog or those guys implement it is that you have some query, and it runs every five minutes. And it’s a query like “Did this thing happen? Did someone change admin permissions on my AWS rules? Or did someone change - whatever? Did we get ingress from like weird IP addresses that we know to be compromised?”, stuff like that. And when those find something, they send off an alert, so it’s like the whole workflow… And those queries are queries that you run pretty much every five minutes, every minute, whatever… But they are also the same query that runs every five minutes. For that particular use case we actually have a bunch of these – we have like incremental caches where you can kind of… If you’re careful about how you select your aggregations, and how you select their mathematical properties, you can actually build like a cool little rollup tree where you – you’re effectively only running that query over your entire dataset once ever, you just do it incrementally as it comes in. Stuff like dashboards, same thing. You can kind of build this cool, incremental cache of everything, and then you just ask the cache for this range, and it tells you… And there’s no query cost involved.

So I think for stuff that isn’t ad hoc querying, stuff that isn’t freeform exploratory querying, which I think in that model that you’re talking about is probably most of that very high-volume automated, like, people doing this query every two minutes on a script type thing, that basically doesn’t cost us anything at all, because of that choice of caching. And it is true that all of it is trade-offs. I think we have made really strong, very specific, opinionated trade-offs. I think I said at the beginning, we have a very opinionated cloud architecture. But I think a lot of the gamble of doing this company such as it is is that we’re gambling on having made the right trade-offs. And I think that’s kind of true for any technical decision. There are trade-offs that you have to make for your problem domain, and you can get way better performance under certain circumstances by making those trade-offs, and not on others.

Deepak Prabhakara: Good question. There’s something that exists today. I think it’s a combination of things, because there is tooling today, but it’s abstracted away. You spoke about Kubernetes. There’s a lot of tooling around Kubernetes cloud native. Then you think about cloud - there’s a bunch of things around cloud security that you can plug in. But again, each layer kind of does specialize in what it does. At the end of the day, for you externally, you’re thinking about cloud security, you’re thinking about your product security, application security across your DevOps pipeline… And then post deployment, and all of those things.

[49:54] So you have today the tooling that’s fairly fragmented, and that’s why security is quite an exciting field today. It’s not yet reached a place where – like infrastructure was a decade ago, where a lot of exciting things were happening, but you didn’t have that holistic approach to it. And the question whether you can get that is also a big one, because security is very fragmented. Your responsibility in each layer is quite different.

So you get tooling in each place, and the bigger question that we are also addressing is “Can all of that be brought under – if not one single umbrella, but at least helping join those pieces together?” …which today is done quite manually on the back of compliance. You want to get a SOC 2 or an ISO 27001 - again, compliance is not really security, but for a lot of practical purposes, they kind of blend together. One doesn’t necessarily mean the other, but it’s kind of assumed to be… That’s why the larger companies will say “Do you have this?”, in which case we check all our boxes, and then we could carry on. Because today, there is no easy way for them to actually say “Does this company have true internal security?” So they kind of rely on some external signals to say “Okay, they think about it [unintelligible 00:51:08.28] and therefore we trust them with it.”

At a very high level there isn’t one tool that kind of unites everything, but you have enough tooling in each area of security you’re dealing with, which then has to be brought together. And that’s why that’s so hard for the CSO and his team today… Because they are not building things, but they’re defining security around it, and I think that that mismatch is what is causing a lot of the context gathering that is quite manual today… Which on the back of security as code should improve as you kind of go along, because as we saw, you can kind of expose a bunch of things to the other side, saying “Okay, now automated”, you can gather X, Y, Z and feed that into your compliance process.

Brigit Murtaugh: Yeah, I love that you brought that up too, because education is the space that we’ve really identified too that these could be super helpful in… Especially in some of the more cloud-powered or automated tools like Codespaces, or other ones. I don’t even have to like install anything locally, I can just go to the cloud and have everything that I need there.

[58:18] I think that’s an awesome opportunity in education, because we’ve gotten feedback from educators and students alike, that “Hey, at the beginning of a new semester I have to spend the first few days or weeks just getting everything I need for this class. And if I’m maybe studying computer science, I have a bunch of different dependencies that I need, across a bunch of different classes… So what if either my professor added a Dev Container, or us as a class, we had some Dev Container configuration that we could share?” So it’s like “Oh, don’t even worry about which version of Python do I need for this class or that class.” “Hey, no worries. It’s just in the Dev Container. Go grab it.”

David Flanagan: So one of the constraints that I set to myself is if I’m gonna move all my stuff, I wanna be able to do it properly this time. I wanna configure DNSSEC and really make sure that things are secure, and I set all my DKIM and SPF records for mail delivery. It’s very easy to just set some MX records and hope that it works. But actually, there’s a lot more that you need to do there. And for the domains where I don’t receive email, I wanna be able to actually set SPF records, so that it’s not used for spam.

I don’t think people realize how easy it is just to send email as any arbitrary domain on the internet. It’s not hard to send that email with sergey@google.com. Very easy. And these SPF records and DKIM records are how we combat spam, and other things. So I wanted to do it all properly, and that meant that I needed to set DNSSEC up; it’s actually quite tricky. It usually involves a manual step, because you have to have your domain registrar create a bunch of key – no, you have to first have your domain, set the name servers on it, then on your name servers set all of the key records, and then propagate that public key back up to the domain registrar. So there’s a chicken and egg problem almost… And I actually moved all of my domains from Dynadot, which was my previous registrar, to Gandi, because Gandi was the only API that supported patching the name servers without updating all the contact records and setting like a renew price guarantee… And I’m like, “No, I can handle that manually. I just want to automate the name servers.”

So yeah, I did a massive migration to Gandhi, which had a Terraform provider. I then bridged that to Pulumi, and I published – there’s now a Pulumi provider for Gandi that I published to the Pulumiverse. That allows me to patch all the name servers; I’m using Cloud DNS on GCP, which allows me to – it also has a Terraform provider, and we have a Pulumi native provider, so I’m using that to then create all of the DNSSEC records, and I then I’m patching that straight back up to Gandi. So I it all automated. It’s all public on github.com/rawkodeacademy. There’s a Rawkode Academy repository, a project directory, and then my DNS automation there. So I’m really happy about that, actually.

Tammer Saleh: If you’re going to be using something like Kubernetes, you need to invest doubly strongly in observability and in all of that metrics. But I’d argue that you need that just as much, if not more, if you are not using Kubernetes. If you’re trying to do raw AWS, for example, it’s even harder to build all that observability infrastructure in place. But it’s absolutely, if you’re just moving into the cloud world and moving into this whole type of world where automation and where – a cloudy world that that’s focused on automation, you need that observability, not only for your own ability to debug, but eventually, you’re going to feed that observability back into your automation, right? You’re going to do automated blue-green rollouts, where you want the automation, over the course of maybe a day, to look for errors, to look for reduced metrics, and to roll it back.

Guillermo Rauch: Everything that’s scary in a lot of cases ends up being liberating. I actually was listening to this great podcast of Gary Kasparov and Lex Fridman… And he lived a very interesting first-hand experience of what it’s like for AI to defeat what seemingly you had that was, first of all, unique in the entire world… Because if you’re rating as a chess player, but also you had what’s seemingly considered for a computer not ever be able to do. What he said in the podcast that I thought was really interesting is that “Well, that’s awesome that they beat us.” Okay, the computer is getting another 9 in the SLA of things that they can do better than humans. So now they’re a 99.9. And ten years pass and they get to 99,99. Here’s the deal - the remaining 0.001, which is human capability - now it just matters more. It’s not that because there’s fewer things in which you truly excel that your impact is diminished. If anything, it’s the opposite, because now you’re truly able to focus on your creativity. And I think ultimately, that’s what we’re seeing with frontend. Like, okay, now you don’t have to worry about the cloud at all. You might not even ever know what a server used to be or looked like. You might not even understand what the infrastructure is made up of. The other day I didn’t even know this anymore; nor maybe ever.

[01:13:59.05] But I’ll give you a really good example - S3, the CTO of Amazon said, is composed of 250 microservices. And I was like, “That’s insane.” There’s 250 independent services that are making up this greater service that I interface through two API calls, get and post. And patch maybe, sometimes. And it’s just astounding. I think that’s just ultimately gonna happen with – again, we’re continuing to like… Okay, now Vercel is helping at the other 9. Not only will you not know what file storage - how it actually works, now you’re not gonna know how a page is built and rendered in the cloud… But that’s awesome, because now your remaining 0.001% is how you express your brand online, how you engage with your customers, how you choose to communicate with them, what your values are as a person doing business, how you express your identity, and all these amazing things… And ultimately, your own product that you’re building, whether it’s hardware, whether it’s software… How you give that service, in what language it is, in what markets etc.

So I think it’s awesome, and I think we’re just gonna continue on that trajectory of more and more undifferentiated work being automated, and more power to you.

Katie Gamanji: I actually have many people asking me that… Some of my friends - they are actually developing products, startups, very small startup companies, and they’re asking “Is Kubernetes the right thing for me at the moment?” And usually, what I answer in that circumstance is gonna be probably no, because it’s just only two people, both of them developers, they don’t have any infrastructure developer or cloud platform engineer within their team… So for them, managing and completely deep-diving into the Kubernetes architecture and management maybe is not the answer.

So in that circumstance, I would usually maybe suggest a cloud provider. Again, you have free tiers you can use from different cloud providers… So I would usually recommend that. However, if you are in a circumstance where you have enough engineering resources and you have enough expertise of maybe understanding how to run an infrastructure - not necessarily Kubernetes, but the basics of what exactly an infrastructure is composed of, then probably Kubernetes is gonna be the answer moving forward. Because when we’re talking about Kubernetes, it’s not just about containers, it’s about how these containers are managed, and what kind of leverage you get in a production environment by using Kubernetes.

I’ve been mentioning the scheduling capabilities… For example, you have – maybe I should introduce Kubernetes very briefly. So Kubernetes is pretty much an orchestration platform for containers that is run across a distributed amount of machines. So you can have different instances, and all of them are gonna be put together to run your applications. Now, on which node, on which instance - that doesn’t really matter. That’s always gonna be abstracted by Kubernetes. So one of the capabilities is gonna be the scheduling.

So based on the requirements you have for your application - for example you can choose “This application should have this amount of memory and CPU at all time. This is the very minimum of the resources I need for it to be up and running.” The scheduler would take these requirements into account and place it on a specific node that will have all of these available resources.

Now, the thing is if that particular instance goes down, usually, if you’d be working in a data center, you will need to migrate the application, or you will pretty much need to trigger your load balancer to point to a different data center. Now, with Kubernetes all of this is automated. So Kubernetes will be managing or monitoring the state of your application all the time. It will say if it’s no longer up and running and it will go back to the scheduler, and this will put it on a different instance, again, with enough resources and capacity to run your application. So all of these operations are automated… And this is just one of the functionality it provides.

[35:56] We have scalability, we have resources which will allow you to scale an application based on different events. For example, if you reach the amount of – maybe the maximum of the CPU or memory your application can consume, you’ll be able to scale forever. But now you can actually scale on external events as well. For example, maybe there is a queue you have in a – actually, an SQS messaging queue in AWS would be able to take those metrics, and based on that, maybe scale further. It can really go the extra mile [unintelligible 00:36:27.04] personalized scale mechanism. And all of this, again, is automated. You have a declarative representation of your application, so your application is represented as code. It’s YAML, it’s not necessarily the most readable thing out there, but it’s out there, and that’s gonna be representing the desires that you want to have within a cluster… And that desired state is always gonna be fulfilled. You have control managers which will always make sure that what you want is actually gonna be in the cluster. And it’s always in a loop to reach that ideal condition that you defined within the manifest.

So these are just some of the functionalities. I didn’t even talk about the Ingress and how you actually manage the reachability to your application, how you have this abstraction across a collection of pods of services, and how can you have granular control of how your application serves different HTTP endpoints with Ingress, or how your Ingress can actually point traffic to different services and different applications. So you have a lot of availability out there, and you have custom resource definitions, and you really can go the extra mile. It’s a tool that is very customizable. But more importantly, again, it has some of the basics very well set, so you don’t have to think about them anymore; you just take advantage of them straight away.

So in that case, if you have a team that would like to run an application within a production environment, would like to kind of take advantage of all of these capabilities that Kubernetes provides, and they have enough resources within an organization to run it, then probably the answer is “Yes, do look into Kubernetes and start rolling it out.”

Johnny Boursiquot: I must say, when I first looked at this framework I was scratching my head and asking “Why would somebody couple that many concerns into one tool?” I’m starting to get that, based on the conversations we’re having. Thank you for opening my eyes to new possibilities.

To me, the most magic I want in any deployment, production-ready, push-it-and-it-goes kind of thing - I know I’m biased in saying this, but I’ve been doing this since before I joined Heroku. But Heroku is to me that magical thing where I just push my GitHub Repository or something like that, and my app is up there. And I don’t need to know how it got there, what they used under the hood, slugs, build packs, containers, I don’t care. It’s just “It gives me an endpoint? Great. I can even map a domain to that? Great.” That’s the most magic I’ve ever wanted.

So now here comes a tool like this, which is not only helping me be sort of developer-time productive, helping with that developer time productivity, but it’s going all the way out to deploy time. So to me, I’m like “Okay, this is innovation, yes. But am I ready, as somebody who wants to fiddle with the bits, am I ready to hand over that much control?” It’s the same dilemma you have every time you choose a piece of technology from a cloud provider. Yes, they might offer a raw tool, the raw sort of “Deploy this thing on an EC2 instance and you’re good to go”, or you pay a little more, but you get the managed service and it’s hands-off.

So the more control you want, the more you have to shy away from the managed thing. So to me, this felt like magic [unintelligible 00:34:53.14] And André, that’s something you were deliberate about - making this as automated as possible. Who hurt you? What drove you to that extreme? [laughter]

Terri Jordan: [16:09] Yeah, this is what gets me really excited, so I’d like to go first on this one, too. So food and beverage I’m seeing increasingly. When you think about raw material quality verification, think about all the places grain goes, or corn, different kinds of corn. So really having quality corn. We have one use case where we’re looking at where we can differentiate between good corn, musty corn and mildewy corn, or sour corn.

So looking at raw material supply, and raw material quality verification. That goes into not just food, but also the entire fragrance industry. As you can probably imagine, we’re working with a very large fragrance supplier that supplies to all of your consumer product goods, and all of the hand sanitizers that we’ve been using so much of in the past year, as well as in fine fragrance. So all of that requires really fine raw material, but also product consistency on the other end, so process control. So did I switch out a batch of orange blossom for vanilla? That’s an extreme case. Did I bring this out of the right vat, and does this look like it’s supposed to look?

Those are some of the obvious ones, and we term those kind of things as cost of quality. So if you’re off-quality, you’re costing a lot for a manufacturer of any of those, whether it’s food, flavor or fragrance.

But some other ones that I think are interesting, that are coming up, that were surprising at least to me when I started is in the automotive industry. So in the future, and really to a certain extent even in the present, we’re seeing more autonomous, and lots of ride-share. So ride-sharing has become more or less ubiquitous, in some form or another, whether you’re talking about a fleet of Hertz/Avis type of thing. Or you’re talking about the Zipcar kinds of things, where you’re actually literally sharing a same car with somebody else and it doesn’t have a place to go after.

So what the automotive industry is finding is that they need a way to take that subjective experience of getting into a car and it smells bad, or it smells like new plastic, or it smells like somebody’s had their pet in here… And we need to automate that. Especially when you think and you go forward into mobility and the autonomous space, there’s not gonna be a place – you know, that subjective thing; you don’t want to use your customers and clients as your sensor, as your detector. You wanna sense that beforehand.

So both for safety and preventative maintenance, but especially for the comfort and experience of driving, all of the automotive tiers are looking at ways to standardize and create sensors that can speak back to the cloud and go back to wherever the controls are for that experience. So that’s been really fun. We’ve smelled all kinds of bad odors in that case, so we could share stories about that. That’s been fun.

And then consumer appliance industry as well. You know, food spoilages and food conservation has become kind of a mega-trend in a way… So food storage monitoring, whether it is on the large-scale supply chain of shipping food, or your own refrigerator… We’re looking at applications where you could imagine in the future it’s saying to your smartphone or Alexa “Hey, am I out of cucumbers? I’m at the store. What do I have?” And be able to really pick up the odors of what’s in your refrigerator and be able to dissect that. Or you might get an alert that you should be using your meat within the next couple of days.

So detecting odors in the refrigerator for a consumer, as well as in somebody’s larger supply chain ones. And yeah, even automated cooking. We did a use case where you could tell when the chicken is done, when the cake is done… Those kinds of applications as well.

[20:01] So there’s a lot of spin-outs from that, including lots of personal care and cosmetic examples… A little bit more, slightly more in the future is more health examples, where you can imagine wearing a watch and breathing into it and having it tell you “Yeah, you should go maybe get checked for strep throat” or something like that. Certain diseases have - particularly diabetes and whatnot - have by-products that end up in our breath or in our skin, so in terms of our roadmap, that’s in our roadmap as well. But the first few examples I gave you are things we’re working on right now, with [unintelligible 00:20:33.26] early adopters in the industry.

Adam Jacob: Yeah, for sure. About 13 years ago I ran a consulting company that did automated infrastructure; I am a systems administrator in real life, that’s my actual thing… And I had spent a lot of time doing automation, and building automation for a string of different companies. Eventually we started a consulting firm, me and some friends. We went out and our consulting firm was doing great; we were building a lot of automated infrastructure for people, but we were struggling to maintain it after we built it. So that led us to build Chef. We wanted a better way to manage bigger infrastructures, spread across more people.

We hooked up with a guy named Jesse Robbins. Jesse of Velocity, fame and former Amazon.com master of disaster. It was Jesse who was like “Hey, this should be a company. We should actually take this thing and build a new product, raise a bunch of capital, and go for it.” So we did, and it really super-worked. We built this product called Chef that did configuration management. You describe how you want a service to work when you run it; it sees if they work the way you described, and if it doesn’t, it figures out how to fix them.

[08:12] Our original business model was that we were gonna open source Chef itself, and then we were gonna run all of the server-side infrastructure as a SaaS. This was right at the peak of SaaS hype; this was when SaaS was taking over the world, and that was the business model everybody was gonna choose. It turned out that was not the best business model for enterprise software; they weren’t quite ready to let their configuration information go out to a random person’s SaaS. But the open source part of what we did - we open sourced it under the Apache license, on purpose, knowing that we were sort of opening ourselves up to competition, but that it would garner the biggest possible spread of adoption for Chef… And that’s super-worked.

We had companies like Engine Yard, BrightScale… There’s a pretty long list of those early cloud providers and the early people who built on top of those cloud providers, all of whom built stuff on top of Chef because of how the license was constructed and because of the way that we built that community, in a pretty open and collaborative way.

As we grew, that changed into selling on-prem software. We went from being “It’s all open source and you run it as a SaaS” to “It’s mostly open source, but we sell proprietary software, too”, so we became open core. And we were always what I would call a pretty loose open core, versus tight open core. The difference there would be tight open core is like there’s core features that exist in the product that you don’t get unless you pay me money.

If you wanna back-up the database - that would be tight open core. Things that everybody needs. Loose open core means you build stuff around the core software, and then I monetize that. So if you wanted to buy a web UI, or you wanted rule-based access control in the server… Some of those things maybe you do. But it was disconnected from the core value prop; you could still just use the core of the software to do whatever you wanted, without having to do a lot of work.

We stuck with that model. We still had a SaaS, we still sold on-prem software, but sort of going back and forth through that and managing both the community, managing our business, what features are in, which features are out, why do people buy… And then most recently we took a long look at our business and why people were buying the software, and it felt like we were sort of on this treadmill of constantly putting features either into the commercial product or out, but ultimately people were buying the product because the product worked. The product itself, Chef, and the thing it did for them - that’s why they bought it.

After a lot of soul-searching and looking at the business model, we decided that what we were gonna do is open source everything. Rather than have any proprietary software at all, we were gonna follow essentially Redhat’s model, where we were gonna open source all of the software, and what we do is build a distribution called Chef. And if you want that distribution called Chef, you have to buy it from me. And what you’re paying for is both the means of that production, my support, you’re paying for the marketing, you’re paying for all of those things that surround it. But anybody is free to take that software and build a distribution of it, and to distribute it under whatever terms they want to. They just can’t call it Chef.

That’s the most recent evolution of Chef’s open source business model, and that’s been working great. They’re having the best quarters they’ve ever had, we have customers who weren’t paying us, who are now paying us, because of that change… So it’s been working pretty well.

From a community point of view it’s also been working well. Before this change no one knew how to build a release of Chef, but Chef. If you didn’t work at Chef, this wasn’t a thing you could do. Now there’s a relatively robust community of people who are building new releases of CINC, which is the community distribution of Chef. They’re hosting it at the OSU Open Source Lab. There’s a much richer ecosystem for the software than existed before we’ve made that choice. So that sort of brings you up to date, I think.

Colin Billings: Yes, and it actually shows you that feed of what’s going on on your doorbell, and it has a small interface for push-to-talk, or unlock the door if you have an automated door lock of some kind… You really start to pull the necessary pieces of those controls and interactions off of that.

Or say you wanna drop in from your bedroom to the kitchen to ask when breakfast is gonna be ready. Microphone, speaker - you can just intercom from one Orro switch to the other. There are really all the ingredients you need for a much more capable home, improved by the fact that it understands how you use your house. I think that’s where over the long-term we’re starting to connect not just the things that you do manually, but the things that you do regularly based on patterns, and have them actually happen for you… And then provide that layer that is the actual manual controls via the screen, or otherwise.

[36:20] The one thing to point out that I think hopefully is one thing that we’re interested in advocating for through our products and the ways that we go is that things like smart things and a lot of these other connected home platforms - they’re actually cloud architectures. So you have really naive controls or sensors that live in your home, and they require talking to the cloud to figure out “Hey, what does this mean?”

Kelsey Hightower: I think that’s a good question, and if you think about who’s building these systems, they’re mainly developers these days. I would like to see more ops people, and we do see them building features around it, but I think you’re thinking about this world where the developers are now spending a lot more time in operations, building operations-specific tools. Stepping back, it’s just all software. This software just happens to deploy other software. It’s kind of like writing your own unit tests, right? You’re writing software to test your application, so in this case it’s software to manage applications, but it’s just all software.

As a developer, you may be less interested in deploying and managing a Kubernetes cluster. There are different roles on the team for that, and if you’re in a position where you have a dedicated group of people who are using a hosted system, that’s great. As a developer, what you kind of care about is “I have this application, an application running on my local machine, and Docker’s a fantastic piece of software that lets you take a Linux machine and abstract away the needs of SSH, systemd unit files, logging - all of that kind of goes away, but it’s still there underneath the covers.” Docker sits on top of a single machine and gives you this really nice API, you’re packaging the app, push it through a repository and running it anywhere that you find a Docker. This is perfect.

But in production, you have a lot more concerns than just starting and stopping an application. Who’s going to collect the logs and push them to a central place? How do you express the need of “I want this application to run across multiple machines, multiple data centers.” That particular set of requirements needs a higher level tool or language in order to express and enforce to make it happen. A lot of those concerns - you start to get into this idea of clustering. I have multiple machines, and the machines that we’re using these days, these are mainframes; they’re not built to last forever. So we’re dealing with these machines that are going to fail. If you’re in a cloud provider, the VMs are ephemeral, so you have to plan that the machines that you’re running on can be blown away at any given time. They can die, like migration, so you probably want a system that can account for that.

[00:27:54.03\] From history in general, the core principle, even if you’re not using Kubernetes, your team is going to have to build something very similar. How do I decide which machine my application runs on? Well, if you have a DevOps team and depending on how they do things, they may be recording that decision in a spreadsheet, in a tool like Ansible where you say, “This app runs the database. This server runs my web application.” That would be manual scheduling, right? That concept of scheduling, as a human you would say “Well, we know that’s the database server, because it has this name or it has this storage.” That act of scheduling in Kubernetes is an automated process where you as a developer can craft one of these manifests where there’s a deployment manifest. You can say, “My application needs one CPU and let’s say 16 megs of RAM. That’s enough to handle this many requests per second. If you want to scale above that, then give me more of those instances running and then we can actually scale horizontally.” So as a developer, there’s that dev test cycle, which is great for a single machine, but when it goes to production, as a developer you need a new set of concepts to express “I need five copies of these applications running, with these resource requirements. And oh, I’d also like to expose these to our customers over this particular port, not that port.”

So if you think about it, Kubernetes takes a DevOps team and rolls it into a system, and in return gives you the developer an API that you can use to express what your application needs to run in production.

Jerod Santo: It’s now time for Sponsored News!

New Temporal capabilities from Replay 2025

The biggest event of the year for Temporal just wrapped up. Replay 2025 in London is where teams learned about evolving legacy systems, modernizing infrastructure, and building scalable, durable software to power today’s top businesses. Here’s what they announced…

Temporal is open source so you can easily self-host it, but many teams are operating at scale and they rely on Temporal Cloud to give them an easier, faster, more scalable way to run Temporal.

• You can now automate migrations with zero downtime. The new tooling is in Pre-release. They’re seeking early users.
• High Availability can now offer another “9” to Multi-region Replication and Same-region Replication allowing you to asynchronously replicate your Workflows to a Namespace in a secondary or same region and will automatically fail over if necessary, to keep your applications online with a 99.99 SLA.

They also released new and updated capabilities to improve your development experience with Temporal:

• Temporal Ruby SDK is now available in Pre-release and it’s at full feature parity with the other Temporal SDKs. You can now use Ruby to write workflows and activities.
• Temporal Nexus is now Generally Available. Nexus lets you connect Temporal Applications across (and within) isolated Namespaces.

Scott Guthrie: [00:32:32.14] Yeah, I think as AI gets richer, you’re gonna see a lot of these scenarios become more like an agent. To your point, instead of the dev having to click 8 things, or type five things in four different files, you can sort of effectively ask the AI “Change this number in my website to be highlighted more”, and that might then update your CSS, it might update the text, it might update the HTML. And again, what’s nice with GitHub is that you have source control, you have diffing tools. It’s very natural to be able to see “Okay, what were all the changes that just happened?” and as a developer, you can review it, you can revert it, you can make different changes, you can commit it. So I think that that model with GitHub and source control works super-well with these types of kind of agent-based activities.

I think the other thing - and we showed this a little bit in one of the demos today - is you’re also going to start to see the paradigm for AI shift from being synchronous, meaning you type something into whether it’s ChatGPT, or Copilot, get a response, to more asynchronous, where you can sort of say “Okay, I want you to go work on a problem, ad get back to me.” And that “Get back to me” might be immediately, but it might also be five minutes from now, or it could be tomorrow. And as we start to think about activities that we do as developers, you know, “Hey, cost-optimize this”, or “Come up with recommendations for how I can cost-optimize this app that’s running in my cloud.” It might need to look at the usage for a day or two. It might need to go examine a bill, it might need to hit a couple of different systems.

Us being comfortable about having an agent that can go off and do that, and then get back to us, starts to open up a lot of activities and a lot of scenarios that I think will really change how we work. And ultimately, if we’re successful, it will give every end user back a whole bunch of time. Because if you think back to what you did in the last day, and wrote down – what you did minute by minute, and looked at that log, there’s an awful lot of busy work that we all have to do in our lives, that hopefully AI will help automate more of, and allow us to actually focus on the things that we enjoy, and ultimately give us productivity that improves our lives, and the businesses we work for.

Autumn Nash: How do you build the trust for them to take that PR though? Because also just - I mean, giving the control away to automate things by other people… You know what I mean? Like, when you’ve been running the Bash scripts, and you’ve been running your infrastructure a certain way… Especially if you’ve been running on AWS, and now you’re learning how to run hybrid cloud. Or you’ve been running with TerraForm or Ansible, and you’ve done all this automation, and now you’re kind of trusting people… I feel like it’s hard, even in open source, where there’s a governing body of some sorts - like, you’ve got maintainers - to get people to trust the next upgrade, or the next change, or the next, you know…

V Körbes: Yeah, so in the beginning I was just thinking about languages, and I kind of forgot everything I worked on in the past few years… Because you know, bad memory, I had too much coffee today… But then y’all started talking about repl and whatever the hell that is; I’m too old for those new acronyms… So anyway, that reminded me that I used to work at developer experience stuff, which I had completely forgotten, for whatever reason… And a lot of the stuff that I used to give talks about was basically how it is important to have a fast feedback loop; and the language in the end doesn’t really matter. What matters is like you need to have a pre-prototype setup that works in your environment. So let’s say “What does hello world look like for your setup?” And then you need to have a system that’s basically watching your code and rebuilding and like refreshing, and doing that faster than you can think about it.

So I was working [00:10:14.27] before, the company has been bought by Docker; I don’t know if anybody’s still working on it, probably not… But that’s something that we used to do, was just get that feedback loop super-fast. And basically, you would write a line of code, press Save, and then your changes are already there and already live. And I think to me that was a lot more important to make developers happy with fast iteration than the language itself; because the setup problem is always going to come up anyway. Sometimes you have Kubernetes, or sometimes you have, I don’t know, Cloud Foundry, or whatever the hell you’re using, and you need to be able to automate that beforehand. And then you need to integrate that with your systems, which also needs to be a thing beforehand… And when you get to actual prototyping, I do think that fast iteration velocity is the biggest defining factor to make it less frustrating to prototype things. This was supposed to be polemic; you guys are supposed to say “No, shut up. Go is good. Everything else is bad. Go away. You’re wearing a Java people’s shirt, shup up.” [laughter]

Abdel Sghiouar: The second one. Do we need it? No. Not everybody needs to be on Kubernetes, the same way as not everybody needs to be on cloud, and not everybody needs to do machine learning. There are valid cases for using Kubernetes, and as well there are valid cases for using serverless. . The part where I said earlier I’m biased - I’m biased because for me Kubernetes is not complex, because I understand it. And I think that Kubernetes as a platform is whatever you want it to be. It’s for me three main things. It’s an abstraction layer on top of infrastructure. So if you have delved like myself with infrastructure before, where you had to write Chef and Puppet recipes, and Bash scripts to automate things, then Kubernetes is pretty nice in that sense, because it simplifies managing infrastructure.

But then you can also look at it as a platform, as a layer that you don’t have to really understand how it works under the hood. It has an API, and it’s actually very funny that a lot of people that says Kubernetes is complex are developers who are supposed to be in love with APIs. Because if there is one thing that developers like is APIs and programming languages.

Then the third part, which I think we don’t talk about very often, is the extensibility aspect. So Kubernetes API is extremely extensible, and you can build your own things on top of it, and you can automate your own little things, and there are companies that are making business and pumping up billions just from this. So I think what I’m trying to say as a TL;DR is it really depends on what do we mean by complexity, and it really depends on what you’re trying to do. And I just need to be very honest here, I have a background in consulting, so the answers to my questions are always going to be “It depends.”

Sebastian Kister: Yeah. I’ve been creating the partnership with the Cloud Native Computing Foundation in 2019… And also, I really learned and liked a lot how the CNCF community works, thinks, and builds, and creates. We also have in Audi something that we call inner source, where you can contribute to certain projects, eliminating bottlenecks, actually… Because the backlog doesn’t reflect your necessity in the project, but you can do it yourself, right? You can solve that issue yourself, contribute it, and we leverage it as a core maintainer for everyone. Something like this. And the same happens in the cloud-native community.

So everybody has a slightly different approach to some things, or some different priorities… But you can create that. You can solve that issue. You can solve your number one priority, which is maybe number ten for the entire community… But you can solve it and you can contribute it; then it’s there, and it’s leveraged. And you profit from leveraging project teams that are concentrating on the other ten priorities, or other nine priorities.

I must admit that working with the community and the technologies has proven to be the best business case out there. And it needs to be a business case. You can’t be sustainable without making it a business; you can’t be idealistic without making it a business. It needs to be a business in the end; it needs to be a business case. Otherwise, change doesn’t work. Nobody changes anything, unless it’s a f***in’ business. And that is something that I really saw happening, actually with cloud-native technology taking off in the enterprises, because it is a business case. And it’s an easy one; it’s a very easy one. You have two develop in your company contributing to that, and the other 4,998 developers - they do something, too. So you can maintain your fork for the next two years with your two guys, until the first breaking change happens, and then you’re done. Your project is done. You can never touch it again, and hope the entire ecosystem it touches will also never be touched again… But you get the idea; that doesn’t fit modern development at all. It doesn’t fit progress at all. So that’s not working.

[51:54] That’s from a technology point of view… I’ve also been in the CTO summit of the CNCF last year; no, this year, actually. It’s 2022 still. It’s so cold, it feels like it’s already next year. [laughter] And we’ve been working together with the CTOs or head of infrastructures from Allianz, Mercedes, Santander, Evori, TikTok, Spotify, Orange… And it was really very nice to see that we have similar issues, enterprise issues… Like, you can separate it into big company bull***t and into real technical requirements. And what is very fascinating is that our approach that I’ve introduced at Audi to put people first, let them use cloud-native, in this case cloud-native tech, and explore it, and automate it, and let the process follow from that… Like, the handover process between friction walls is completely eliminated through cloud-native shift left technology. And the automation is a very natural process. And when you do that in one enterprise, and you can inspire, let it be one CTO per keynote - and I do like 25 keynotes a year - that will actually make the life of all your operations teams, and the company, and the developers, a lot easier. And they will not be called at four in the night for some stupid incident, because they had a container image not checking out through a policy, and whatnot. Or less. Let’s say less. [laughs] And it will make their life easier.

The culture is just so positive, actually, in this way of working with projects, and not very anonymously self-service, clicking something together, and sharing competencies, sharing passions, working in communities… And obviously, it’s just more fun to be in teams like that. And people are happier. They’re more passionate about something, and it needs passionate people to create great products. It just won’t work with people not being passionate about something. Putting up a great shop maybe… You can do that for a while, but you will not do that for very long. And they’re happier. And I tend to think that if I create like 200 happy people with every keynote, in the end, that’s 200 parents that have happier children in the end, because they can be there for them, at home, or whatever, and have more time. They nail their goals faster, which is also paying into a different type of work ethic. Solve issues, then have time, and not trade time for money; like, you have to work eight hours a day, I tried eight hours a day. No, that’s not what I want. I want people to get something done. If the stuff is done, and you’re happy with it, and it’s a great product, then off you go. And if you nail that in two hours now, because you have the tech for it, and the process doesn’t hinder you - off you go. I don’t care. And then you can focus on the next issue, or take a half a day off, because nothing’s burning. And it’s more relaxed… And like I said, and the endquote here is really that people are happier. In this sort of culture, the people are happier, and you don’t have too many levers, you don’t have sick days… They’re just happy, and… Is it work-life integration in the end? I don’t know. It depends on each person. But they’re definitely happier.

Rob Barnes: So I think that one of the things that – so I’ve been working with Vault many years, and how I got into Vault was every time – so I used to be a consultant, I used to help my customers increase their security posture when they moved to the cloud, and just extract more value from the cloud. And a lot of the challenges that my customers used to come up against from a security perspective - I’d always kind of look out there and see what other solutions that kind of tick all the boxes to deliver the business value they need? And Vault - it just kept coming up. So I thought, “Hey, I’d better kind of really dig into this and just understand why it keeps ticking the boxes, and understand the inner workings of it.”

So when we talk about taking away some of these manual steps - and yeah, what you’ve done is you’ve taken the approach that a true DevOps professional should take. So instead of trying to just build automation, you kind of automate the little pieces, and then you automate the automation, right? And that’s kind of the approach you’ve taken. And I think that’s good for a lot of things. But where it comes to security, I think we need to start to put systems in place to protect ourselves from ourselves, right? And this is where things like Vault can help. Because when we talk about the automation, rather than us having to write scripts, the automation is built into the functionality of the secrets management platform. In this case, we are talking about Vault.

[12:19] So we briefly discussed the concept of ephemeral secrets… That’s one form of automation. So it will create the secret and it will revoke it after a TTL has expired, which is going in and making the API call to the target platform, and doing it again to revoke it, and so on and so forth.

You also have things like – okay, so you have your secrets now in Vault, in this case. How do your secrets get from Vault to your application? And there are different patterns there. The common pattern, the favorite pattern, I’d say, in my opinion, is one where your application doesn’t even need to know about the secrets management platform, right? It doesn’t need to know about that. It just needs to know that I need to look somewhere to get the information I need. Maybe it’s a mounted volume, or something like that, whatever.

Ben Ford: Yep. That’s a very good question. The typical engagement that I have in my mind is that – I’ve got to the point where I’ve picked a decent set of tools and I’ve got something that I can spin up on low-cost cloud infrastructure very quickly. Which is good, because that means that the cost of that is minimal and we can spend the cost on doing the human things of uncovering where the business wants to get to, what the goals are, what the drivers are, what the constraints are. Because you have to have that kind of big-picture understanding first.

The rough view is that we would spin up the cloud infrastructure, which would be all the bits that you need; that would be owned by the business that I go work with. I’m a purely service-based offering here. I’m not making any margin on any of the tools or anything like that. It’s a purely service-based thing to build a capability that that business owns and operates and it becomes part of them. Then, there’s that discovery process of where does the business want to get to, what are the biggest kind of bottlenecks or constraints, how might we want to reconfigure parts of the business model, and the operations in light of having this new capability?

The last you want to do when you… This is the big problem with approaches like RPA. You don’t just want to go in and blindly start automating the stuff that’s there that might not necessarily want to be there. You’re just accelerating chaos if you do that. If you’ve got a process that’s not particularly well-designed in the first place, you don’t necessarily want to just blindly automate it, because then if it’s really badly designed, you actually could be driving the business into the ground if that process isn’t profitable or whatever. There has to be that big picture, first look. Then, it’s a case of building a targeting list and biting off the parts that are causing the most pain.

If you go into an engineering business, it might be that, “Oh wow, we’ve got all of these expensive engineering managers that are spending 20% of their time collating reports.” Click, that’s easy, we can sort that one out. We might have something where the breakdown of customers and orders and their behavior once they become customers is unclear, because you’ve got the customer acquisition data held in one system and the operations in another. Then what we might do is just centralize that and build some dashboards over the top so that we can see different segments of customers and how they behave, which is not impossible to do in Excel, but it absolutely sucks and it’s really painful to keep that up to date. Whereas what we can do with these capabilities that I just mentioned is you can have a real-time feed from all of those operational systems into the single source of consistent data. Backward-looking, because we don’t really care about… It’s not a transactional system, it’s a business intelligence system so it works in retrospect. Then you use Metabase, or something similar. There’s another great one called cube.dev, which is a headless BI system.

The general principle is that all of these different parts of the system, where possible, should be open source. They should run on cheap PaaS platforms that we mentioned, so that the overhead of them doesn’t require a full-time engineer just to keep it up and running, and you can still use all of the engineers to build the product. But at any point, the bus factor of me is zero because it’s all built off commodity open source software. So if you’re an engineering company, if the worst was to happen, then you would quite easily be able to manage and operate that system yourself.