Ahmad Nassri: The same patterns. And I think the short summary of that - I’ve had the privilege of being in leadership positions or executive positions where I don’t only look at the technology and the code and the infrastructure, but I also have to look after budgets. I also have to look after people’s salaries. I also have to look after sustaining people’s salaries and sustaining those budgets ongoing, year after year, quarter after quarter.

[12:04] So the dynamic shifts a little bit when you start thinking about “Are we making the right choices? Are we doing the right decisions?” When it’s not just “Is this the best library that we can use?” or “Is this the best framework that we can use?” or “Is this the best cloud system that we can adopt?” It’s also like “Are we investing money in the right space?” because if I’m gonna throw my money at a vendor – I don’t wanna name names, but like big enterprise vendors and it’s gonna cost millions of dollars, what happens to the team that I’m paying also their salaries, and how does that relationship work together? Am I gonna have to hire more people to maintain the product that I’m also spending millions of dollars on? Or I’m gonna have all of a sudden no need for the team members that I have, because I’ve shifted all my processes to a product that can, for whatever shape or form, replace a lot of the functionality that we’re doing already…

So those are both hard problems, but also critical problems, because we can debate all day long the value of frameworks and tools and services and libraries and all that kind of stuff, and the shape of an architecture that we wanna debate - and trust me, I love that stuff… If left to my own devices, to my own means, I would probably over-engineer everything, because why not? I love to create new things, I love to experiment with technology, I love to create all this complexity of cloud architectures, and challenge existing patterns, and create new ones… But at the same time, if I have an accountability and responsibility for people’s livelihoods, their salaries, their job functions and their job growth, I have to be very critical in the way I pick choices of technology, even down to the framework, even down to the process of how we implement code and how do we do automation testing, how we do deployment CI/CD, all that kind of functionality.

The patterns that I’ve noticed, again, in big enterprises, in small companies, in agencies, in outsourcing companies - there is a disconnect between developer experience, as we were talking about earlier, before we started recording, and user experience. And even though some companies have the kind of values and espouse things like “We care about the user, we care about the customer and all that kind of stuff”, it doesn’t really translate to technology and how technology is implemented. Somehow there is this [unintelligible 00:14:12.25] barrier in front of developers, to how that actually reflects on the way that I do my work.

So if I’m a developer and I’m given a story in a backlog or a task as part of a team to implement a frontend experience, or a mobile app, or a backend server, or an API, or whatever - yeah, sure, the company has these big values about users first, blah-blah-blah; it doesn’t matter, because the code needs to be the code, right? That’s how most people think.

But the reality is there’s this concept of – again, as a person who has to own budgets and has to own long-term planning for team members and technology, there’s a concept of the total cost of ownership of something. If I’m gonna make a decision on something, I need to measure the total cost of ownership about it. And to me, that kind of breaks down into five different things - the cost of knowledge, meaning if I’m gonna adopt their library, what do I have to expense in terms of time, energy, and dollars to make sure that the knowledge of that library is equally distributed across the team members that I have? And there’s a relation there to how big the team, and also in relation to how long I need to keep this thing going…

Which leads to the next one, which is the cost of maintenance. Like, fine, you’ve adopted a library, you’ve adopted a service, you’ve adopted an infrastructure tool, but how long is this gonna be going on, and how long are you gonna be invested in maintaining it and invested in having team members continue to run it, or update it, or deploy it, or monitor it - whatever the functionality is, given the component that we’re discussing.

There’s also the cost of the ecosystem, which we’re very familiar with in the JavaScript world… Basically, pick the library that has the most contributors. But even in a tooling sense or infrastructure sense – if you’re gonna buy into Kubernetes, for example, you’re gonna buy into a whole set of other things that you’re also gonna buy into and pay the cost of… Which also leads to the cost of infrastructure. If you’re gonna adopt a vendor product - cool, you’re adopting a vendor product. But is there infrastructure costs associated with that?

[16:07] Most people forget about that when they’re doing the choices of technology, saying “I’m just gonna use Gmail.” Okay, you’re using G Suite then, which means you have to do everything through APIs.” You no longer have access, for example, to the low-level systems of file storage and data storage. Now you have to have tools and libraries and desktop application management for IT teams.

There’s all these kinds of concepts that people have to measure before they get to the cost that most people actually think about, which is the cost of adoption, i.e. “How long it will take me to implement.”

So when I look at this big picture of the cost of ownership, I look at the way people make their decisions and the way people try to take shortcuts by saying “You know what - I can build a login system myself. It’s only a couple of libraries, and my cost of adoption is a day’s worth of work, or two days’ worth of work before I go and just deploy it… And then I’m done and it’s clean. This is it, this is the cost.” And they forget about the maintenance, they forget about knowledge sharing, they forget about the ecosystem around it, and they forget about the infrastructure involved.

So that’s kind of like the preamble to where are those solved problems that exist already, and how people try to re-solve them or reinvent the wheel around them… And my experience has been also in - like Amal’s earlier point - all the stuff that I was able to get things shipped; I get things shipped by simplifying the problem and getting down to the basics, rather than trying to break things apart and say “Hey, let’s use this library instead of that.” I start with the basic thing of saying “Why were you even doing this to begin with? Let’s discuss there. Are there shortcuts and cheaper options to follow?”

Adam Stacoviak: You were talking about Tigris, and correcting yourself… Tigris is basically like the promise of object storage, but it allows you to actually build a CDN. Like, you can basically run a – you can write some JavaScript, use Tigris, and you have a CDN baked into your application. You couldn’t build anything like that before. You could try and do this with S3, but it’s so complicated to like manage multiple regions of data that it’s impossible. Supabase for Postgres is like this. Supabase is a Postgres that lets you build new kinds of apps, or build apps faster. It really gives developers power.

And so the Rebel Alliance idea was if the future is developers are picking cloud platforms, the best possible cloud platform for developers is 10 companies that have built a very special version of infrastructure, that makes developers more powerful by itself… And then obviously, developers would just use those 10 to 50 things together. Like, why would they use my object storage when they could use Tigris? Why would they use my Postgres when they could use Supabase? Why would they use my GPUs when they could use Replicate, for example? And I think it’s like – I don’t know, it’s one of those things that I could still rationalize it. It seems like a thing I’d really enjoy, but there’s actually huge structural problems with it, which is what people tend to want when they launch an app, is actually a really consistent UX, that solves a higher order problem than they can solve by themselves. So a PaaS like Heroku – Postgres isn’t a product. It’s actually part of the product. It’s a feature of the PaaS. And so you can do cool things like do PR reviews with your existing Postgres data, for example. And so there’s a UX issue here, where you can’t actually solve problems as well with 10 to 50 different companies as you could if you were just doing it all yourself. And there’s weird compliance issues…

One of the things about getting a HIPAA BAA, so you can do a healthcare app, is you actually end up having to sign with each of those 10 companies. You can’t just do one HIPAA BAA with us… Which I think is a real burden for developers. I don’t think they wanna do – even if you can unify [unintelligible 00:26:52.14] I don’t think they wanna do multiple contracts with companies, you know? It just creates more and more friction.

So there are actual practical issues with like a Rebel Alliance cloud. But also, I think that a lot of it is just politics. There’s just very few companies who have a low enough ego to give up a big chunk of potential revenue like that. Us saying “We’re not gonna take the Postgres revenue” is pretty big, because EC2 – I mean, most of AWS’es profits come from RDS. That’s like a huge market, that we just said “Hey, we’re not gonna have this. You can have this.” And that’s not a thing most companies would be saying. In some ways, we needed a company to come back and be like “Fine, then we’re gonna give up the compute revenue”, you know? And that was actually relatively rare, to find people who were that… Communist? I don’t know. [laughs]

Narayanan Raghavan: So it’s an interesting question, because it depends, to be honest. But before I answer your specific question, I’ll have to walk you through how we do upgrades, because this ties into that. We give our customers the option to upgrade whenever they want to upgrade. So in fact, we give them three options. We tell them, “If you want to upgrade now, here’s the big, red, easy button. Click it. If you want to schedule your upgrade on December 31 at 3am UTC, have at it. Or if you want to automatically upgrade your cluster every time there is a new version of OpenShift available, you can actually set it up like your smartphone, so every time there’s a new version that’s available, you pick the day and time, the cluster will automatically upgrade during that day and time.”

Now, when that happens - and most of our customers set it up for automatic upgrades, which is great… But when that happens, one, when there is a, you know, CVE, or there’s a kernel bug, or what have you, we inform customers to say “Heads up. This is happening.” Because again, communication, communication, communication. It’s always key. So we make sure that customers are aware that this is happening. Customers have the option then to say “I want to upgrade my cluster right now, because I have an upgrade available. I have a fix available.” But in some cases, not everything requires an upgrade. And if it doesn’t require an upgrade, we patch it behind the scenes and push the change throughout the entire fleet.

[42:01] The catch with either customers scheduling an upgrade, or us pushing a change through the entire fleet, with either one of them, we generally advocate and train customers to say, “Follow best practices. Follow Kubernetes best practices. Follow cloud provider best practices etc”, because that’s going to help them in the long run.

So for example, if you don’t have pods with requests and limits set - well it’s an anti-pattern. Go fix it. If you have a replica count of one - that doesn’t make sense, because you’re gonna have an outage when we do an upgrade. So I actually tell our customers, “Don’t worry about the upgrade. The upgrade can take two minutes, the upgrade can take two weeks. It doesn’t matter. Worry about your workloads.” Because as a managed service, it’s my job to worry about the upgrade, right?

So as long as you’re following best practices, the upgrade will happen. We will guarantee that the upgrade happens. As long as you’re following best practices, your workloads are not impacted; you won’t notice a thing. Because OpenShift and Kubernetes is going to manage scheduling your workloads, moving your pods around etc. So you don’t experience downtime because of that.

Recording begins: [02:09]

“Alright, let’s do this.

Do you feel kind of nervous about functions as a service?

It makes sense to be on the fence. That serverless learning curve is kind of steep. It’s not cheap. Even if it’s affordable, know that Node code you wrote ain’t portable. Yeah, good luck.

Serverless is not a panacea, it’s an idea… That infrastructure can be structured like it’s from IKEA… Running your code on compute you don’t manage, freeing you to pursue your true business advantage.

Now, does the tooling have some gaps? Sure. But just like our apps, perhaps, it’ll mature. Meantime, the hype is a little bit fair… For the first time in ever, you don’t have to care. If you’re over-provisioned, or paying for idle, or managing servers where each has a title, like cattle01, or myspecialpet, when they’re really just beasts you should try to forget.

No maintenance windows from midnight till 2. Jeff Bezos will pay those sysadmins for you. [laughter]

Amazon hasn’t got your back, forgot them; try Azure, or Google, or - yes, IBM… Use Cloud Functions, Step Functions, Firebase, Cognito, or wrap them all up like a steakless burito. In short the quick kick of that serverless stack is addicting as heck, and there’s no looking back… So go write a function, deploy and test, and let the cloud do what the cloud does best.” [applause]

Gerhard Lazu: Yeah. One thing that we talked about previously, and I thought was a really important point, was around what makes a good developer. [unintelligible 01:12:26.29] go further, what makes a good engineer, a good software engineer. And it is not the documentation, even though that plays a very important role. It is not the code that they write, even though that has to be present. It is not how they present and how they talk. But again, that has to be present. How they share ideas, how they show into the world, and do whatever they have to do… Those are well-rounded individuals. And that well-roundedness, I can see how it translates to what is the cloud, what is the on-prem… Whatever the next thing is, you need to be well-rounded in all these things. And you should be using all of them. Because if you don’t use it, you lose it, right? Very simple, very true. It will never change.

So you should be using the cloud, but you should be using the cloud the way the cloud was meant to be used. And Justin, you made a very good point, at Disney, how you were using the cloud; possibly the best example of using the cloud correctly, right? Span out, stand up all these instances, and then tear them down. That’s how it’s supposed to be used, right? It’s capacity on demand, massive capacity. You would not want to buy that. But when you need it, it’s there, and it’s a great commodity. It is a hundred times more expensive, because that’s what this commodity costs you. Where else can you turn up and say “Hey, spin up a hundred beef instances?” That wouldn’t take months, it would take maybe years to source all of those things. And by the way - no, no, I don’t want that AMD CPU. I want that Intel CPU. Oh, ARM? I think I’m going to have some ARM. And by the way, I’m going to have, I don’t know, 10,000 ARM CPUs. Where can you get that? It’s the convenience of that.

[01:14:09.20] Now, if you know what you need, as a business that is successful, as a startup that’s successful - well, for the things that you know that you definitely need, and that’s like your baseline operating budget in terms of infrastructure, have that in a place that makes sense. Have that cheap. Have that - cheap, I say cost-efficient. Do what makes sense. Hybrid cloud.

What about WASM? What about the WASI runtimes? How do they change how we see containers and how we work with containers? Is that coming? Well, I can tell you that React did not work for Dagger Cloud. React, the DOM, to generate all the things that we had to generate, was just breaking down. The tech was not up to scratch. So what did we do? We looked at WASM v3. That’s the v3 cloud. We went through three rewrites to get it to a point where it’s as performant as it should be.

And there’s so many dragons there. Don’t think that this is like the holy place where you pick it up and everything is awesome. Oh, no. The cycle starts again. But that’s it. That’s the beauty of it. Start those cycles, keep going through them, keep learning, keep iterating… Eventually, you will be able to consider yourself well-rounded. And others will look at you and will say “Wow, I wish he responded to my pull requests. I wish he left some comments, and did some reviews, because I like his reviews. I wish he blogged more. Or I wish she went and gave some talks, because she’s an excellent presenter.” So look for well-rounded, not for 10x.

Benn Stancil: So taking the modern data stack, for instance. The primary thing that the modern data stack sort of did was say “Let’s move a bunch of data tooling to the cloud.” That’s probably the biggest thing that it was – its biggest sort of philosophical bent, was cloud-based SaaS software versus some sort of like heavy infrastructure that you buy from Oracle or Microsoft or whatever. And so there’s a lot of things that people did to make that easier. Data pipelining can be now moving from SaaS – rather than these sort of big, heavy data pipelines that you have to write yourself, to sort of push-button stuff from SaaS products like Salesforce, into data warehouses, and easier ways to be able to share data back and forth between other tools, or share data between… Like, how does the data team share their results with the marketing team easily, and stuff like that… All those sorts of things.

So there’s a bunch of stuff here that like – the way that data teams work kind of changed. They also became – there’s this sort of new team… Like, prior to 2010 data teams were kind of either capital D Data Science teams that were doing hard math, or they were kind of like business intelligence reporting teams that would be building just like binders of reports for executives… There also kind of was this rise of like the analytics team that was supposed to help people make better decisions by doing a bunch of analysis that wasn’t “We’re gonna build some crazy model”, but was “We want to help you decide how to be smarter.” The easy analogy for this is like sports teams. There’s analytics teams in sports franchises now, that are data scientisty types, but they’re not saying “We’re gonna build some crazy predictive model that we can plug into anything.” They’re saying “Oh, we want to help our coaches make better draft picks.” Businesses try to do the same thing.

There were a bunch of tools built for all of those things, for those people, for those different workflows, things like that, that were kind of new and novel approaches to it. But during kind of the frenzy years, those things, one, were all new, and so they were still figuring out how to make it work, they were still figuring out “What’s the best way to do this and move it to the cloud? What’s the best experience for that?”, all that kind of stuff. And because there’s so much money in it, people are trying to move really quick, they’re trying to like have some – like “We’ve got to build a giant business.” And as a result, you build kind of shoddy products. They’re not bad, but they’re frontier; you’re figuring out what to do. You don’t really know what works. The ground is changing underneath you some…

So for instance, Mode got built – started originally in 2013. There’s this really popular open source tool called DBT, that’s like a data transformation tool that helps you define how to basically build data pipelines. That thing got popular and became pretty ubiquitous 2017-2018. Now what Mode does would be better off served if it also kind of understood the way that DBT works, because a lot of customers are using DBT. However, Mode was built before that. And so if we were to rebuild Mode today, you’d kind of rebuild it knowing that “Okay, there’s this new set of technologies that most people are using.” There’s a lot of examples of that sort of thing, where products got built when the ground was still evolving, and everything was still changing… And so they’re not quite built for the world that exists today.

And so I think there’s there is an opportunity now to basically say, okay, the ground shifting has settled. The earthquake is sort of over. There’s a bunch of half-built buildings that were built on the ground before it shifted entirely to where it is, and they’re a little bit shaky, and all that kind of stuff… And actually, you can now go back and basically say “We’re just going to build with those things built.” The ideas were good, but they were early. There were products that we’re still figuring how to do it; they were products that were built for a slightly different time. Let’s just say we take the really good ideas and rebuild it for the landscape as it exists today. And you can’t do that when everything’s crazy and there’s so much money everywhere. But when things are calmer and more settled, there’s a lot of opportunity, I think, to do that.

Alex Gallego: Yeah. You know, strategically, one of the best things that cloud has allowed us to do and explore is this idea of data sovereignty, which is this totally new idea for cloud products, that in my experience only started to become popular really in the last six months. And with cloud – okay, so part of the reason why this was late, just for a full context for the audience listening, is that we did launch a cloud, and we actually onboarded public companies in the cloud, so it actually did work, but it wasn’t the architecture that I had in mind. I was like “This is not the future of cloud.” And so we threw it away, and then we wrote a new cloud. [laughter] So maybe the context is that we did end up writing it, but we just wrote two clouds before we called it GA.

And so on the second version that we just launched, what it allows us to do is this idea of “Bring your own cloud.” Still fully managed. But the hard drives, the data that Redpanda writes - those hard drives belong in the users of VPC, but the control plane is in our VPC. Now, from a user, it’s still a fully-managed SaaS, just like Mongo Atlas, or something; you go in, you click-click, deploy a new cluster, in your region, in your cloud etc. That all happens exactly the same. But the user owns that data. And what’s interesting in that is that at least the computational model that we define is unified for both dedicated, and this idea of BYOC.

I think that privacy is easier to achieve the data sovereignty, where for some industries like insurance, or marketing, where if they add a new vendor, they have to send an email to like 10,000 customers. They’re “Hey, we’re adding a new vendor. These are the risks” etc. And so we de-risk all of that, because they get to run their code, Redpanda data plane, in their VPC, and we simply do all of the management. It’s all automated. And so that cloud was really what took us two years to build. It’s like, a new version that gives us the flexibility to either run the data in your cloud, or in our cloud. I think that’s probably why it took so long for us.

Jerod Santo: I mean, it seems like to me the best pitch for cloud things is like “Hey, here’s an easy adoption avenue”, right? Like, you can be there with your deploy to cloud thing, or develop in cloud thing x, just like the one Codespaces works on github.com right now. And then on the indie side, like the developer side, I just feel like it’s such an easy sell, because it’s like “Hey, don’t you want your open source project to be immediately hackable?”

Zac Smith: It’s so important in startups, because you haven’t figured it out. You don’t know the How, you know the Why. And “Why are we doing this?” And then you’ll figure out the How, which will probably change. You might even change the What. You know, what we’re trying to do – for example, the mission of Packet, beyond a very high-level one that we put on our T-shirts called “Build a better internet”, was that we wanted to connect the world’s developers to the technology, no matter where it was, no matter what it was, and no matter what you wanted to build on it. That was the What. We were like “You know what - we believe that software is gonna be portable and free, and innovation happens, magic happens when you put the right hardware and the right software together. We wanna do our job and make that better. And how we’re gonna do that - it turned out to be a bare metal cloud. That actually wasn’t the main vision. The main vision was like “Be the best bare metal cloud in the world” - it was not that. It was like “Get technology in a sustainable way out to the developers of the world, so they can create magic.” That’s what we wanted to do.

Peter Wang: I mean, just configuring NGINX, or setting up Apache, getting SSL Cert in there, doing all this other [unintelligible 00:52:24.23] that’s impossible. And even for a dev like myself, it’s really just annoying. So what the Jupyter Notebook did - believe it or not, a lot of the value is simply making the web writeable. Making a writeable web technology accessible for people who were not even programmers, so were not familiar or comfortable with the shell. That’s the other thing that’s true about data scientists - a lot of them are not comfortable with the terminal at all. A lot of them are on Windows, and now they can build websites on Windows, with interactive widgets, running massive-scale computation. Holy crap. To complain about the Jupyter Notebook at that point is like complaining about the cup holders in the Starship Enterprise, or something. It’s like, stop it. You’re moving at warp 9, just shut up. [laughter]

So I think that’s the thing that devs like myself, who have a dev background - we look at the space of technology and it’s a relatively flat landscape. “Yeah, I can go learn this language. I can do that thing. I can go grab a cert, spin up that AWS credential, no problem.” For the average person, every single one of these things is a cliff that’s insurmountable. And this is kind of to the point of Anaconda, as well - actually, a lot of advanced users do use Miniconda as their deployment technology; like I said, within Dockers, and whatnot. But there’s also others who are like “Well, I can do my own thing. I don’t really wanna use this package manager thing.” But there’s so many people out there learning how to do this stuff - they’re on Windows, they have no idea what a compiler is. They just wanna do their jobs, [unintelligible 00:53:44.18] So that is where accessibility, again, building on the Python motif of being accessible as a language; you want the tooling around this to be accessible…

And this is now - to answer the second part of your question - where we’ve fallen short. I think that as the technical space of data science and machine learning has grown up, put on a suit, got a real job… We’ve got software developers coming in and saying “Hey, I’m gonna retool as an ML engineer; I can learn the stats”, and they can, they’re smart people; they can do all these things.

Part of what’s being lost nowadays that I see in the modern tooling is kind of the taste is lost. The taste of making this accessible for people who are not ops geeks. I think some of that drive has gone away. So when I go and look at the documentation for any kind of ML framework - and this is not to put a ding on them; you’re doing complex orchestration, there’s gonna be some work involved. But just in general, that sense of like “How do we make this dirt simple for that poor atmospheric scientist trying to model hurricanes, trying to understand climate change? How do we make it simple for these guys trying to build better machine translation for African languages?” and they’re out in an African village trying to get the locals involved. How do we make stuff really easy to use for them? That kind of thing is, I think, being crowded out a little bit. That kind of sensibility and taste is gone, I think. And it’s unfortunate.

There’s also willingness to embrace really big, corporate open source, which - again, I salute the companies for open sourcing their technologies, but I think the open source ecosystem around SciPy and whatnot never really had some of the corporate open source hegemony kind of thing happen to them that happened in the Linux space. So they’ve never seen weaponized open source… And I’m one of the few people trying to go in there and raise the banner and being like “Hey guys, recognize there’s community grassroots, open source community innovation”, and then there’s big companies saying “Here’s our big ol’ toolkit, a million man-hours, and it’s all your to use it, please… And by the way, it runs best on our cloud framework.” That’s – meh, I don’t know how to feel about that exactly.

Justin Garrison: Well, and I’ve completely changed my position on good cloud architecture, where I think that lift and shift is the best model for the cloud, because it’s the easiest way to get out of it again. And the cloud should be temporary leasing, like you do if you go travel to someplace on vacation. You’re not going to go buy a house; you’re going to be there for a week and then you’re going to leave. And that is how you should treat the cloud. You should lift and shift there, and then you should get out as fast as possible, back to stable things that you own and you know the long-term costs of.

Alon Klein Orback: On the tractor, yes. We have a few devices there, and they are running the K3s. It’s the light version of Kubernetes. It’s really nice, because we get best practice from the cloud, and we get best practice to the edge. And, for example, if we want to use ClearML from the Edge or from the cloud, it doesn’t really make any difference to us. So we know how to press and move the secrets, and how to use it, and we just do it in details, yes.

Andrew Thorp: Yeah. So basically, right now, just to summarize and make sure I understand this correctly - right now just because GitLab is itself open source and maybe gitlab.com the cloud is not the best solution for an open source project right now, because there are no public repositories. But eventually, if that happens, you would consider moving GitLab itself into GitLab to be hosted there…?

Nikita Shamgunov: But think about it the following way - from the VC standpoint, there is a market; it’s called the database market. That market has players, and those players have share. And that share is on dollars, and on usage. And then there’s – you can also measure share in terms of mindshare. So if you look at share of usage, Postgres is going up and up and up. So that’s data point number one. And earlier, I said out of the top five databases, Postgres is the only one that’s growing share.

The second thing is, within the database market there’s an on-prem market, and there’s a cloud market. And the cloud market is the much faster-growing market. That market is dominated by the cloud hyperscalers - by Amazon, Google, and Microsoft. There is only one public database company that’s relatively modern and relatively recent, and that’s MongoDB. There isn’t a public relational database company, and developers are increasingly choosing Postgres over MongoDB.

Another data point is AWS Aurora is a $3 billion run rate business, going into potentially four and a half, growing 50% year-on-year into the next year. And that’s MySQL, Postgres, and MongoDB. MongoDB is kind of fake. It’s built on top of Postgres, because MongoDB’s license prevents them from running MongoDB compute.

So all of those data points highlight that there are a lot of dollars in the cloud database market. Postgres as a matter of fact, in a way, it’s kind of like Linux. So it’s not like you can – you can’t own Postgres. That ship sailed. But can you have the best in the world Postgres cloud service? I think that’s an open question, and that’s what we’re going after. That’s really the pitch, it was that simple.

Jerod Santo: It’s now time for Sponsored News!

WarpStream lets you bring your own cloud

WarpStream’s BYOC deployment model for Kafka-compat data streaming gives you the best of both worlds between self-hosted & cloud. This model also give you a lot of zeros (that’s a good thing)

Zero disks - WarpStream’s Zero Disk Architecture eliminates local disks entirely and reduces storage costs by more than 24x.

Zero inter-zone networking fees - More than 80% of Kafka costs are not hardware – they’re inter-zone networking fees. Because WarpStream runs on top of S3-compatible object storage & doesn’t manually replicate data between zones, those fees are completely eliminated.

Zero ops auto-scaling - WarpStream replaces stateful Kafka brokers with stateless Agents, so your team can skip the weekly burden of partition rebalancing, scaling headaches, volume management, capacity planning & more.

Zero access - WarpStream Agents run on your VMs, in your cloud account / VPC & storage data in your object storage buckets. Traffic flows seamlessly from producers to consumers without ever leaving your virtual private cloud (VPC).

Here’s a non-zero - WarpStream works in every cloud! They have native support for AWS S3, GCP GCS & Azure Blob Storage. It also works with any cloud or self-hosted solution that has an S3-compatible object storage.

Head to warpstream.com/byoc to learn more and get started for free.

Benjamin Consolvo: Yeah, I can start, and then Greg, you can fill in anything, Greg, that I’m missing. But I think the best way is to get onto the Intel Tiber Developer Cloud, which is kind of our Intel cloud where developers can come and try out both our hardware and our software that’s set up right there for them. We’ll offer kind of our latest, whatever we have on that platform. So we have our Xeon product, we have our Gaudi product, we even are going to be having a dev kit kind of for the AI PC, a simulated environment; even though it’s not a local machine, it’s still a cloud. So that’s, I think, the best way to get started, is the Intel Tiber Developer Cloud. And Greg, did you have anything to add there?

Jerod Santo: Google’s position is the most tenuous, since generative AI is a direct threat to search, their cash cow.

and Microsoft seems the best placed of all. It has cloud services like Amazon, deep investments in OpenAI, and perhaps now a way to make Bing truly compelling over the next few years.

Those summaries were not written by an AI, but they also weren’t that great, so you should read Ben’s words directly, I think.

Jason Keene: Our hope is that we have perhaps distanced ourselves away from being specific to Cloud Foundry. There’s a lot of neat things happening everywhere - for example Kubernetes - and what we don’t want is to have the Loggregator system be so opinionated that it would only be useful within Cloud Foundry.

Loggregator is doing its best to be a distributed logging and metrics system for several distributed systems, and not just Cloud Foundry. We had our original iteration where everything was specific to Cloud Foundry; every metric type was something that only Cloud Foundry could ever care about, and the consumers that would only ever care were at Cloud Foundry. What we noticed was other interesting distributed systems were coming up, such as Kubernetes, and Loggregator was in a technical position where it could service these things, but we were so opinionated about Cloud Foundry we really worked putting ourselves in front of that.

We decided to move our API more towards a generic place, and have generic metrics with a subset of the information we previously had, that would be useful therefore for any different number of distributed applications, and enable distributed applications to use Loggregator to optimize their apps, or maintain them, do DevOps work and just ensure that iterations are happening.

Pivotal’s main focus is to enable developers to do interesting things, so Cloud Foundry wants to enable developers to do hard, complex distributed systems, and maintain them, and iterate on their software quickly, and have confidence that their stuff is actually working, so they need visibility in that.

Jerod Santo: Yeah, exactly. And they’re doing that almost purely for cost savings. I would say that’s their primary motivation. And the cost savings are significant if you are in that group of people who have a stable business, with known growth trends… Because the cloud provides a lot of things, but the main thing it gives, especially for startups, is that dynamic scaling. You know, scale it up, scale it out, scale it back down again… That’s really powerful for companies who don’t know what their server demand is going to be. So that’s one.

And then there’s also this movement of taking cloud APIs to either your own hardware, or to collocated and cheaper VPS-style infrastructure. So now you’re still – everything we learned about the dynamic scalability of cloud APIs and AWS and others have given us, what if we could use those same APIs, but build them on our own hardware, for instance? That’s another movement. And that one’s about – I think it’s also about probably privacy plus cost, but without losing a lot of what the cloud provides. It’s trying to have the best of both worlds, I guess.

Jay Patel: The first step - and Sathy can add some context, but the first step is to collect as little as possible. We don’t know our users’ names, we don’t ask for things like age or other demographic information… Obviously, we need a phone number, and we’d like to know which health facility they signed up in, and how many months pregnant they are, so that we can tailor all the message campaigns according to the stage of pregnancy, or whether they have delivered.

[36:03] Then on the backend just making sure that everything is stored according to industry best practices on one of the major cloud providers, and using their security tools. I think that rather than me trying to manage a server here, just using the resources that are already available helped us be better able to secure the data. Sathy?

Jerod Santo: [55:47] “You’ve now left the ‘90s…” Maybe I should explain why we’re using local storage. Some of it is actually just technical debt. This was a decision I made when building the platform back in 2015, around how we handle uploads. Not image uploads, but mp3 uploads… Which is one of the major things that we upload and process. And these mp3’s are anywhere from 30 to 100 megabytes. And once we have them, we also want to do post-processing, like post-upload processing on the mp3’s. Because we go about rewriting ID3 tags, and doing fancy stuff based on the information in the CMS, not a pre-upload thing. So it’s nice for putting out a lot of podcasts, because if Gerhard names the episode and then uploads the file to the episode, the mp3 itself is encoded with the episode’s information without having to duplicate yourself.

So because of that reason, and because I was new to Elixir and I didn’t know exactly the best way to do it in the cloud, I just said “Let’s keep it simple. We’re just gonna upload the files to the local disk.” We had a big VPS with a big disk on it, and were like “Don’t complicate things.” So that’s what we did.

Jerod Santo: “You’ve now left the ‘90s…” Maybe I should explain why we’re using local storage. Some of it is actually just technical debt. This was a decision I made when building the platform back in 2015, around how we handle uploads. Not image uploads, but mp3 uploads… Which is one of the major things that we upload and process. And these mp3’s are anywhere from 30 to 100 megabytes. And once we have them, we also want to do post-processing, like post-upload processing on the mp3’s. Because we go about rewriting ID3 tags, and doing fancy stuff based on the information in the CMS, not a pre-upload thing. So it’s nice for putting out a lot of podcasts, because if Gerhard names the episode and then uploads the file to the episode, the mp3 itself is encoded with the episode’s information without having to duplicate yourself.

So because of that reason, and because I was new to Elixir and I didn’t know exactly the best way to do it in the cloud, I just said “Let’s keep it simple. We’re just gonna upload the files to the local disk.” We had a big VPS with a big disk on it, and were like “Don’t complicate things.” So that’s what we did.

Erik St. Martin: We are back with for another episode of Go Time. This is episode number 13. Before we kick off this show I wanna give a quick shout out to our sponsors for this episode. Linode, Changelog’s cloud server of choice, and Equinox.io, the best way to package and distribute your Go apps.

Today on the show we have myself, Erik St. Martin, Brian Ketelsen is also on the line…

Autumn Nash: But it makes me so upset that people always want to say that’s like a non-traditional – people are like “That’s a non-traditional background.” And a lot of people have degrees that were working in the helpdesk. Dude, some of the best principals I know were either helpdesk people, engineers, cloud or support engineers, or they were test engineers; QA testers. They are the best. They will debug stuff in a way that you’re just like “I didn’t even know you could do that.” Literally, one of my favorite principals - she is just amazing. And she used to be a support engineer, and I’m just like “How can we down that?” People look down at people, like “Oh, you’re like a non-traditional.” And I’m like “They are badass. They can do the most amazing things with a terminal, and find this problem…” And I’m just like “Teach me your ways.”

Tammer Saleh: In terms of knowledge, I think it’s almost more important to have a deeper understanding of Linux networking, and just networking in general. From our experience, understanding how a cluster IP service works, for example, and all the IP tables stuff that happens there, understanding how load balancers work, understanding why node ports are a terrible idea, or understanding how Ingress works at layer seven, right? All of that is conceptually harder for our students, from what we’ve seen, conceptually harder for people who are new to Kubernetes, because they just never had to deal with that kind of networking knowledge.

I think another thing that’s important for a team who’s getting started with– well, first of all, let’s talk about how you should adopt Kubernetes. First of all, even though I kind of pooh-poohed the value of the Kubernetes managed services like EKS, AKS, and GKE, you absolutely should use them. I mean, yes, you can deploy your own cluster, but why? Just go with one of the managed solutions. Frankly, they’re cheaper, especially GKE, right?

[32:08] And if you have a choice, just to– if you have your druthers about which Cloud to be on, GKE is by far the best experience, and Azure is by far the worst experience, not just in terms of Kubernetes, but just across the board, right? And AWS is what it is. So if you’re on AWS, you’re probably forced to be on AWS, and whatever; you’re on EKS. And then once you’ve got that, as I mentioned before, there’s so much other stuff that has to be configured and deployed on top of that, and our best advice is just to keep it as simple as you can.

Most of our customers have already spent so many innovation points when they are adopting Kubernetes. We kind of feel it’s our mission, our job to help guide them towards more conservative solutions, and fewer moving parts… Because it’s so tempting, once you’ve got Kubernetes, like “Ah, I guess I need Istio because Istio does all these cool things. It does. And if you need those things, that’s great, jump on board. But holy crap is Istio complicated, and it’s dangerous. I mean, if you misconfigure Istio, you can really do damage to your production traffic. And avoid any tooling that you don’t have an immediate pain point for. When you look at the CNCF landscape, it can often look like you’re in a toy store. You see all these wonderful, cool gadgets and you just want to grab them all up into your basket, but you need to show a lot of restraint, because every one of those that you add is something else you have to manage and understand.

William Morgan: Yeah, so for me I think it’s a theme more than anything else… And it’s a theme that we didn’t really expect when we were first starting to develop Linkerd, but it’s one around security, around especially security of the traffic in your cluster.

So we came into Linkerd in the early days of the project very reliability-focused. Our background was at Twitter, and Twitter was constantly down, at least at the time… So our vision for what we were doing was we were gonna have load balancing, and retries, and blue-green deploys, and all these reliability techniques. And what we learned early on was that a lot of the use – I mean, some people love that stuff, but a lot of the use of Linkerd was for Mutual TLS. Why? Because people wanted to encrypt the traffic in transit. Why? Because either they had these regulatory concerns - “Oh, we work with financial data, and the government basically says we have to do this”, or they just have security concerns. “We’re running in the cloud. We don’t have any control over the network. The best practice is we should maintain confidentiality.”

[08:09] So that was like our foray into the world of security, and that theme has continued to develop through the policy features, the micro-segmentation, and onto other features, more types of policies… You know, there’s a lot more we can do in this area of “How do you secure the traffic in your cluster?” And it’s a blossoming area, because everyone I think is becoming a little more comfortable with Kubernetes, so the operational concerns are – I wouldn’t say they’re taken care of, but they’re understood. And now they’re in the world of “Oh, crap… Now I can run it, but how do I secure it? How do I make sure that if one node gets hacked, that everything doesn’t fall apart?” Or more likely, if someone deploys a mistake, it can’t accidentally delete our users, or expose sensitive information to the outside world.

So that theme has been just developing for us over the past couple of releases, and it’s gratifying not just because things like that are cool, but because people are using it, and they’re getting a lot of value out of it, which is kind of like the end goal of Linkerd; if no one’s using it – I don’t know, to me that’s a little unsatisfying.