Shawn Wang: I went through his stuff, and I was like, “Yeah, okay, this is on the high end.” I came in between like one to three as well. It’s fine. And then for training the thing - so it’s widely known or widely reported that Stable Diffusion cost 600k for a single run. People think the full thing, including R&D and stuff, was on the order of 10 million. And GPT-3 also costs something on the order of tens of millions. So I think that is the cost, but then also that is training; that is mostly like GPU compute. We’re not talking about data collection, which is a whole other thing, right?

[13:46] And I think, basically, there’s a towering stack of open source contributions to this data collective pool that we have made over time. I think the official numbers are like 100,000 gigabytes of data that was trained for Stable Diffusion… And it’s basically pooled from like Flickr, from Wikipedia, from like all the publicly-available commons of photos. And that is obviously extremely valuable, because – and another result that came out recently that has revolutionized AI thinking is the concept of Chinchilla Laws. Have you guys covered that on the show, or do I need to explain that?

Gerhard Lazu: That’s it. I mean, that’s the key, right? It doesn’t matter how many lines of code you write; people that still think in lines of code, and like “How big is this change?” You actually want the really small, tiny decisions that don’t change a lot at the surface, but have a huge impact. Some call them low-hanging fruit. I think that’s almost like it doesn’t do them justice. I think like the big, fat, juicy fruits, which are down - those are the ones you ought to pick, because they make a huge difference to everything. Go for those.

I’m wondering, how did you figure out that it was the database, it was like this buffer pool, and it was the disks? What did it look like from “We have a problem” to “We have a solution. The solution works”? What did that journey look like for you?

Adam Stacoviak: I’ve gotten a pool design before, and they’ve taken me around the whole entire pool right, and into the pool, and up around the pool, and over my house…

Chris Benson: Yeah, well, to some degree, I’m trying to think – so I’m trying to kind of bring it back around from our side to connect the quantum computing benefit with the things that our audience is typically engaged in, which is trying to get AI algorithms, the models developed, and then deployed out there. And so as they’re going through that process - I’m trying to kind of pool it all together now. What does that look like? Is there a quantum computer with the control system that you’ve produced sitting beside a classical computer that has a GPU in it, and a CPU in it, in the classical sense, and there’s some networking between them? What does that look like if we’re five years out? …or whatever, you can pick your timeframe. But the point where we’re now starting to integrate that into some sort of practical workflow, and people in our audience might be using it. What might that look like, from your position today, forecasting into the future?

Maggie Johnson-Pint: Yeah, so right now, I actually mostly pull alerts. So alerts can be generated out of Datadog metric alerts, because CPU is running hot, or something like that. Or you could pull it all out of like Sentry; it’ll give an alert to your frontend dev when the errors cluster, or things like that… And we’ll parse actually service names, AWS resource names, things like that out of alerts, CloudWatch alerts we can do, and we’ll pool them together and we’ll cluster them, and in that we can start to build out a graph of what you have in production. And because we can time-cluster them, and also because we tend to know with alerts the nice thing about them is what team they belong to, because they get sent to someone to page (it’s terrible), we can use both team relationships and time clustering to start to bring together and infer what relationships you have in production.

So that’s a lot of the data we use… Over time, we really want to start to bring in log data where we can, and start to bring in OpenTelemetry… You know, OpenTelemetry is always going to be more accurate than us when it can be there, and when it’s complete and plumbed; and so ideally, you start to overlay the OTL traces onto the inferred data to get that more complete picture, and that’s pretty high on my roadmap as well, to start to do that overlay.

Kris Brandow: [38:16] Yeah, one of the thoughts I had during what you were saying is - I express this in private to some people, and I always get kind of the “You’ve just said heresy” look or comments… But I have been wondering, is open source sustainable as the method of how we do things in this industry? Is this focus on sharing so much of code actually going to be something we can continue doing in the future, since there’s so much ambiguity around all of this? And quite frankly, I think it also atrophies the whole industry, because we’re not rewriting things. We’re not reimagining things. I think that’s one of the core problems with copyright in general - there’s this whole thing that Disney has done, where it’s just like… Copyright used to be like 20 years. Now it’s almost forever; or as Disney would like to have it, actually forever. So it’s like, “Oh, well, now these things are just kind of sticking around, and it’s so much harder to move things forward.”

If I remember the kind of genesis of copyright, or the vague genesis, it’s like “We want to protect people, making so they can profit off of their creative work for some time”, but then it goes back into the general pool of things so we can kind of continue making progress forward.

John Spurlock: [00:39:56.27] Yeah, exactly. I try to keep the conversation very open, just kind of let it hang out there… Because if you think about it yourself - and you guys are very open with your download numbers - really, it comes down… It’s all of the internet entrepreneur, you kind of want to fake it till you make it; you kind of want to put out the presence of, “Oh, this show is bigger than you think.” And once you get the real numbers, then it’s like “Oh.” And it is negative if – let’s say your show numbers are public, and you’re doing great; there’s no problem there. But then your numbers start to trail off. And it’s one thing if it’s only the podcaster that knows this; maybe they get a little bit depressed, or they get energized to do a better job… But now it’s a little bit worse if that’s public, right? You can see that heading in bad directions, potentially. So it’s really just making that [unintelligible 00:40:40.10] But Twitter - everyone sees your follower count.

I feel like especially younger people, they’re much more open and authentic from the beginning, so I think that’s not as much of a problem. It’s really more of people that are still in that old mindset of “I want to project that I’m this global show”, whereas that may not reflect reality. But again, I think having everyone’s stats in the same pool makes it a little less – because as you say, there’s actually a lot of difference between hosts, which is really tough right now. Even if they have the same IAB certification, the certification is such that the major inputs to that function can be variable. So which IPs you use to do different things are not specified. So even though you paid hundreds of thousands of dollars for these certifications, you don’t get apples to apples comparisons out the other end. And so you move shows, and all of a sudden your show numbers move, and that can be – some people actually use that as a selling point; they say “Actually, your numbers will be higher on our list, because we do a worse job.”

John K. Thompson: Exactly. And you could set your own objectives. So you want to say, “I want to maximize the amount of money that I accrue, because I’m going to take that money myself and spend it.” And it is money. It’s not credits, it’s not units, it’s money. It’s dollars, it’s euros, it’s drachma, yen, whatever it is. So you’re actually piling up money in your account that you can spend.

Now, your other objectives may be “I want to reduce the usage of my data by people who are climate offenders. Or maybe I want to help these charitable organizations understand my activity better.” Or maybe you find a group of people that are like-minded or have the same affinities as you do, and you grew up together, and all your data can only be used in aggregate as a pool. There’s a million different ways you can take this.

Chris Benson: You said something that struck me right there about having that level of confidence and the fact that you already knew it was hard to build those things out… That is something that stops a lot of would-be entrepreneurs right in their tracks. You’ve dived into the deep end of the pool… You’ve said a couple of times in our conversation that you were not coming into this as a world-class deep learning expert yourself. You’ve built a team obviously around, but you were coming in at someone with an idea. What gave you the confidence or the bravery to dive into the deep end of the pool and do something that we normally associate with people who might have a different background? … you know, have all that heavy math, and years of deep learning, modeling and stuff like that. How did you get past that? Because there are probably 1,000 people listening right now that want to be entrepreneurs, they’ve tried it, maybe they’ve tried and failed… How did you get past these hurdles?

Emma Draper: So I worked at a small startup, and that interview process, when I was going through it I did interview with the CEO. So speaking of just when you have really small organizations - the pool of people able to conduct an interview is much smaller. And that interview was intense. I mean, I think it was 20 minutes, and it was just rapid-fire, whatever questions kind of popcorned up, I was expected to answer, and to be really – I think that’s where brief and concise answers… It’s a good muscle to exercise, like, how are you able to answer the question pointfully, without rambling. So I think at a startup, that was more – I think it’s applied at any level of interviewing, but I definitely noticed it when I was on a call with the CEO, and just… What you say matters. Their time is really limited, and so you don’t wanna mess it up.

Gerhard Lazu: So after all my years on the RabbitMQ team, I can tell you that SSL is one of the hardest things in Erlang. The hardest. Not just when it comes to outbound connections, but also when it comes to internode. Internode communication can be very, very difficult because of SSL. And it has to do with the protocols, with the ciphers, sometimes with what is on the host… It’s a neverending pool of problems. Angela, she’s been on the OTP team…

Mat Ryer: Yeah. Pedantic. But the last question was – it just said, “Design or explain how you would regulate the temperature in a swimming pool.” And I wrote the BASIC code; because I used to write BASIC at home. So I wrote this little BASIC program that did that… And I don’t know, it was just like – we didn’t learn any of that at school, so of course, they were surprised, to say the least… But I did alright.

Chris Benson: I agree with you. Not only that, but we tend to – as we’ve been tracking what’s happening in the world over the years we’ve been doing the show, and we tend to gravitate naturally toward kind of the new stuff coming out, and all that… And what I was struck about is this is – I don’t know the details, but this looks to me like probably a fairly simple, by today’s standards, old-fashioned convolutional model, I would guess, that was doing this. I don’t think it’s necessarily the most cutting-edge deep learning of 2022… And I’m not being critical of those; if we happen to have listeners, I’m just saying you’re being very practical, in that you’re using a technology that’s been around for a while to do that… It’s really mundane, but with some fairly substantial effect, it probably did not cost very much to put this model together, and train it and get it better compared to many of the things we talked about now. And yet, they note that a 30 square meter swimming pool will result in about 200 Euros of extra taxes per year. So every pool is another 200 Euros that they can do. And apparently, that has amassed something along the order of 10 million Euros in new revenue for the French government. So enough to easily pay for the time of your data scientists that are working on that, plus quite a bit more. So I would not be surprised to see a lot more.

Amal Hussein: [56:03] It was a real pull request, but an old one, that had lots of mistakes, from a contractor, I think. So they literally used that exact pull request, and we’re like “Here. [laughs] This is from our actual codebase, seven months ago.” So I think that’s cool.

Other general positive experiences have been interview committees having an objective group of people from even different parts of the company come together and interview. I think it’s cool to do that, because then somebody’s interviewing for a role, not a job… And you’re interviewing for the role of software engineer. And “Oh, here’s the wide pool of different people who can give you this interview.” And it doesn’t matter which combination of those people; you’re hired generically to be a software engineer at this company, and then getting to pick your team after that is a pretty cool thing, having teams compete for you; it’s been nice.

But overall, the process is still way to difficult, and we can talk about that for hours and hours and hours. I’m curious to shift this conversation more towards what we were talking about earlier, so that we can end this on a positive note… What are some other things that we can think of - briefly, everyone - to kind of improve the status quo for today? Because I think some of the stuff that came up earlier was we’re gonna abolish – maybe we allow people to bypass interviews in exchange for more of a formal probationary period; maybe that could be an opt-in thing, and if you wanna do the interview, go ahead… But even then, I don’t know… And nothing is ever really guaranteed. I feel like people kind of freak out about probationary periods way more than they should. It’s like, we’re all employment at will, last time I checked… And that goes both ways, you know? [laughs] So I don’t get it, but… You know, whatever. What are some things?

Jerod Santo: “I used to think being swamped was a good sign. I’m doing stuff! I’m making progress! I’m important! I have an excuse to make others wait! Then I realized being swamped just means I’m stuck in the default state, like a ball that settled to a stop in the deepest part of an empty pool, the spot where rainwater has collected into a puddle.”

Reading this reminded me of Woody Zuill’s distinction between productivity and effectiveness.

Kevin Ball: That makes a lot of sense. And I think when you look at something like the JavaScript ecosystem, there is this sort of approach of “Let’s pool out lots of small projects together. Let’s create many composable pieces.” Those composable pieces may be related to each other; they’re trying to accomplish the same goal, and so a monorepo feels like it fits very naturally into this ecosystem.

Jerod Santo: People prefer to take pictures at the pool, of their laptops, versus working.

Adam Stacoviak: Revelbikes.com, check it out. Adam Miller, thank you so much for your journey as a founder, as an entrepreneur, and the scrappiness you were willing to put in to build a bike brand I love. I really appreciate that, because I get to go and shred on your bike. That’s my fun, that’s my outlet. When I’m not hanging out with my family, we’re not in the pool and doing our fun thing… You know, even with my kid on the weekends we’re riding. And my youngest, the two-year-old, he loves it, too. He’s got a little bike, and he’ll eventually have a bigger bike, but… You know, we’re a biking family, so I appreciate the journey you’ve gone on to enable my family to enjoy fun times. So thank you.

Adam Stacoviak: Would you hear stories though? Would you hear “This is how I’m using it. This is how it changed my budget.” “Wow, Mint has great potential, but it just is X, Y, or Z wrong for me, and Actual is better” or “YNAB was x, and this is better” or “Every dollar has a good possibility, but it’s just too clunky”, which I find that. I love the concept of every dollar, which is Dave Ramsey’s application kind of goes back to Jerod’s seller acquisition kind of thing… I’m asking these questions because I feel like – and I’m not in the weeds with you, but just on the outside, looking into some of the things, if you got to 800, which is a pretty respectable number, with no money spent in marketing really, just effort and showing up, what if you did a little bit more on that other front, which is more storytelling of your user base, and things that? Could you have gotten to 3,000 users or whatever number that was double or triple that? …to give yourself - I guess not just more money, which is a resource, but just almost the belief that what you’re putting out there is changing lives.

Because my hypothesis in this space, this budgeting space - one, it’s psychological. People do not like to budget. When you talk to anybody about budgeting, it’s almost like dogma. It’s almost a religion in a way even. It’s like “Well, hang on, you’re talking about my money? What do you want to know about me? Are you trying to sell something to me?” So everybody has a different way they think about money, they think about budgeting… So it’s almost like counterculture to even talk about budgeting, one. So it’s a hard job to solve for. And then two, almost every application out there for budgeting is not that good. So you almost have a disbelief as a user who desires to budget better, save for things in their life, buy a home, build a pool, buy a bike, buy a truck - whatever; buy these things you want to have in your life. You want to do these things, but nothing out there really offers it to you, and then here’s this thing that has this potential, and I see it as like a really good potential, you just need some more support to get you to that next big win, that next gust of wind to get you through the choppy waters and out to sea.

Ken Kantzer: [01:01:47.11] So the one that happened the most recently, that I think I linked to in the article, is related to DeFi. Essentially, the smart contract had logic in it for being able to almost like index a lot of different cryptocurrencies and autobalance them. And so what the guy did was he found a way to inject a lot of like a very cheap cryptocurrency into the pool that was used for calculating the balance of the index, and through everything that was completely legal within the smart contract, was able to extract tons of money from the system. And maybe it’s similar to how the stock market gets manipulated sometimes, like pumping and dumping stocks. It felt a lot like that. It was legal, perfectly legal within the system, but it just was not the original intention of the developers.

Ian Lopshire: Yeah, exactly. They are getting something in return, right? Having Go as a strong community provides them programmers they can hire, expands the job pool… It does a lot of different things for Google. So I also don’t want to forget that it’s also a benefit for them to have a strong community. We do should appreciate the support.

Matt Moore: [20:08] And in particular, there’s sort of two classes of apps. There’s apps that ignore signals and they stick around for what’s called the termination grace period, which is between the SIGTERM and the SIGKILL, which luckily defaults to like 30 seconds. So it’s not sitting around forever.

And then the other class of people are the people who – well, there’s a third, which they do it properly, but that’s like super-niche. The second big category of people are like, “Okay, well, I’m going to do signal handling. When I get SIGTERM, I’m just going to quit”, right? And that’s actually not what you want to do, right? You want to handle SIGTERM by starting to fail readiness probes, but all your normal requests will be handled properly, because it takes time from when you start to fail readiness probes until your pod is marked not ready. That’s the failure threshold on the readiness probe. And then once your pod’s marked not ready, that has to roll out to all of the network programming, right? So your pod’s endpoint has to be removed from the endpoints on the API server. So the service controller has to see that your pod’s not ready, remove it from endpoints… But you’re not done there, right? Those endpoints then have to be propagated, in vanilla Kubernetes, to all of the nodes, which have to reprogram their IP tables, or if you’re in mesh mode, every single pod sidecar now needs to know that like, okay, that endpoint is no longer available, right?

So in some cases and some scales of clusters, I don’t think that 30 seconds is even necessarily long enough. But the reason I bring it up is we did a whole bunch of magic in Knative, since we know it’s an HTTP based service, to make it so that it is really hard to get that wrong. Because it’s really hard to get it right in vanilla Kubernetes, but it’s actually really, really hard to get that wrong in Knative.

One of the things we do is we have a pre-stop hook where we do something somewhat magical where the pre-stop hook is on one container, but the place to send it is on the other container. So we have a proxy that sits in front of the application container, and when Kubernetes is going to go stop the pod, instead of actually sending any signal to the user container, it sends it to our sidecar first, and our sidecar starts to fail probes, and do it properly, so that you don’t have to.

So if you’re in the first camp of folks who doesn’t really handle the signaling at all and just continues to serve traffic normally, you will still drain properly, because our - what we call the queue-proxy - will actually handle that for you. And if you’re in the second camp, where you just do what I call a YOLO exit, you’re like, “I’ve got the SIGTERM. I’m out”, you’re still good as well, because since we have that pre-stop hook, we get the signal first, we make sure traffic has drained, and then by the time you’re actually getting that signal, traffic’s been routed away from that instance of your application. And so it’s really, really hard, actually, within the context of Knative, to handle that wrong. And I think that’s a really important thing to get right if you’re using any sort of auto-scaled application, because when you scale up, there’s a window where the new pod’s coming up, and if it reports ready before it’s really ready, you’re in trouble; you’re going to serve 500s. And when you’re scaling down, if traffic continues to go to those pods after they’ve started to shut down, you’re going to get 500s, right? So the goal is zero 500s, and we have all kinds of tests in Knative where we’re like, “No, there should be zero 500s.”

[24:05] The other thing that we do that is really hard is - and the networking layers make this incredibly hard to do, and we work around all kinds of stuff in basically every Ingress provider - is ready means ready, right? Everyone at the networking level is like, “Yeah, it’s eventually consistent. It’ll get there at some point.” But it’s like, no, if we roll out a new revision, we want to know, when we tell the user like, “Yeah, yeah, you’ve got your new code”, that we’re not lying, right? And so Knative does all of this fun stuff where we actually inject hashes of the network programming into the network configuration in ways that our elements of the data path will respond with the header that’s being injected by the network programming, and then the components we have can actually probe different things to understand what version of the network programming has been rolled out. And then once it’s been rolled out everywhere - we can’t do this in mesh mode, because we can’t probe mesh sidecars, but we do this for probing the pool of envoys if you’re running outside of mesh mode. So for instance, traffic serving off cluster, we can probe and make sure that once we fully roll things out and we say it’s rolled out, you should never get the old version. It is at the new version, because we’ve confirmed all the networking programming is there.

Adolfo García Veytia: So right now we did a demonstration of the SBOM Tools. SBOM is Software Bill of Materials… That we built for Kubernetes during the KubeCon in L.A. So those tools have become more sophisticated in the six months that just passed. We now have the ability, for example, to attach multiple – to split SBOMs into multiple SBOMs.

For example, an SBOM is a list of materials and assets that you release with your software. It can list, for example, source code, container images, binaries… Everything that you put out as a software release is listed in there. And usually, you will find that you get a huge list of everything which you download, and you use. But now we’ve been focusing on splitting the SBOM into micro SBOMs, which you can store right besides the images, for example. So instead of having one central SBOM, you will have the contents of the image stored inside of the registry, and we now are working on the tools to pool those together, and that you can present a central view of all of the information in separate SBOMs in one central place.

Then we just finished signing Kubernetes. There will be some demos about those signatures, how they’re structured, what information you can get from them, and if I have time, I will also like to show some of the provenance attestations that we publish with the Kubernetes releases. We’ll see.

Anton Titov: Yeah, but if you know how to pack it down to this level, you can create a lot of scalable applications. Because when you create traffic filtering software, or vpn cores the things which actually – well, a real IP, let’s say, not just API endpoints… That’s where you have to optimize it. And knowing these basics and knowing that “Yes, 64 kilobytes sounds like a small amount”, and it’s a joke which Bill Gates said back in the day, if someone doesn’t remember; obviously, he’s wrong, but… If you realize how actually huge is this amount… 64 kilobytes of stack on some hardware chip can let you stream a ton of traffic, and build something like a Netflix, build something like Starlink, because all these things which are doing great, great things - they have to be optimized. You can’t pool ten CPU servers in space; this is not going to work.

Mark Ericksen: Yeah, we had gone through multiple different jobs. Sometimes the companies were large enough where we had our own operations teams, and they managed all of that. It was completely invisible to me. So I was just among the pool of developers, right? And then other places, much smaller companies, development and management of those servers fell to our responsibility, and we would have problems where sometimes we would be on a server, and the server would just die, and we didn’t really know why… And so you’d restart it. And eventually, it was like, “You know, it’ll just be easier if we went to Heroku.” Even though it was more expensive, it was moving the app to run on Heroku when we had Rails, just because we didn’t have the platform experience for maintaining that.

Jerod Santo: I mean, I’m an old-school sysadmin, and I was like “SSH into the machine, set up some cron jobs, copy the stuff, rsync the things, backup the database…” And once I didn’t have to do that stuff anymore, with Heroku, I never wanted to do any of that stuff ever again. I am a loyal listener of Ship It, but I don’t do any of the things that you guys talk about doing on Ship It. I just like your show, Gerhard. But once Heroku came around, I was just like, “Yeah, let’s just let Heroku do all the things.” And when it came to Elixir, I lost that, when we were gonna go deploy this Elixir app. So that’s when I brought you in to help me do the things that I used to be okay at, but also don’t know how to do things well anymore in this ecosystem, and thus began our story.

When it came to Ansible, I was along for the ride. When it came to Concourse CI, I was just riding your coattails. When it came to Kubernetes, I was like “I hope Gerhard knows what he’s doing, because I don’t…” So just the Heroku style, PaaS, for me is exciting because I feel like it’s a pool that’s shallow enough that I can swim in it safely, and not have to turn to you and say “What’s the kubectl thing, or the K9s thing?”

Alex Sims: It did. I think it was within a few months after I joined the company; we split the database off onto a second server, and then I think we finally had – I think we’d always had web servers, and then worker servers that processed all of the tasks. But I think when I joined, shortly after in that first peak, was when we started adding more and more of those nodes to the pool.

Leo Mutuku: [20:28] So when I think about open data, it is in a way democratizing access to data… And this means that if some investments are put in place to collect this data, it’s still beneficial to our wider group of people that just those who collected the data.

Now, specifically in AI and agriculture, as I’ve mentioned earlier, it’s quite difficult to get accurate groundtruth data to support interventions. So there are a few projects now that are supporting the collection of that data; this data has been collected from a small sample of the farms in the region, so the applicability of this data is only as useful as whether it can be reused on repurposed in other areas, or at least used as training datasets for algorithms that may not necessarily impact the places where the data has been collected, but are for similar contexts.

Traditionally, we have been saying that one of our biggest development challenges here in Africa is the scarcity of data. So open data I think creates mechanisms and communities around sort of meeting that data gap, and being able to support further application of datasets.

You’ve mentioned the issue of community, and I find the concept of open data serves communities in two ways. So there’s the community of the producers, that can pool together resources, they can pool together data, and reused them, reproduce whatever experiments or research that has been done, without having to expend resources. But at the same time, open data I think has a close relationship to what is usually called citizen science, that we can also encourage the communities themselves to contribute to the creation of this data, and then when it’s placed in an open format and in these open repositories, it’s also accessible back to them, either as outputs of technologies applied on this data, or they can use it directly.

So I think open data creates this community resource that, again, preserves some of that indigenous knowledge we alluded to earlier, but at the same time promotes the development of new technologies where there might be sparse data previously, because of this increased availability or relevant datasets.

Solomon Hykes: I think of BuildKit as a VM. A VM in the machine sense. It’s a new kind of programmable machine, and we’re building an operating system for it… Or that includes it. I don’t know exactly. But yeah, if we talk only about long-term - so ignore for a second our focus today on specific problems for specific people; to solve that problem, deployment is painful, DevOps engineers reinvent the wheel, all of that… That’s kind of an opportunity to build an engine that solves the problem better. And then we have an engine, and we have a community, and a product, and we’re launched; we’re on orbit. So that’s gonna be, I’m gonna say, a good year of work to finish that. But then you’re in orbit, and you have this community, using this engine, and that engine gets better and better.

I think you can do a lot more with Dagger if you think of it as a new kind of operating system… Because a lot of engineers I think have this feeling that the state of the art of software needs to evolve; we’re all stuck with these old paradigms and interfaces, but there’s this gravity pool of compatibility. It’s gotta be POSIX, it’s gotta be libc… You’re kind of stuck in that world. But then there’s all these distributed systems, and all these incredible capabilities now, but your software is not native to them.

So the hard part is not inventing a new operating system from scratch that’s native to distributed systems, and the cloud, and all these incredibly powerful things computers can do now. The hard part is getting developers to actually write software for it.

Deepthi Sigireddi: So one of the things you can do by having a connection pool is that you can convert load into latency. So if you can serve ten thousand users without any noticeable lag, and suddenly you get a hundred thousand - if you can make them wait, then maybe they will get a response after 2 seconds, instead of 10 milliseconds or 20 milliseconds. But you get the ability to handle that load just by delaying some of that work.

Obviously, if you have a sustained peak, if your traffic doubled overnight and it stays double, then you have to start provisioning more capacity at the backend. But transient peaks you can handle with a connection pooling strategy.