Jeff Lindsay: Yeah, so a lot of kind of function as a service is usually designed around HTTP, so you’re writing web handlers and you can kind of shell out and do whatever if you have containers. Usually, these things aren’t made to work for – you know, let’s say you have a script. A lot of people’s automation in their companies is based on Bash, or various scripts, and in order for people to run those scripts, they have to download the script, or install it as a package, make sure you have all the dependencies, and of course, you’re not running in containers because you’re talking about people’s development environments… So everybody has to have the right dependencies, and then if there are secrets – like, let’s say your tool is gonna go do stuff with AWS and it needs your AWS credentials and you have to share your credentials with everybody for it to work… So this was a way to basically say, “You can put that script in the cloud… It’ll run in the cloud, it’ll run with secrets that you specify that nobody else has access to, and you can run those commands with SSH.” So you can say “SSH my deploy site command” from your company or your app or whatever, and it’ll go and deploy from anywhere.

They are stateless in that you don’t have – but you can pull from Git, you can kind of do anything you want from the command line. However, one of the things that I’ve been working on for a while is a way to expose your current directory to it. That will open up the possibility of writing tools that actually interact with the file system and particular files in your project.

Again, it’s kind of like the speed to get somebody working on your project, have all the right tools and stuff like that - you can actually put all those in the cloud, and then they kind of run a command… And it’s working on your file system, but the code is executing somewhere else. So it’s run consistently, and any secrets or passwords are protected, and you can share access to commands. So it’s a really powerful – it’s a dumb primitive. I did it because it’s – it’s not like, you know, the most amazing primitive, but there are use cases, especially when it comes to groups, trying to automate things like deployment in kind of an opsy world… And you could easily throw a Slack interface onto it, and now you have ChatOps type of stuff.

[44:25] So function as a service, focusing on command line… That’s really what it is. And yeah, it’s a great tool. So I was gonna rebuild that using this component; it’s using an older version of it, and isn’t as componentized as it could be, but this is a way that people can kind of see “Oh, here’s how you’d build an application or a system that’s fairly complicated (not super complicated) from scratch, using these kinds of components.” And a lot of those components will go into future projects as well.

A common one that we have besides HTTP is SSH. We have a really great SSH library for Go, people don’t know… GliderLabs SSH. It wraps the existing crypto SSH library and gives you an interface that looks a lot like the HTTP interface for servers. So it becomes really easy to build SSH servers and do stuff with SSH… Which is a really cool protocol by the way, because it can do connection tunneling, and all kinds of neat stuff. As a protocol it has a lot of interesting primitives inside it.

Brad Fitzpatrick said he thought that it was dope, so I think that’s good. He said it’s refreshing to see good API design… I was like, “Well, I’m just copying HTTP, so…”

Marques Johansson: So you’ve been moving your infrastructure from – a term I first heard from you, I think, which was [unintelligible 00:45:26.13] You’ve been moving from that to some sort of stateful configuration where you can treat your entire deployment as [unintelligible 00:45:34.23] And I think that’s probably come up a few times. You’ve probably hit some walls and just taken advantage of that kill switch and just rebuild… And Terraform was that answer, I think, for a lot of people, and it’s still where a lot of people are. It allows you to just have however many components you need, have each one expressed as a few lines of HCL configuration, destroy the entire environment, reapply the entire environment.

One of the hurdles of that situation though is when things don’t apply cleanly, or you need somebody to actually push that button, and that’s where Crossplane comes in. Crossplane takes advantage of the Kubernetes reconciliation loop to bring these infrastructure components back to life, provision them the first time, sync things up. One thing is in a failed state, another is in a successful state… That failed state is eventually going to turn green. Your deployment is going to succeed, whereas in Terraform you’re generally not gonna have that experience. You might have to destroy the entire environment and bring it back up, and you’re gonna have to probably push that button to reapply it.

So what do we have on the Equinix Metal side that allows you to use Crossplane? We do have a provider, and that provider allows you to deploy devices, [unintelligible 00:46:50.05] and IP addresses. There are many more infrastructure components that we can introduce, but we started with the ones that are most relevant.

There are some other integrations with Crossplane that are useful to consider here, because when you are provisioning something – if we take the Terraform model, you’re provisioning infrastructure and then a lot of folks will rely on SSH-ing into that infrastructure to get it configured the way that they want it to be configured. We don’t have an SSH provider in this Crossplane ecosystem, at least not a fully fleshed out one… So we have to take advantage of user data. And what user data allows you to do is when you’re provisioning a device define all of the scripts that need to run on that machine on first boot, and that takes out all of the variability of SSH, of “Am I going to connect to this machine? Am I going to run the same script multiple times?” You’re going to define with your user data what to run at boot-up. You will not require external access to that machine, because the cloud provider’s API is going to make sure that that script is executed.

[47:55] In our environment, where we have layer two configurations, you cannot SSH into the machine to perform the actions that you want without going through a gateway node, or without going through a serial terminal. So the way that you execute code on those machines or execute scripts on those machines is through user data.

One of the formats that’s popular for configuring your user data is – well, cloud-native is the tool, cloudconfig is the format. It’s something like Salt, or Puppet, or Chef, where you have this declarative language to describe all the packages that you need installed, whether or not the system should be updated, describe files that need to be created, services that need to be running… And a common way to approach user data is to just provide a cloudconfig file that declares all of that.

So one of the Crossplane providers that I worked on introduces [unintelligible 00:48:44.02] to Crossplane, which you can use in conjunction with the Equinix Metal provider, and you can stagger your deployment, in a way, to say “When this resource is fully configured, take some component of that, tie that into this cloudconfig script, and then when that script is ready, use that to deploy this Equinix Metal device.” So you can get these complex compositions, taking advantage of Crossplane’s compositions, and I think that that’s where you’re gonna wanna go with this complex deployment that you have with Changelog.

Kris Moore: But some of us have lives we want to live also, outside of just sitting there, troubleshooting our own gear… And so TrueNAS comes in like “Look, we’ve done all that work for you.” It’s a pretty good sized team we have here just to make sure things like replication just always works. Backups always work. They’re reliable. Because once you go down that rabbit hole, it’s like “Oh, well, I got the replication working. But how do I monitor? Oh, I’ve gotta set up alerting. How do I do that? How do I make sure that that –” And just on and on it goes. Well, how about data integrity? Am I monitoring my disks enough? Am I getting alerts properly? Am I backing up – now I’ve gotta backup to the cloud too, because I need off site… So now I’m scripting [unintelligible 00:23:51.00] or something else, and it’s just like… At some point it’s like I’ve got better things to do.

Stephan Ewen: Let’s say you’re recording the episodes and then every time an episode is done – no, let’s do an AI thing here, or so.

So you’re building your chat where you can chat with an episode, like “Okay, tell me where did they talk about this”, or “Tell me what episodes talked about these topics”, and so on. So what you’re doing whenever an episode is done - you’re feeding it first through a model that transcribes the audio, then you’re chunking it up, feed it through embeddings models stored maybe in a vector database, and then you have kind of a RAG style way of… You know, when a query comes, create the embedding, look up the similarity search in your vector database, feed it to the model to get the answer… For something like this, if you – let’s say you started just building the flow in let’s say a Node.js application, like in a simpler way; you just said “Okay, here’s the episode.” I have something, it gets uploaded… Let’s say you’re uploading it to an S3 bucket and there’s an event whenever something gets uploaded to this bucket; you have an event that represents this, and then it starts a Node.js script, or something like this. And this script is of the type that if it fails, somebody would have to restart it. Now let’s say you’re trying to implement that with Restate. I would say approach it the following way… The first thing is get a handle of Restate itself; there’s a cloud service that you can use on our site, which has a free tier. Either go there, or just use one of these ways to run it yourself on a single machine with an EBS volume.

[01:22:06.14] Then you have the server there. Then put your Node.js script maybe – you can actually put it on something like Lambda or ECS; just like use a serverless option to host this. And then use the Restate SDK to define the entrypoint and tell Restate “Okay, hey, here’s the service that you now should durably manage.” So Restate will then go there and discover this and understand “Okay, hey, there’s this (what what do we call it?) video transcriber, or video embedder service.” And then Restate knows about this. And then you would go to your Amazon console and say “Okay, for this type of event I want to create a webhook to Restate”, so that it makes an invocation to Restate and says “Okay, this thing has been updated.” You know, the kind of event that would previously call directly your Node.js process or script, you actually make it an HTTP call to Restate, and Restate would then call your process.

You’ve already gained one thing right away. You now basically have a reliable queue in front of it, right? Like, just that if you don’t do anything special. So when the webhook comes, it’s going to be acknowledged back, and Restate has this [unintelligible 01:23:15.22] of your process if it crashes, it will retry this… It will actually give you a nice observability; much more than you would get from your average message queue, about like individual retries, configuration about time-offs and back-offs and timelines and so on.

As a next step, you would actually then go into your script and say “Okay, let’s actually identify the steps where if something fails in this step, or after that step, I don’t want it to go back.” Like let’s say forking the process that does the transcribing, or like calling the LLM to create the embeddings. You introduce then the Restate context that you get by using the Restate SDK and just say, “Okay let me wrap these API calls just with restate.run.” That will capture the results of this durably, and basically turn – you’ve now turned it basically into a workflow.

Let’s say you want to do something like parallelize the different steps. Maybe just typing this one by one through this embeddings model is a little tricky thing… You want to fan out. You could then go and say “Let me try and do the exact same thing I do in a regular Node process and just make a bunch of function calls record, like remember the promises”, sort of a way to promise that all for those in the end join the results, and put those in the database. You can do exactly that in your code. Just, again, anchor this in the Restate context, so you get like this durable parallelization, durable stuff like scatter, gather, and so on. And so you would then incrementally sort rewrite your code to say “Okay, let’s actually make this step durable, let’s make that step durable, and that step durable.”

Say as a next thing maybe one of your folks wants to approve it before it really goes out. So you then possibly – you “Let’s do that in the simplest most possible way”, which is we create an awakeable or a durable promise in Restate and say “Okay, somebody needs to complete this actively”, like send an event, make an HTTP caller to complete this and say “Okay, this is approved, go through” or “No, this is not approved. Abort.” You can then use, for example – you could put the result, the transcription just in Restate, save it, and somebody could look at it from the UI and then say “Okay, yeah, I’m making an API call here to approve this”, and it continues. And so you can then incrementally rebuild your process into durable steps.

As the next thing you could then, for example, take it and migrate it from a long-running process to a Lambda function. Because one of the nice things you have with durable execution is when it’s waiting for something else to happen, it can actually just make this thing go away. Because with durable execution, it knows how to recover back to the place where it was by replaying the history of durable steps.

[01:26:14.28] So you could then say – you know, if you’re on vacation and you approve it a week later, you don’t have some process running and waiting for it. It’s just like, it’s going to go away, and when the approval finally comes, it’s going to come back, use the durable steps to replay back to the point, and then to the remaining steps. And so typically, folks would incrementally then rework their non-durable services; first connect them to Restate to basically get the equivalent of a durable queue, and then incrementally rework it and say “Okay, I want to add durable steps here, maybe parallelization, maybe a signal”, and…

David Crawshaw: Right. Yes, I think that lines up really well with the way Josh and I think about these things, where today if you open up a model, a cloud provider’s frontier model, or a local DeepSeek, or even a Llama 70b, you can ask it to write a Python script that does something. It could be a Python script to go to the GitHub API and grab some data and present it neatly for you. And it will do a great job. These great models can basically do this in a single shot, where you write a sentence and out comes a Python script that solves a problem. And that’s an astonishing technical achievement. It’s amazing how quickly I’ve got used to that as a thing, but…

Adam Azzam: So you had said “Talk to me the experience of taking my Python code and converting it to a Prefect workflow.” For folks that are listening, it’s as simple as from Prefect import flow, and then @flow on top of your function. That’s all the work it takes to take a Python function and give it superpowers, basically. To add all the stuff that we’ve talked about. And that doesn’t take away any sort of attributes of that function. If I take my Hello World or my ETL flow that runs locally and I start adding Prefect things on top of it, it still runs locally. And so it’s not like – we don’t sort of require additional infrastructure just to have your stuff run. We detect whether or not you’re running it locally or not. And if you’re running it locally, we execute it as if it’s regular Python functions.

[00:32:03.29] Now, there’s kind of two stories of “Well, this thing is working on my machine. How do I get it to run somewhere else?” We have two ways of doing this - and I’m sorry for the vocab lesson - which is let’s say that… What’s the crawl/walk/run for anybody trying to get something to run remotely? It’s you start off on your machine, and then it’s like “Well, I’m gonna go get a server somewhere, and I’m just – if I can close my laptop and go to bed and this thing is still running on my server, that is what counts for me as running it remotely.” That’s the next step in the hierarchy of needs.

So for that, if it can run locally on your server - you go to EC2, you SSH in, you’ve got your console, you pull your GitHub repo, you pip-install your requirements… You’re still able to execute that flow on your machine, and now if you execute that function and you hit say like .serve, now in the same way that you would start a fast API application, it is now running, you can specify a schedule that it runs on… It exposes an HTTP endpoint that you can call out to if you want to invoke it on demand, and it listens to events.

So maybe you don’t want to just run it on a schedule. Maybe sometimes you just want to be able to hit a specific endpoint to manually invoke it, maybe from your Django app or something like this, or maybe you want to emit an event into the world that says “Look, when my user signs up, when my dataset is ready, I want to have this ETL flow go and operate on it. Because my data is not always ready at 6 a.m. Sometimes it’s not uploaded till 6:02.” So sometimes you want to have things be a bit more dynamic. That’s what happens in a [unintelligible 00:33:47.10]

So where are we in the hierarchy of needs? I had it on my laptop, but it stopped running when I shut it. I go and I get an EC2 instance, I have it running there… Now I can shut my laptop and it’s still running. And then now how do you do this at a massive scale? And how do you do this auto-scaling with respect to the amount of work that you have, right?

If your EC2 instance - you hit it 10,000 times, and all of those things, every invocation requires downloading some data, that machine’s gonna fail over quick. So what happens when you want 10,000 things to spin up just as many machines and you want to fan out? That would look like my flow.deploy.

So I’m here on my laptop, I’ve got my Hello World flow… I would literally just say “If name equals main”, like I’m writing any other script, flow.deploy, and the sort of autocomplete in your IDE is gonna tell you “Alright, what do you want to name this thing? Where do you want it to run? Do you want it to run on ECS? In Amazon? Do you want to run in Google Cloud Function? Where do you want it to run? Can you point us to – do you have this in a Docker container somewhere? If so, we’ll just like pull it from whatever Docker registry that you have. Or do you want to point us to a GitHub repository?”

And one of the nice things is when you build a tool like Prefect that treats failure as a first-class citizen, you see the failure modes of tens of thousands of people trying to get their code to run, and so this really allows us to build an ergonomic experience that’s like “Look, here’s the minimal stuff that we need to know in order to have a high-reliability guarantee that this is gonna work” when you shut your laptop, when you submit it to a Kubernetes cluster, what have you.

So the experience is really pretty dead simple. If it’s running locally on your machine, you can hit .serve, and now it starts a process on your machine. If you want it somewhere else, it’s .deploy, and we can guide you through exactly how to point it at remote infrastructure. Does that answer your question, Daniel? I hope I’m not glossing over anything.

Steven Wu: If it does come, I think there are definitely – if you’re automating a query, for instance, there’s actually things. So there’s this notion of a thing called like an automated detection rule for like XDR type stuff, and basically, the idea is you have some query… So the way that Splunk or like Datadog or those guys implement it is that you have some query, and it runs every five minutes. And it’s a query like “Did this thing happen? Did someone change admin permissions on my AWS rules? Or did someone change - whatever? Did we get ingress from like weird IP addresses that we know to be compromised?”, stuff like that. And when those find something, they send off an alert, so it’s like the whole workflow… And those queries are queries that you run pretty much every five minutes, every minute, whatever… But they are also the same query that runs every five minutes. For that particular use case we actually have a bunch of these – we have like incremental caches where you can kind of… If you’re careful about how you select your aggregations, and how you select their mathematical properties, you can actually build like a cool little rollup tree where you – you’re effectively only running that query over your entire dataset once ever, you just do it incrementally as it comes in. Stuff like dashboards, same thing. You can kind of build this cool, incremental cache of everything, and then you just ask the cache for this range, and it tells you… And there’s no query cost involved.

So I think for stuff that isn’t ad hoc querying, stuff that isn’t freeform exploratory querying, which I think in that model that you’re talking about is probably most of that very high-volume automated, like, people doing this query every two minutes on a script type thing, that basically doesn’t cost us anything at all, because of that choice of caching. And it is true that all of it is trade-offs. I think we have made really strong, very specific, opinionated trade-offs. I think I said at the beginning, we have a very opinionated cloud architecture. But I think a lot of the gamble of doing this company such as it is is that we’re gambling on having made the right trade-offs. And I think that’s kind of true for any technical decision. There are trade-offs that you have to make for your problem domain, and you can get way better performance under certain circumstances by making those trade-offs, and not on others.

Anurag Goel: [00:36:03.18] Yeah, absolutely. And like I said, I think this continues to evolve, and we’ve had – I mean, obviously, Docker or just container capabilities are a great way to figure out what people can or cannot do on the platform… And so we rely a lot on that. We rely a lot on other kinds of protection mechanisms… But what we’ve learned is actually that the weak links aren’t at the container isolation level. When you work with cloud providers, the weak links are often in the primitives that the cloud providers might expose themselves. So you have IAM, and you have some workload running on a machine that can contact your control plane database. And you want to make sure that parts of your system are completely isolated from your own control plane - and I’m not talking about the Kubernetes control plane - but you still want to be able to run your own code close to the user code. And there’s a lot of complexity involved in that, that we learned to build around. And Kubernetes control planes are shared. Obviously, as a customer, you cannot access the Render Kubernetes control plane.

There’s also isolation by default, in that the workloads that customers have on Render, they don’t run on machines that are directly accessible over the internet. So those machines don’t have public IP addresses. Everything is behind NAT gateways. And that allows us to control traffic and isolation a little bit better as well. So customers don’t have to worry about script kiddies trying to look for open ports on their apps. And everything is behind our routing and load balancing layer. And then there’s firewall rules on top, and stuff like that. So I think that is where you often find issues.

There’s also this notion of shared load balancing and shared router layers. And early on, very early on, we ran into cases where one customer went viral, and brought our load balancers down… And so we’ve learned to, over time, build out isolation techniques there, and things like rate limiting we’ve had to build ourselves into our own load balancing layer, at the domain level or at the customer level… A lot of that multi-tenant work happens at one layer above. At the same time, we’re tracking all these CDEs that might show up, and we make sure that we’re updating the kernels, we’re updating the VM operating systems, all of that. All of that happens behind the scenes, so we’re never exposed to the kind of container vulnerabilities that might happen.

And having said that, I think there’s a world in which, for specific customers, we make sure that they have their own VMs, for example, so they don’t have to worry about other containers running on the sort of similar infrastructure.

Jason Hall: Only. Only 99% of the use cases. As for our release schedule, we have a scheduled build every - I think it’s two hours. But also during the workday, we also just push every change from main. So as someone is doing fixes to the image, it’s also rolling out those config changes.

You mentioned testing… There’s a ton of - well, there’s a ton of interesting stuff about testing. We don’t do, I think, enough testing. I don’t want to make it sound like they’re untested images, but the testing bar is very high, I think, and we can always improve there.

The testing we do on the images as they go out the door is actually part of a really cool thing we do. We build everything with TerraForm, we configure all the images with TerraForm, and one of the nice things that gives us is TerraForm is also usable to spin up and deploy resources, cloud resources [unintelligible 00:12:01.21] K3s clusters locally, and interact with them, and deploy Helm charts, and things like that. So we will build an image, let’s say for cert manager, and before we tag it, we’ll send it to a Helm release for that image, make sure that it works, run a script against it, do some sort of basic smoke testing on it… And then only after those tests pass will we tag it as Latest and release it out into the world.

Zach Lloyd: Yeah, so a really simple thing; here’s what we use it for at Warp. An example would be we have a command that we run in G Cloud, that whitelists an IP address so that from that IP address you can access our staging rig if you need to. And so developers have to run that when they go work from a coffee shop, or like switch locations, wherever their IP changes. It’s a thing that people wouldn’t know how to do. You can put it in a script, but what Warp Drive is - it lets you create one of these templated workflows. You create it for yourself first and foremost; people will make these “Hey here are my common or uncommon Git things that I do, my uncommon Docker things that I do.” They’ll save those… You can parameterize them, meaning you can be like “These are the arguments that go into them as flag values.” You can give those parameters actual documentation, and then as the final step, what you can do is you can share it with your team. And then everyone on your team, whenever they’re like “Oh, crap. I need to whitelist my staging thing”, rather than slacking, or going to Notion, or pulling up a Google Doc, or searching through their scripts or whatever, they as part of basically our command search, so as part of Ctrl+R, you could just be like staging whitelists, and all of a sudden, you have that templated command. So it’s a much easier way of sharing knowledge, and sharing commands in the terminal. So that’s the first thing.

Now, the way that we’ve built Warp Drive is that it won’t – this will not be the only type of thing that you can share. For example, as a next thing that’s coming - and this is public on our website, so I’m fine saying it’s coming - we’re building a notebook data type that lives in the terminal. This is something where if you add an entire documentation plus commands workflow - and that could be like “Here’s how you do cherry picks into a release”, or “Here’s what you do if there’s high 500 errors on a server, if you want to debug it from the command line”, that type of thing can also be created for an individual and share it to the team.

So we’re trying to bring these pieces of terminal knowledge that either live in tools like Notion, or confluence, or GitHub readmes directly into the tool, and integrate them deeply, so that they’re there, they’re easy to find, and they’re easy to use. So that’s the vision of it.

Jerod Santo: No, not at all. So day three, I was just thinking when I was writing the script to this, I was like “I’m super-angry, I want to get that across, and I will also want to give people a good overview of what the situation is and where it’s going from here.” I saw immediately, like, Red Hat - it’s hypocritical what they did. And I think anybody outside of Red Hat saw that, but there’s a lot of people internally at Red Hat that kind of can’t see that, because of where they are… But I saw immediately, like, of all the companies in the world, there’s only really one; like, CIQ can’t. Rocky Linux, Alma Linux, Cloud Linux, all these different groups that are doing the downstreams that are basically getting torn apart by Red Hat - none of them could come back with anything. But Oracle has bazillions of dollars. They could do something. So I mentioned like “Wouldn’t it be funny–” And I never thought it would happen, because I was like Oracle is –

Erik Bernhardsson: The thing that I personally spend the most time on is probably figuring out the ergonomics of the SDK itself; like, in code, how do you express programs that execute in a distributed way in the cloud, and still making it feel like intuitive and easy to the user, without having to think about the fact that this function runs in a different container than this function.

We made that work reasonably well for online inference, but I think when you go to training and start dealing with file systems, there’s certain things that are still a bit like gnarly, and I’m working a lot on that right now. So making that user experience good and sort of intuitive I think it’s really important.

On a similar note, Modal right now is somewhat janky when you run it inside notebooks, for some particular reasons; I’m not gonna get into it, but it’s something I definitely want to make – the user experience, if you’re running Modal inside a notebook, I think should obviously be… You know, we need to fix that, too. It’s fine. It’s not like terrible, but I definitely don’t think it’s quite yet where it is if you run Modal in a script.

There’s always backend stuff… We definitely need to scale this up 10x or 100x the scale we are. We see a lot of demand… Modal does not have a publicly available sign up right now. Like, you sign up and you go on a waitlist. And part of it is that we just want to have a little bit more control over the scale; there’s a lot of work we need to do on the backend to build the foundational architecture running all of this stuff. It’s a very hard problem; it’s building, essentially, our own lambdas, our own Kubernetes.

There’s a lot of work we need to do on GPU support, and in particular, cold start with GPU models, and fast-loading of GPU models. So those are some – there’s a lot of cool work, we’re spending a lot of time on there, especially when it comes to like containers, and a general like isolation of VMs. It turns out that supporting GPUs in a secure way, in a multi-tenant environment is quite hard, so we’re going very deep - I’m reading about Linux Device Drivers, and CUDA, and trying to understand all those things.

Yeah, I mean, those are all the things we’re working on. I think, in a year’s time, I think Modal will see a lot more traction for other things than just online inference. We’re gonna see a lot of people using Modal for training, we’re going to see a lot of people using Modal for parallelization… I think we’re going to have much more customers on the enterprise side; right now we’re focusing very much on the startups, but we’re laying a lot of the security and compliance work to be able to go upmarket. Yeah, those are some of the things we’re pretty excited about.

Daniel Whitenack: [13:54] I actually – so I haven’t used it a lot, I have to be honest, but I did spin up a couple of environments in brev.dev leading up to this conversation, because I wanted to understand a little bit more about it, and it was really fun. Like Chris was saying, I think it’s true that the sort of dev and onboarding experience is really nice. I was using like the UI configuration, and the experience I had was that – and I don’t know, I’m kind of curious what you’ve heard from other users, I guess is my question… Because my experience was similar to like - okay, I created the dev environment with the UI; it’s a little bit different UI than I’m used to, but there’s like familiarity with certain of the things, right? I’m pointing it to a Git repo, I’m maybe defining, like you’re saying, like a startup script, or something like that. I’m naming it, okay, it’s creating this thing, I add a GPU, whatever… There’s similarities between that and like what I would create in an instance in the cloud. But then I have this dev environment, and I think the point where something switched in my brain was - I was local in my terminal, and I actually even forget the command now, but like Brev open, the environment name, that’s what it was… Brev open, and it just popped up VS Code, and then I realized - like, I had my VS code open, and I could open like a terminal in VS Code, but that was running in the environment that I created remotely. So that’s where things switched in my brain, like “Oh, I’m now using that environment that I set up, and I could share that environment with someone else, and then they could pop open their code editor and see this.”

I’m curious, other people that you’ve talked to, people that have started using it, where are those light bulbs going off for them, and what are the things that they’re really getting excited about, I guess?

Ryan Dahl: Yeah… Mostly because I don’t have a Twitter. I need some outlet to write somewhere, and very, very occasionally I’ll dash off something tinyclouds.org. Yeah, JavaScript containers - so we’re building this serverless system called Deno Deploy, and we’re doing this as a business; this uses the same technology that we’re building Deno on. The concept with Deno Deploy is that you are not – most cloud providers, you’re basically writing Linux programs. You’re writing a Linux server. Maybe it’s in Python, maybe it’s in Go, maybe it’s in JavaScript with Node, maybe it’s in Deno… But at the end of the day, you’re packaging it up in a Docker container, shipping it off to DigitalOcean, to Fly.io, you’re using AWS Lambda, but like the abstraction layer that you’re working at is ultimately a Linux abstraction. And maybe you make that as thin as possible, maybe you make that Alpine Linux, but ultimately, you have this Unix system that you’re working within. And that starts becoming a little weird after some time, because you realize that almost everything that your application is doing is in JavaScript. Essentially, a lot of these API servers are taking incoming requests, maybe doing some authentication, making an outbound connection to a database, maybe calling out to some other APIs, kind of gluing together the responses somehow, sending that back to the user. And the Unix part of it is shrinking and shrinking, and the business logic, JavaScript part of it is growing and growing. And at some point, you realize, what’s the Linux part of this actually doing for us? Isn’t this just an unnecessary complication? And I think in particular for us JavaScript users who are used to opening Chrome, opening Chrome DevTools, having this little shell in there where you can type JavaScript is pretty similar to your terminal where you can type shell script. And shell script is kind of similar to JavaScript in a way.

[22:10] And you start thinking - in Chrome DevTools, it’s not Unix in there; there’s no real directory tree, there’s no environment variables, there’s no systemd, there’s none of this Unix stuff. You’re just in this browser world, and it doesn’t matter what operating system ultimately is underneath it if you’re abstracted into this nice, scripty, kind of JavaScript world.

And what we’re trying to do with Deno Deploy is really raise the abstraction layer for a cloud service. And so instead of providing JavaScript inside of a Unix container, we’re just providing the JavaScript. And so you can start thinking of servers written in JavaScript as being their own kind of container, rather than packaging this up with Node inside of a Docker container. As long as you’re comfortable being explicitly JavaScript, you can kind of just live in this browser worldview. And this has a lot of nice benefits, right? Your JavaScript server package, if you use Deno, is very small; on the order of kilobytes, where you essentially just zip up your JavaScript source code. And this is quite small. So our deployments happen in, I don’t know, five seconds or so… The cold start time for this, because we’re not booting up a Linux instance and starting up other stuff, there’s just less involved.

Narayanan Raghavan: [33:34] So in the context of Red Hat, SRE, especially with managed services, first and foremost we are - Red Hat as a company - putting out managed services, with OpenShift as a managed service. We have a first-party offering with AWS called Red Hat OpenShift service on AWS. We have a first-party offering with Microsoft called Azure Red Hat OpenShift, and we have OpenShift dedicated as well. So we are actually running OpenShift as a managed offering for our customers, in their accounts, in the customer’s AWS account, in the customers Google account, in the customer’s Microsoft account etc.

Now, it’s not just the platform; we’re also running other managed services, whether it’s our Kafka service, or our data science service. So for us, and for me in particular, what SRE means is “How do I build and scale a team that cuts across all of these products that Red Hat is putting out, and all of these services that Red Hat is putting out in the cloud, and being able to manage it in a consistent fashion?” Because running thousands of these services - it can quickly go to different extremes, where you’re customizing one service by hand, and doing something else with a random script etc. All of that - again, going back to my Top of Mind comment - how do I make sure that we provide a consistent experience to all of our customers, in a way that it can also scale for us as a service provider?

Jason Carter: Yeah. And in fact, it was almost trickier to build that testing framework than to kind of get the initial prototype up. And it is kind of backwards. Yeah, I think the best way to describe it is that for us, scale is the core feature that we wanted to start with. A lot of other providers in the space - they have participant living limits, or you can have a certain number of people in a live meeting, but then you have to redirect everyone else to a YouTube stream, or say, “Sorry, watch the recording afterwards.” And so we really wanted to have no limits on that; you can have as many participants as you want… You’d have to limit a little bit who can send audio and video, otherwise it would be just an absolutely crazy meeting… But we decided that that was super-important to us, and some of the initial people we talked to were really interested in it.

So to test something like that, you have to do two things. You have to scale up your infrastructure to be able to handle that amount of traffic… And there’s really sort of two things that you’re watching out for with video and audio streaming. It’s CPU load of just transcoding, or in a lot of cases forwarding the media from one user through some servers to others… And it’s bandwidth. So in order to figure out what is the cost that it would take us, per user, to run a large meeting, we had to scale up and test it. No one’s going to trust you to run a meeting of that size if you’re not able to prove that you can do it well. And so we tried a lot of different things over the last six months or so. We started out with headless browsers, so deploying hundreds or thousands of Google Chrome instances, scripting them to connect to our application, and tell some of them to turn on their cameras and microphones, tell others to just listen… We found that we were able to kind of deploy that on Kubernetes, and orchestrate lots of them… But there’s a huge resource cost just to the overhead of a browser.

[09:59] So the next thing that we did is we started going lower level. What if we can just connect WebRTC’s process called signaling - that is essentially how you get two peers in a call to know about each other, and establish a connection. So we wrote a much lighter-weight script in Python that could handle the signaling, and then optionally send audio and video from a file. And that got us to around 50,000. But again, the CPU cost was very high. And when you’re bootstrapping a startup, you can’t really afford to spin up lots of servers. So we were able to get a lot of credits in Google Cloud and AWS. There’s great programs out there for startups to get free credit, and we were able to then kind of rewrite our script in Go, to again reduce the CPU cost. And eventually, the largest test that we’ve done to date is 500,000 users receiving video and audio from a couple of presenters.

Adam Stacoviak: The closest we came so far to the solution really is what Feross and team is doing with Socket. I don’t know if you’ve heard of this yet or not, but… The antithesis essentially is that the CVE is like too late. If it’s a CVE, which essentially is what Dependabot does, right? It’s CVE related. It’s sort of like documented known issues, but they take a more proactive look at it when they look at the supply chain issue, which is like, “Have install scripts been added to the repo? Is there native code? Has there been script confusion?” I’m just reading the list, by the way… “Is there file system access? Is there network access for things that shouldn’t have network access? Is there shell access, debug access?” All these different things that could be in a dependency that wasn’t there before, that could have been a source of social engineering; “Hey, let me find a way to get your GitHub repo keys, change a thing on npm, millions of people download it, and now I’ve got bin access, or I’ve got a shell access, or network access to this thing that never had it before. Now I’m in your cloud, or wherever I’m at, and I’m doing my thing.”

They’re taking a more proactive look at it, which I think is pretty interesting. It’s the most interesting thing I’ve seen thus far in the supply chain attack issue. The only issue I see really is that they’re only focused on JavaScript right now. So there’s some things we talked about Feross with on this show, in that episode. I think once they get past JavaScript and they do open source at large, Or Rust, Go etc, that things get more interesting. But JavaScript is a big footprint, but it’s not all of it.

Solomon Hykes: Yeah, so the development flow that Sam described, what you’re doing there is you’re writing – that’s when you’re writing a configuration for Dagger. So you’re telling Dagger how to deploy your stuff. And without Dagger, you would typically do that in a bunch of places, which is one of the problems. We can’t do it just once and then run it everywhere. You probably have to repeat and duplicate information and fragment it. It’s gonna deploy from your laptop. You’re gonna have shell scripts, a Docker Compose file maybe, a makefile, a custom Python script, some Ruby scripts, a custom JavaScript script…

There’s a lot of custom and reusable tools out there for deploying from your laptop. Then sometimes you’re gonna reuse the same scripts on a deployment server, a staging server, maybe you’ll bring it into your CI… But then what happens in your CI is that CI wants to be CI/CD; because if you only do CI, there’s not enough money to be made, because CI is infrastructure. A CI system is a runner for your scripts everytime something happens on source control; a very valuable thing to do. But there’s what like 100 of them. So now, what all these CI systems are doing is adding more and more sophisticated pipeline systems. And a lot of those are configurable in Yaml. You write a Yaml description that then says “Run this script” and “Run that script” and “Connect these things.” They all have a different system, but they have in common that they use Yaml, which is an awful development experience… And also, it’s not your shell script; it’s different. So now you have two things.

And then what happens is your CI/CD process is in place. It uses Circle CI, GitLab, GitHub, Jenkins, whatever, and it’s very, master that Yaml thing, and you update it and you add – now, sometimes there’s an Action thing on GitHub, for example, a Docker container you plug in… So you start kind of adding things, and now all of a sudden that doesn’t work on your laptop anymore. And also, you can’t just look at the code for it. You’re looking at a Yaml file, and that Yaml file says “Run this container.” So you’re running a container, but that’s a binary thing. So now you’re gonna go look for “How was that container built?” “Oh, here’s the Docker file. I’ve found it. Oh, this one is a Python thing. It uses the APIs for this particular CI system, so it’s not portable.” “What if I have a different CI?” Well, you’ve gotta start over.

[24:20] So you have this fragmentation problem where the actual deployment logic is split up into lots of different pieces, using different languages… So you can’t reason about it as a whole, number one. And each piece is tied to a specific runner, a specific piece of CI infrastructure. So when you’re writing a configuration for Dagger, you’re doing the same thing once more. You’re writing a configuration that describes how to deploy. But the big difference is, 1) it’s a better development experience, because it’s a language that’s better than a shell script, and also better than Yaml. It’s sort of like the best of both standard, imperative programming and the best of a declarative system like Yaml. So that’s kind of – like you said, it’s a building block. 2) There’s reusable packages. So if someone is really smart - maybe yourself, you wrote a piece that you need… It could be a pattern, it could be an automation, it could be integration of a tool, TerraForm, as Sam mentioned, Argo CD, whatever, over time an ecosystem builds and you can reuse those. That seems obvious, even trivial for application developers. But as we know, in DevOps and cloud land, that actually does not exist, amazingly. Not fully. We’re in the uncanny valley of delivery as code. It kind of looks like code almost, and it makes it weird, but it’s definitely not as fun to do. Writing all these Yaml files, templating them, and copy-pasting shell scripts is not as fun. So it’s a better experience, and you can run it anywhere.

Dave Farley: [35:43] I think that the gestation of the cloud was kind of in parallel with the starting points of continuous delivery. We published our book in 2010, and that kind of put a name on these practices, I suppose. And people these days talk about continuous delivery and continuous integration and all these sorts of ideas… I think we helped to popularize that through our book. But we’d been doing it for several years before that. And certainly in the early days, the cloud wasn’t around, so all of the projects that continuous delivery began with were cloud projects.

I think that the cloud makes some of the continuous delivery thinking more obvious. I mean, you’d be absolutely insane to be an organization like Amazon or Google and to manually configure your servers. It’s such a bizarre idea; it’s just laughable. You couldn’t do that. You’re gonna automate that, unless you are crazy.

So ideas like infrastructure as code just seem obvious in the cloud. And they didn’t always seem obvious to other people. People thought we were strange when we started automating those things, and making our servers in our data centers more repeatable and reliable. There’s that kind of stuff.

If I’m honest, I don’t think that cloud had a huge impact on the kinds of projects that I was working on during that time. Certainly, up until after the continuous delivery book came out. Of course, like everybody else, it has an impact on me now in the way that I think about these things and the way that I advise my clients how to approach solving problems.

I am, I suppose, an old school developer. My formative years in software development probably predated the cloud… The cloud is obviously a good thing; it gives us lots of opportunities. It commoditizes compute, but to my mind, it’s not a fundamentally defining thing. It certainly changes some of the dimensions of design that I care about, the way that I would think about design…

One of the things that seems to me that changes is the economics of the design from an architectural point of view. In the old days, when we were doing stuff in our own data centers, we were probably more worried about managing storage, because storage was expensive. And that’s largely become commoditized. And the price per bytes of storage has kind of just dropped through the floor with the introduction of Moore’s Law and the introduction of cloud-based services. The big difference with the cloud now is that the unit of cost is really around compute. That’s certainly if you start thinking about serverless things. So that ought to change the way in which we apply our design thinking. For example why bother normalizing data anymore? Why not just shard it out and make everything separate to optimize the compute cycle? You could do that. We could have processes sucking in data and just allocating them out in a way so that they’re all more parallelizable… And I think that’s kind of interesting, that it has those sorts of things. The easy access to be able to spin up some compute resource or storage resource or whatever else is fantastic… And the ever-raising of the bar of abstraction that the cloud services add is kind of interesting. And I think there’s much more to come.

One of the things… We were talking about the continuous integration and continuous delivery tooling - I don’t think we’re there yet. I think there’s more to do. I would like to see tooling that’s more opinionated. I would like to see tooling that just – bang, gave me a deployment pipeline. If I follow the rules, it’s just going to run my unit tests, run my acceptance tests for me and deploy into production. That’s doable, it seems to me. We could do that. I’m hoping to see the continuous delivery cloud vendors do more of that kind of thing - be more opinionated.

[39:43] One of my favorite technologies - we were talking about tech earlier on - is Gradle for build systems in the Java space. And one of the things that I always loved about Gradle is that if you don’t care, if you just are willing to buy into its model, you can write your build script in one line. You can just say “I’m doing Java” and it’ll do it for you. It’ll compile to Java, it’ll run the tests, it’ll do all of those things for you. But if you want to override almost any behavior, you can do that, too. It’s a whole programming language built on top of this well-designed domain model for builds, is what Gradle really is.

I like opinionated software. I like opinionated software that says “Do it like this”, and if you don’t like it like that, it’ll get it out of your way really quickly. I’d like to see more tools like that, because I think there’s a tragedy of the commons kind of thing goes on a little bit. If everybody has a choice, everybody’s rediscovering everything from scratch, and I think we ought to be able to build a little bit more things and the cloud is one of those things that is doing that in some context. It’s not doing it enough to my taste for build systems.

So I am currently in the middle for a piece of work that I’m working on to demonstrate how to build deployment pipelines, building a little sample application using GitHub Actions. It’s alright. It’s nice. I quite like GitHub Actions, it’s okay. But it’s too fiddly. I’m fighting with it, trying to get my Docker images to communicate with each other so I can run my acceptance tests, and this sort of thing. I’d like something that just worked. I’m not really interested in that part. I’d like something that just works if I wanna do something simple.

Dave Farley: [35:45] I think that the gestation of the cloud was kind of in parallel with the starting points of continuous delivery. We published our book in 2010, and that kind of put a name on these practices, I suppose. And people these days talk about continuous delivery and continuous integration and all these sorts of ideas… I think we helped to popularize that through our book. But we’d been doing it for several years before that. And certainly in the early days, the cloud wasn’t around, so all of the projects that continuous delivery began with were cloud projects.

I think that the cloud makes some of the continuous delivery thinking more obvious. I mean, you’d be absolutely insane to be an organization like Amazon or Google and to manually configure your servers. It’s such a bizarre idea; it’s just laughable. You couldn’t do that. You’re gonna automate that, unless you are crazy.

So ideas like infrastructure as code just seem obvious in the cloud. And they didn’t always seem obvious to other people. People thought we were strange when we started automating those things, and making our servers in our data centers more repeatable and reliable. There’s that kind of stuff.

If I’m honest, I don’t think that cloud had a huge impact on the kinds of projects that I was working on during that time. Certainly, up until after the continuous delivery book came out. Of course, like everybody else, it has an impact on me now in the way that I think about these things and the way that I advise my clients how to approach solving problems.

I am, I suppose, an old school developer. My formative years in software development probably predated the cloud… The cloud is obviously a good thing; it gives us lots of opportunities. It commoditizes compute, but to my mind, it’s not a fundamentally defining thing. It certainly changes some of the dimensions of design that I care about, the way that I would think about design…

One of the things that seems to me that changes is the economics of the design from an architectural point of view. In the old days, when we were doing stuff in our own data centers, we were probably more worried about managing storage, because storage was expensive. And that’s largely become commoditized. And the price per bytes of storage has kind of just dropped through the floor with the introduction of Moore’s Law and the introduction of cloud-based services. The big difference with the cloud now is that the unit of cost is really around compute. That’s certainly if you start thinking about serverless things. So that ought to change the way in which we apply our design thinking. For example why bother normalizing data anymore? Why not just shard it out and make everything separate to optimize the compute cycle? You could do that. We could have processes sucking in data and just allocating them out in a way so that they’re all more parallelizable… And I think that’s kind of interesting, that it has those sorts of things. The easy access to be able to spin up some compute resource or storage resource or whatever else is fantastic… And the ever-raising of the bar of abstraction that the cloud services add is kind of interesting. And I think there’s much more to come.

One of the things… We were talking about the continuous integration and continuous delivery tooling - I don’t think we’re there yet. I think there’s more to do. I would like to see tooling that’s more opinionated. I would like to see tooling that just – bang, gave me a deployment pipeline. If I follow the rules, it’s just going to run my unit tests, run my acceptance tests for me and deploy into production. That’s doable, it seems to me. We could do that. I’m hoping to see the continuous delivery cloud vendors do more of that kind of thing - be more opinionated.

[39:45] One of my favorite technologies - we were talking about tech earlier on - is Gradle for build systems in the Java space. And one of the things that I always loved about Gradle is that if you don’t care, if you just are willing to buy into its model, you can write your build script in one line. You can just say “I’m doing Java” and it’ll do it for you. It’ll compile to Java, it’ll run the tests, it’ll do all of those things for you. But if you want to override almost any behavior, you can do that, too. It’s a whole programming language built on top of this well-designed domain model for builds, is what Gradle really is.

I like opinionated software. I like opinionated software that says “Do it like this”, and if you don’t like it like that, it’ll get it out of your way really quickly. I’d like to see more tools like that, because I think there’s a tragedy of the commons kind of thing goes on a little bit. If everybody has a choice, everybody’s rediscovering everything from scratch, and I think we ought to be able to build a little bit more things and the cloud is one of those things that is doing that in some context. It’s not doing it enough to my taste for build systems.

So I am currently in the middle for a piece of work that I’m working on to demonstrate how to build deployment pipelines, building a little sample application using GitHub Actions. It’s alright. It’s nice. I quite like GitHub Actions, it’s okay. But it’s too fiddly. I’m fighting with it, trying to get my Docker images to communicate with each other so I can run my acceptance tests, and this sort of thing. I’d like something that just worked. I’m not really interested in that part. I’d like something that just works if I wanna do something simple.

Kelsey Hightower: It’s funny, people complain about YAML, like “Oh my god, YAMLs… So terrible.” And you have to step back for a second. I’m gonna just talk through my journey of this to get to where I get to the concept of infrastructure as data. If you have five Linux machines, the first thing you’re gonna try to do is write some Bash scripts. You’re gonna write some Bash scripts, some for loops, you’re gonna SSH things around…

[12:09] And if you’ve ever seen people write Bash scripts over time, you go into their home directory and you see files like “Do this stuff 01. Do this stuff 002, but don’t use it anymore, because you should be using the other script.” [laughter] So you have no version controls, you have no semantics, no abstractions. You’re just writing Bash scripts.

Fast-forward to configuration management. We get things like CFEngine - big shout-out to Promise Theory - and then we get Puppet, Chef and Ansible, and then they formalize. It’s almost like the Ruby on Rails for shell scripting. So now we have this configuration management error and we all start to say “Infrastructure as code.” The problem is now you have to test it, people can write any code they want, it’s unbounded context, and it’s the same problems we had with software - how do you secure it? You’re gonna have bugs… it’s just all over the place. But it is a better place to be than we were before.

Now let’s get to infrastructure as configuration. Now we’re removing all of the abstractions into the runtime, so the how. How do you create a load balancer, what goes in the load balancer and how do you remove it? That implementation detail - we’re gonna have a lot of discipline and we’re gonna move it into these controllers. So if we’re talking about Kubernetes, these are gonna be the controllers. If you’ve been in cloud, you’ve already done this. We always have control planes that do the heavy-lifting. Same is true for networking - we expose ports and protocols, not the control plane.

So configuration as data - we get to something very similar. Now what you say is “I want a load balancer in this region, pointed to those services. No for loops, no if statements, no programming language concepts.” So all you have is a data model. And that data model represents everything that the state machine on the other side can do. Why is this more powerful? Well, when you’re working with data, then you can manipulate the data much easier than you can manipulate code.

We’ve seen this before, in the Go world - there’s 10,000 Hacker News posts, “Oh, the same thing, but written in Go.” I prefer Go as my favorite language. So every time we do things in a language-specific way, we end up having to rewrite this thing to be compatible with those libraries, and so forth. But when you move to infrastructure as data, we can have these high-level APIs. You can write them in JSON, you can write them in YAML, or if you’re an enterprise, you can go XML if that’s your thing…

Dan Guido: Yeah, pretty much. We wrote the readme so that it is simple enough that if you can open a terminal and copy and paste what’s in the readme into the terminal, then you can set up Algo VPN.

Basically, at a high level, the steps include - on Mac or Linux, or Windows if you have Windows Services for Linux - you download one of the releases, you unzip it, you CD into it directly, and then you install some dependencies, Python if you don’t have it, and then you just run the install script. Once you have those dependencies, it’ll just work.

The install script will ask you some questions, like “What do you wanna name the server?” You have to give it a name. It has a default name, and you can just press enter. You have to give it an API key, and that means the one cloud service that you’d like to deploy it to - you need to go log into that cloud service with your account there, and then grab an API key from that service, that lets the Ansible scripts do things to it.

[39:41] Now with that API key Ansible and Algo VPN will start up a server, add the VPN software to it, generate the keys needed on your local machine, send those keys over to the remote server, and then lock the whole thing down. And locking it down means a lot of things. it means setting up AppArmor policies for each service, it means reconfiguring a couple of Linux defaults to be a little bit more secure, changing some file permissions in places, some kernel parameters in places, setting CPU accounting to make sure that certain services can’t run out of control…

And then at the end of it it prints out a congratulations message that says “You’ve got a new personal VPN, and here’s the key for it.” At the end of that message, now you’ve got a bunch of files that are like preconfigured profiles for the VPN users that you wanted to create. So those profiles are things like an Apple profile, so if you have an iPhone, you can take an Apple profile, send it over to your iPhone, and then now you’ve got your VPN on your iPhone.

We usually suggest people Airdrop those, because that’s kind of an encrypted local communication between you and your desktop.

It also has WireGuard profiles. WireGuard profiles are cool, because they’re QR codes, so yo don’t have to airdrop anything. You can just take your phone, hold it up to the screen and scan it, and then you’ve got your configured WireGuard VPN on your phone.

James Fletcher: To close down what we were talking about just there, the whole learning approach that we’ve been building, and all of the research that we do on top of knowledge graphs - I’ll emphasize that - we release all of that as code available via our GitHub. Specifically, we have a library called KGLIB. That’s our knowledge graph library for machine learning.

KGLIB is a sensor of those projects, and the main one that we’re running right now is Knowledge Graph Convolutional Networks. That’s how we apply those learners on top of both the reasoner and the knowledge graph shaped data.

The starting point is how do you actually get a knowledge graph, right? How do I actually get my knowledge graph together? Now, the components that you have there, as you pointed out, is something we should start with. So you have Grakn itself. Grakn Core is released open source, you can download that from GitHub, or install it with a package manager… And that’s a database which is gonna run – you can install that on your local machine, and get it up and running, or put it in the cloud. So you need that back-end service running.

Now, when it comes to actually accessing that, we have three officially-supported drivers at the moment. We have Python, Node.js and Java. We make sure that all of those are up to date and working with the latest Grakn. What’s really interesting there actually is the communication protocol between those clients and Grakn. It’s called gRPC. So that’s something from Google, Google’s remote procedural call, that has replaced using REST services.

What’s really nice about this, and the actual end goal that that gets you to, is it means that when I’m accessing the database with Python, I get to actually use native Python functions. All I have to do is import the package that talks to Grakn, import the Grakn client in Python at the top of my script. Then I can just instantiate a communicator that will talk to Grakn and make queries to the database just out of my native Python. I can just launch them straight from my application, and it doesn’t feel like you’re talking to a database anymore. It just feels like you’re making function calls, which comes back with information that’s pertinent to your knowledge graph.

Stevenson Jean-Pierre: Use cases for me have always been very small microservices things that aren’t even worth spinning up infrastructure to run. It could be maybe like a 50-line script that does some specific functionality. Also, like I mentioned earlier, the event-driven stuff things that are dependent on some event happening before it fires off… Another great use case has been as a just general cron replacement. You always have the issue with having highly-available cron, and having multiple services with the same schedule and not stopping over each other, and you’d have to implement weird locking mechanisms… But by having serverless functions, you could depend on the higher-level timer from the cloud provider, and you could have a single cron source.

[08:04] And for not so good use cases - anywhere that I’ve had to maintain state, or maintain some kind of cash for speed, and things like that; serverless is not conducive to state, as it actually forces you to be very stateless, unless you wanna go to some network storage… So any of those use cases where long-term state on the app tier is important, serverless hasn’t been good for me.

Francesc Campoy: [20:02] Yeah. [laughs] No, but once you tell them – many banks do not really even know how much code they have. There’s so much of it that when you tell them “Okay, so how much COBOL do you think you have?” and they’re like “In-between 100,000 and maybe half a million…” I was like, “Well, if you’re going to put some budget to go and rewrite that in something more modern, good luck, with that estimation…” So the idea is we’re gonna be able to bring all of these data to them, so they’re gonna be able to make informed decisions. Counting lines of code per language, which for us is literally a “group by” query, it’s super-simple to do… For them, it was like “This is actually really interesting.”

The other change that lots of banks want to do is going back to the inner sourcing. Large banks, they have many IT groups all around the organization, and they want them to work together well, and the first piece is to figure out who is doing what, what resembles to what, how much code duplication you have… We have a thing that analyzes code duplication not character by character, but rather extracting the abstract syntax tree, modifying a couple things, so it’s actually a very smart way of figuring out whether two pieces of code are very similar. They’re so similar that if you saw them next to each other, you would say “You need to refactor them and just write one function.” We’re able to detect this automatically, and this helps a lot, because if you imagine you’re like the CTO of a bank, and you have this codebase that dates from the ‘60s, and they tell you “Please put it on the cloud.” That’s hard. That is a harsh thing to ask to anyone.

The idea of being able to tell them, “Well, actually all of this source code - let’s see which parts are gonna be the easiest ones, there’s MTA, “modernizing traditional applications”, which is not really cloud-native, but we can make it be cloud-native, we can make it run in Kubernetes… And then what is the COBOL that – you know, that’s gonna be an interesting challenge to migrate. So having a view of all of this by just running a couple queries is really powerful. The other option is literally running – I’m gonna be very hopeful and say that you could run a really huge Bash script calling Git very often, and maybe you will get something similar… But it would take hours instead of seconds.