Stephan Ewen: Yeah, yeah, yeah. Yeah, something like that. I think durability is probably the same as persistence maybe, with a bit of a stronger emphasis on it really doesn’t get lost after it happens. So durability is the D in ACID when it comes to databases. Databases say “We’re giving you atomicity, consistency, isolation, and durability. Once you do an update, we’re not going to lose it.” No matter what crashes, the database has a mechanism to bring that change to the database back. If I told you I’ve recorded that row, I’ve recorded that change, it will be there, no matter what. And in the context of Restate, that doesn’t mean – for example, the core building block of Restate is a stateful, durable function. You can think of it like that. And the stateful, durable function, when you schedule an invocation for that, or as you go through the code of that stateful, durable function, it has like multiple steps, recording a step. Whenever you go beyond a step that you asked Restate to treat as durable, you know that no matter what happens, you will never re-execute that step. You’ll never come up with a different value. Like, if your machine goes down, the Restate server goes down, if you deploy it across availability zones, the data center goes down, the network gets partitioned, whatever, you’ll never ever go back and re-execute that step if it once told you that it’s done it. TThat’s sort of the meaning of durability. Once it says it’s there, it’s always going to be there. And I think this is, in a way, almost one of the magic ingredients.

The way Restate looks at making distributed application development simple… I’d say there’s two core pieces that you need to think about. One of them is the durability; make durability extremely fine-grained, and extremely cheap. Because if you can apply durability in fine-grained steps, you always have to worry about very little after a failure. Let’s say your durability is coarse-grained. Let’s say the other workflow is one durable step, and it crashes in the middle. It gets retried. It’s up to you to figure out “Well, did I actually process the payment already or not?” Maybe there’s a way to just like assume “Okay, it’s idempotent. I can send it again.” Or I might even not be able to ask the service “Did I do that or not? Did I actually decrement the available kind of product already or not?” Maybe I have a way to, again, make this durable, or not. I don’t know. These things tend to be harder than one thinks, because sometimes the API gives you – you know, it might’ve given you an error back the first time, and you thought “I didn’t do this” and followed some control path flow… And then the next time you actually get, not an error, but the real result, and then you follow a different path… So people mess up this all the time. It’s really hard to reconcile if you have these multiple steps as [unintelligible 00:35:08.20] atomic unit. “What did I do? How did I do it the last time? How do I recover from this?” But if you have extremely fine-grained durability, if you’re recording every individual step as durable in the system, and when it comes back, it can tell you exactly like “This was the last step that you recorded”, then you just have a very small amount of uncertainty. “Okay, here’s this one thing that I might have tried already. I have to just worry about that bit”, instead of the whole history, and possible control flow, and all the choices, how I might have ended up here that I need to reconstruct in order to proceed consistently from there. So just like very fine-grained durability is extremely powerful in simplifying things.

I’d say the second magic ingredient is then how do you anchor this in the whole retrying and resolving potentially inconsistent situations with partitions, with timeouts, with zombie processes, and so on, so that there’s always a very consistent view of what the last durable step was. I think that’s the second sort of ingredient of Restate. It’s not just durability, it’s actually durability and consensus, and giving you a very, very crystal clear view on where you left off, where you need to continue from. I think if you take those two things in conceptually, you’ve simplified the problem massively, and the rest is almost API sugar that you built on top of that.

[36:27] That’s, I would say, the magic that happens in the Restate runtime. It’s a very low latency, durable consensus log that fuses queuing, state management, locking, fencing, creating futures, resolving futures… All these kind of operations that tend to be part of a distributed coordination process.

Adam Jacob: Someone’s using BladeLogic. But you can sort of force those – if the technology has enough compelling value, people will replace it over time. But yeah, our goal is that we definitely – I lived that experience with Chef, where… I didn’t love how long the adoption cycles were, and how all or nothing it felt. And so yeah, we’re hoping that part of that adoption cycle is actually just – it’s going to start with just importing the things you already have, that you want to deal with, and then over time you could track just the resource side of that and be like “Well, I can show it to you in a diagram, and we can see what its resource value is.” But then it sort of starts to make sense that like “Well, why wouldn’t I also manage it through here? Because it’s here, and it’s easy, and it’s straightforward.” I think that’s probably how that goes.

There’s all sorts of interesting possible futures, where you do like automatic discovery of things… But there’s a lot of interesting problems with how do you lay it out, you know? And – anyway, there’s a lot of interesting stuff hiding inside there, but like…

Adam Stacoviak: The future’s coming, and we’re going there.

Daniel Whitenack: Yeah, well, I would hope that in all of our futures, as we have medical procedures done increasingly by AI or AI-assisted, and other things like that, that those things are very secure, as they happen; that would be very important, I think.

Paul Copplestone: As in what we would probably compete with is more cloud services on their data products. And you can use all of those; like Django, you can use it with us, and you can tie in… But our kind of overall mission is that if you’ve got data, it gets stored on Supabase. It could be OLTP data, or later on some OLAP data. We’re not fussy with how – for example, we’re very clean with our integrations. We’re very clean with our open source and protocol support. Our storage engine supports the S3 protocol, so you can plug anything on top of it, and then the services that we provide also work on top of that. So the idea is – for example, I’m a big fan of Iceberg, Apache Iceberg, the catalog format for OLAP. We’ll probably do some tooling around this inside our S3; I don’t know, so these are all futures where we’ll let our users drag us. But that’s where I can see some pull coming from our customers. They have a ton of data that they need to store, and it doesn’t always fit Postgres. So where are they going to put it? And we’ll help them solve that question.

Kris Brandow: These sound like they could be interesting hybrid game shows… Just a little bit of fun. Alright, alright, so before we move on to unpopular opinions, assuming – I mean, Johnny already said he has one, so we’re good to go. Assuming that we’re gonna go into unpopular opinions, any last words about Go Time, multiple choices, or our futures out there? Any closing words, closing thoughts?

Adam Stacoviak: Yes, yes. Well, I think we’re getting close to the length of – we’re beyond what I thought we would actually go in terms of timing. The only question I had left for you, and I think you may want to answer this, because you said that you wanted to talk about the other topic, which was the business of YouTube… What are your plans for the future? I mean, you do a great job with your content. I appreciate the journeys you take to create the content you do, because in a lot of ways you’re the recon; you’re the recon team, and I’m the follow-up team. I’m just watching what you’re doing. Some things I just learn from, like with the PiKVM. I don’t really leverage that really much. I don’t have a lot of headless things where I’ve got to deal with that. I SSH into most things I need to, so I don’t need a visual for almost anything really, unless I have to… And maybe then I’ll go back and watch it, and maybe implement a PiKVM, or something like that.

[01:24:15.05] Proxmox - you’ve helped me get into that, obviously, considering low-power consumption… But things like that. So what are the futures for you in terms of like the business of YouTube? Do you plan to grow a team? Do you have an editor? Will you build an empire? Will you be the next Linus? What’s going to happen? What do you want to do? What would be ideal for you?

Mat Ryer: The future’s in the forward, so how can you go back in it?

Tim McNamara: …that we want to create a pathway for teams that are looking to adopt Rust, to be able to give them a path forward… Because most services at Amazon are written in Java, and teams choose their own tools. So there’s no way to sort of centrally dictate that such and such a team or such and such service must be implemented in such and such language. Instead, we kind of need to work kind of slowly, organically, with teams that are interested, but trying to work through their pain points with the language and with the ecosystem.

And one of the things that we’ve found is that teams that are looking to adopt Rust will typically have one or two people who have been tinkering with the language, and they’ve found that things run really quickly, or that they save a lot of memory, and that they reduce the operational burden… So their systems are more stable, even in a prototype stage. But then there’ll be a point at which they hit this kind of organizational inertia, where at like the engineering manager level, or let’s call it the middle management tier, will suddenly – the immune system of the organization will respond. And it will say, “Whoa, whoa, whoa, whoa, whoa… This is making me really nervous. You’re implying change, and change is very dangerous.” But at a more senior management level, some of the – Rust is being talked about so pervasively now that it’s impossible for senior leaders to ignore it, and they’re really sold by performance benefits, by safety benefits, by security.

[10:17] And so these are kind of the internal dynamics, that we have teams that are very optimistic about their futures, and they’re keen to experiment; let’s say one step above or beyond them might be a little bit of resistance, or a skepticism of a lot of the claims. The Rust community seems to make these ridiculous, audacious claims, like it will save you, let’s say – actually, I’ll talk about my own ridiculous, audacious claim, which is that “Why don’t we as a software industry reduce our energy use by 50%?”

Tim Stiles: So this is a conversation I come into a lot. And actually, I have friends who - we are like polar opposites. It’s one of those things where we both agree where the future’s going, but we disagree on how the future’s gonna get there, and what the practice of it is going to look like. So for me, the practice of it in my mind is, you know, software engineers who have a decent enough biological experience, it’s easy to learn, hard to master, all the theory behind it. It’s a lot of little tiny variables, which is - computer scientists and engineers, we know; we have lots of tiny little variables all over our code. We’re aware of this. We’re gonna have to keep track of lots of little tiny details. But there’s this whole breadth of research, and every once in a while I get dragged into a field that’s adjacent, that I know nothing – like, I had to learn about immunology recently. Immunology is like its own thing, where it’s heavily related to like genetic engineering practice, especially with these new immunotherapies we’re doing for cancer, and stuff. But I had to go and learn about immunology as like a biological basis in like human medicine.

But the thing is that we’re gonna get to a point where hopefully there’s enough software engineers with biology experience that can talk to these people that have this experience in more pointed, niche parts of the field, like immunology, or plant physiology, or these different biological topics, where they can work together and some will say, “Hey, I need to go probe this.” And they’ll talk to the engineer, and the engineer will get it enough that they can work with them. And that’s what I would like to see, is two scientists with like 20 software engineers, writing some awesome code, doing stuff.

And then my friend - he’s entirely the opposite. He thinks we should have a few software engineers writing - not GUIs, but kind of like that; you know, report generators maybe. And there’s like 20 scientist analyzing them. We have entirely different outlooks on how it’s going to be. So what I’m betting on is that in the future we’re going to have a lot of software engineers coming from languages like Go, and Rust, coming into biology and sort of speeding things up.

One of the things that’s fun is that every time I rewrite an algorithm from Python into Go, it’s like 25 times faster by default. We have the world’s fastest DNA synthesis-fixing function by several orders of magnitude, which is kind of nuts. Like, we weren’t aiming for that. That’s just kind of how it happened.

So there’s a lot of work to be done there, and I think the real impetus for most software engineers is the biology is intimidating. And I think the one thing if there’s a software engineer listening to this right now who’s interested in biology - don’t be so intimidated by the biology. Seek to learn it and be humble when you ask questions to scientists, because they’ve gotten a lot of physicists and software engineers over the years coming into the field and acting cocky because they see this code that they’ve been working with, and they’re like, “Oh, these guys don’t know how to code. They don’t know nothing.” It’s like, “No, they know something. It’s just they’re not software engineers.” And you’ve really got to listen to them. You’ve got to be humble; you can’t come in with this attitude that you’re going to solve all their problems, because you’re not.

My favorite is – do you guys know the three-body problem? …like, this classic problem of physics that’s been around forever…

Adam Stacoviak: Yeah. Part of me looks at this too that… I’ve been listening to a lot of what I would consider plausible science kind of books… Dennis E. Taylor Andy Weir, a couple other authors I could name off - they very much talk about the possibility of humanity, everything from artificial intelligence, to sentient beings, to Voight Kampff machines that actually can replicate themselves, and build, and mine based upon planets… Some really interesting, long-term fetching things. And I look at this as the initial spark of this long-term possibility. And you can see it short-sighted and say “This is the negative impact to, say, artists now.” But how will humanity be able to move, I guess, into the far, far future - which is inevitable, The future is inevitable. Time is linear. We won’t go back, we’ll only go forward, so the future’s coming, no matter what. How do we look at something like this and press it down, or ban it, or push it away, when we can look at the long-term humanistic opportunity versus the short-term individualistic opportunity?

Roger Peppe: That said, I think it really helps to do a lot of thinking through. You can actually design stuff; it might not be set in stone, but I do think that you can eliminate a lot of bad code paths or bad futures by just thinking them through to start with, and thinking, “Well, if we went that direction, where would we end up? That’s a bad place to go.”

I think a lot of people are like “We shouldn’t do any design upfront.” I really don’t agree with that approach at all. If you look at Go, for example - they designed a lot. They wrote the spec before they did the implementation.

Solana Larsen: [07:49] I think that there’s so many ways, so many directions you could take an answer to that question… But I definitely think there is this kind of exclusionary, kind of it’s magic, there’s an element of dark arts to it, this mystique around AI that really serves the people who use it to exert power in different ways. And so when we’re asking for AI to be more transparent, and more understandable, it’s partly about demystifying to an extent that we can actually get to the heart of “How do these systems work? How does it affect me? What can I do?”

And so yeah, I think that kind of obscurity, and also elevating it to this higher art form that normal people can understand - I think that’s part of where the power lies. You have that with other forms of knowledge and power systems as well. So a big part of what we’re doing is bringing it down to a level where we’re explaining how it works, how things can work differently… And oftentimes, and which is what we have in the podcast, is people explaining how they were harmed by a system, but they’ve decided to design it in a different way, to do something different. And sometimes it’s through those stories of just people building something, even if it’s on a smaller scale, building something, and that gives you this kind of realization, “Oh yeah, I was just completely taking for granted that we have to collect data in this way, or that we have to ignore privacy in this way.” It just opens your horizons for the hoping and the dreaming.

And I think even when we’re talking about data futures, or AI in the future, a better future, what we don’t want to get into is “Oh, robots…” It’s not like SciFi future really, it’s more like you and me real-life kind of future. Because even though we’re talking about very advanced technology in some cases, it affects people who aren’t even on the internet. It affects you when you’re walking down the street. There are all kinds of ways that aren’t very high-tech, and just very basic daily life where you encounter these technologies. And so yeah, it’s bringing it down to the ground level where we can talk about it, and where we can also approach it with grassroots communities when it’s appropriate.

Eran Yahav: Absolutely not. Not even close. [laughter] I think it’s totally unrecognizable. Some of the core technology is still there, but it is utilized in completely different ways. So what I showed him back in the day was being able to take a huge codebase, train on the codebase and generate a model that if you give him a small prompt of a part of a program, it gives you a larger program that contains that piece. And so it’s like a super-sophisticated code search based on ML models. That’s what it was at the time. So it could be utilized to do a similar task to what Tabnine is doing today, which is kind of I give you the first three lines of the program, you give the name of the next three lines; but it wasn’t even close, again.

It’s funny how these things evolve, because a lot of challenges– the bottleneck for a lot of these processes is actually the human, which is surprising, right? So you have this very tight loop of human and machine who are writing a program together. This is what’s going on in Tabnine and in other synthesis systems. And the limiting factor is the human, because the human in this tight loop has to say, “Oh, yeah, that’s what I want. Oh, yeah, that’s what I want. Oh, yeah, that’s what I want.” And only the human knows the intent. And so the machine can generate this massive map of what could be possible futures, and the developer has to navigate through these maps and say, “Yeah, that’s what I want.” And it’s very tricky to get the granularity of that interaction right… Like how much code should I generate?

Adam Stacoviak: It’s a good stepping stone. Well, a stepping stone is easy to get to. That’s the whole reason it’s a stepping stone. And I think I want to frame clearly why I said they think it’s their futures, is because they’re not intending to move to something– they don’t have their next stepping stone in sight, based upon our current conversation. Of course, 5, 10 years from now, probably, and maybe not, who knows… But they’re not seeing it as a beta stepping stone, or “We’re going to try this, and maybe it’ll work.” It’s like, “Okay, this is the way right now.”

Adam Stacoviak: One part of the conversation that we actually had - and Paul was a part of this as well -was the Amazon versus Elastic essentially, this idea… And one of the opinions shared there from Adam Jacob was around essentially AWS being the marketing funnel for an open source project, because they’re just so massive and just so big… While they may compete with you, they’re also giving you extreme brand distribution. So if you can win… I even recall Paul talking about how with IOx and some feature products that we might talk about here on the show today, that you’re by design designing some of the futures of Influx data, by design, to be competed with… With InfluxData Cloud, for example.

So I think there’s some disadvantage there, but I think it’s also important to talk about the advantages of AWS choosing Elastic or some sort of other thing. And sure, there definitely is some misuse there, but I think there’s also an opportunity to say “That’s a significantly larger audience that hears about my thing, that would not have heard about it…” Because open source is all about adoption, and adoption requires distribution.

Daniel Whitenack: Yeah, for sure. Maybe I’ll just illustrate what I’m talking about with one of these articles, which I think is a good starting point for this discussion. It’s an article called “An electronic brain. Naming, categorizing, and new futures of AI.” That’s a guest post by a doctoral researcher at Oxford, Yung Au. Sorry if I mispronounced your name. This article I thought was just fascinating…

Jerod Santo: [laughs] So there’s your top five most unpopular opinions to date. We also have some runners up that we will just briefly here mention… On episode 167 Ian Lopshire said that he thinks futures have a place in Go. 76% of people disagree with Ian on that point.

One episode 183, Preslav Rachev said that Go needs more magic. I did not expect that to be popular, and it was not. 75% of people disagreed with Preslav on that point, and that actually generated a nice conversation on Twitter about such things… Which we’ll link to, of course, on this episode, so you can go read that conversation. Quite a debate around that point.

And then somehow, Mark Bates got himself onto the show, and on episode 171 he confessed that he doesn’t particularly like bacon, which was also 75% unpopular.

So that is our top five most unpopular opinions… Baseball - the most exciting. Err, hard to read. Chocolate kind of nasty. JS Party is better, and paste with formatting - a default.

Break: [23:42]

Kris Brandow: I think there was a proposal we talked about… What was that, last week? Yeah, last week… That had this kind of deferred processing – I think it was called “eval”, that is this similar concept to this, where you can actually make it so a function doesn’t execute unless it’s actually used… Which I think is kind of related type of functionality… But also slightly different. But I could also see how that could easily be abused to make futures and put them everywhere, and have Go be inherently less readable.

Andrew Burt: Yeah, so I would say – and it’s been interesting, because we also coming out of the gate have been in discussions with a whole host of regulators. I think there’s a direct connection between the headlines about things happening when AI goes wrong - and those are only a fraction of what’s actually happening out there - that public concern about this, and then regulatory reaction.

So regulators are going to up the ante, so to speak, in terms of what the liabilities are… So I really see two futures, and they’re both probably better than the present. The one that I’m gunning for, and why Patrick and I launched this BNH.ai law firm, is because I think there’s an opportunity for data scientists to work together with policy folks and legal folks and get this right, and be proactive rather than reactive. Our aim is to help that happen.

So I think that’s one future, where before there’s any real major incident, we as this – I was gonna say community, but it’s not really a community; it’s not like a legal and a – but as a group, we can start to build that best practices… And even, Dan, what you were saying about just documentation, and not overriding models when they’re updated, or at least just some basic best practices… I can see a future that is frankly a lot less bumpy, and where maybe the hype of AI can actually be met in practice.

[54:13] I also see another future, which is probably most likely, and I think what Patrick’s frustration is indicative of, is where regulators just say “This is not good enough”, and kind of slap down some much more spirited standards. That’s where things appear to be heading, although it’s not inevitable. But if I had to guess, I would say just like we were with – frankly, I was gonna say GDPR, but I think CCPA is a little bit more stringent. CCPA is this very overbearing privacy regulation that came through California, that now basically affects almost any big organization using data in the U.S.

I think the likelihood, if we’re not careful, is that there’s gonna be the same type of thing, and then every single data scientist is gonna have specific additional trainings, and they’re gonna have specific additional forms, and they’re gonna have lawyers breathing down their neck, who might not necessarily fully understand all the nuances of their day-to-day.

So I think no matter what, the life of a data scientist is gonna change. Risks are increasing, it’s gonna be harder just to deploy models without illustrating how those risks have been reduced, and I think how that happens is really up to us.

I think smart companies - I’m obviously biased, but I think smart companies and organizations will start to think about this stuff now, so that they’re not surprised when all this happens.

Jon Evans: Yeah. But yeah, that was interesting. There was something called The Arch Project, which is actually dropping copies of various things on the Moon, to just sort of crash-land various archival facilities, drives and so forth onto the Moon. Again, very cool idea, but not super-useful in the case you wanna access it at any time in the near future, or in a whole bunch of possible futures.

And then we had found out that there is a Norwegian company called Piql, a little software joke for the relatively few who will get it; I guess a disproportionate number in this podcast…

Isobel Redelmeier: That’s a good question, and it varies by language. If you have something that has – let’s say, worst-case scenario, only has global variables (the base case, basically), then you would need to add locks and stuff in order to ensure that if you’re trying to parallelize that loop, let’s say, that they don’t clubber one another’s state. If you have something like context local variables in Python, then you have it per future, I believe. I don’t write much Python; I’m starting to write a lot more Python right now, and it is making me miss Go… [laughter] Where was I going with that…?

So if your parallelization is using something that maps nicely to your context local variable type, like futures that are aware of context local variables, then you’ve got it for free. However, if you have something that – if you’re using, let’s say, futures, and you only have ThreadLocal variables, then again, there’s this mismap, so you need to add a lock or something in order to ensure, again, that you’re not clobbering state across one another. It’s kind of that same classic global variable problem, and sometimes it’s better mitigated, sometimes it’s really not.

Mat Ryer: Well, it’s been an absolute pleasure. Thank you so much for spending this time with us. I think you’ve all got really great futures; please stay in touch… We’ll see you hopefully online, we’ll see you in meetups, speaking, and we look forward to seeing how things progress. Thanks so much.

How about we have you on again next year and see where you are? We can check in… It could be really cool. Thank you again - thank you to Aditya, Shaquille and Ben. We’ll see you next time on Go Time.

Johan Vos: That is indeed one of the possible futures that I see. So you mentioned Shor’s algorithm, and then one more…?

Quincy Larson: So you touched on the fact that you were working, and you were working in another field, and just pursuing software development as a passion. Let’s back up a little bit to high school, you’re in this town where generally people don’t go on to bright futures… How did you break out of that?

Ron Evans: The other two announcements were the future… The future’s future. So the second one is the TinyGo Playground.

Suz Hinton: That is very cool. I’m very excited to see the future of that as well. So I wanna take this moment to thank Dylan so much for coming on JS Party and sharing past, present and futures of Dojo, too.