Alex Moore: Probably related to the complex set of contracts that Google has with all the OEMs, bundling the Google Play Store and Chrome together to be on the homepage.

Cory Doctorow: So there’s tons of other great platforms. Yeah, Libro, and even… Google Play will sell without DRM. But Audible won’t. And so you can’t get my books on Audible. You can’t get them on Apple Audiobooks, which until recently was just a frontend for Audible. Now it’s a competing product, but they kept that DRM rule. It’s another thing that you lose if you give up your iPhone; it’s not just the blue checkmark in iMessage, it’s all of your audiobooks, at 25-30 bucks a pop.

And so I have to find a way to sell these books, because I want audiobooks. I love audio. I’m a podcast and audio person; I’ve been listening to audiobooks since I was a kid.

Like, ever since those Disneyland little long-playing records, you will know it is time to turn the page when Tinkerbell rings her little bell like this.

Philipp Heckel: Yeah, yeah. I mean, you gave a good intro. Ntfy is a push notification system. I think that the website calls it a Pub/Sub system, which I might change, because people don’t know what Pub/Sub is, and they don’t care. Because first and foremost, it is meant to send push notifications to your phone, from wherever you want. So the API is HTTP-based, so you can just send the actual message in the body of the HTTP request, and everything speaks HTTP. So you can send yourself whatever message you want.

The way to use it is you go to the website, or you go to Google Play or the App Store and you download the app; then you subscribe to a topic… So this is where the Pub/Sub comes in. You subscribe to a topic; the topic can be any word you can think of. And then you can push messages to this topic. And everybody who subscribed to the topic receive it. So that means you can have multiple subscribers to one topic. So if you’re a group of people, they can all listen to the same topic, they will all get the same message. Or if you want to have a private topic, you can either pick something really complicated, similar to a password, or reserve a topic if you have a private instance. Since it’s open source, you can host your own private instance, or you can purchase a plan for the Ntfy Pro Plan and reserve topics that way.

[12:20] And then the message itself can be just – everybody’s seen a push notification, so they know what they look like, but there’s tons of stuff that you can do with it. You can give it a title, they have different priorities, different ringtones… You can override the Do Not Disturb, you can make it so that it persistently rings until you tap it, you can add emojis and tags that I’ve added as a classification method, and so on. Yeah, I mean, I could list all the features, do you want me to list all the features, I suppose I could.

Christina Warren: “We have to be everywhere.” No, when they made that distinction, I was so happy too, because I was somebody who’s like – I mean, the Movies Now service that Disney owns, and a lot of studios participate in, but not all… Not Lionsgate, not Paramount… There are a couple of other smaller ones. It means that your libraries will reflect back on one another. So if you buy it on Google Play, or on Amazon, or on the Microsoft thing, it’ll also be in your iTunes library and vice versa… Which is nice, but there will be a ton of stuff that I would have – obviously, the 4k stuff would all be on iTunes, and there’d be some stuff that’ll only be on iTunes, or Apple TV, whatever… And I was like “Okay, I don’t want to travel with an Apple TV for this stuff. I’m gonna have to find a way to rip this.” And then as you said, when they realized they were a production house too, and they had to have wider distribution, and they brought that to all of the other devices, I was like “Okay, thank you. This is good.”

Ivan Kwiatkowski: Yeah, this is a very good question. I do have very ambiguous feelings about them. I do believe that on the security perspective, it’s kind of a good thing, in the sense that yeah, it’s another one of those safeguards that prevent people from doing stupid stuff with their devices… And having to go to some friends’ places, or more specifically, friends of my mom’s places to debug computers, and uninstall malware, and fix the printers, then I’m very happy when there are protections that prevent them from doing that kind of stuff. But then again, they are not a perfect solution either. I think the Apple Store in terms of security is pretty good. The Google store, the Play Store has a bad track record when it comes to hosting malware. I’m not saying that they’re doing a bad job; I think it’s a very, very difficult job. But the fact of the matter is there are a number of apps on the Google Play Store that turned out to maybe not be total malware; some of them are, but a lot of them are just there to collect personal data, or that kind of stuff.

[58:23] So I think a better way of securing those devices is not to control the app stores. Creating protections on the device level is probably where I would work. So when you look at both iOS and Android, they are doing, I think, a very good job of – or have been doing a very good job, at least in the past years, of making sure that apps would not be able to access anything just because the user clicked Ok way back when they installed the app. So I think making sure that all those personal information cannot be pulled so easily is going to be a much better way than trying to police all the stores, and look at all those thousands of apps that are updated there every day… Which I do not think that you can realistically ensure that they are always going to be safe.

But overall, the other issue with walled gardens, which is, okay, maybe they do provide something with security, but also I feel like they take away some agency from me as a user, right? I really like to own the devices that I use, and having some restrictions that tell me “Oh, you cannot install this app because Google says you can’t”, or “You cannot uninstall this app, also because Google says you can’t” is something that tends to make me extremely, extremely angry.

Matthew Hodgson: It depends on the app that you use… Many people use Riot, which was the flagship app that we built to give people an easy onboard… And you can install that on Android off Google Play, or F-Droid, or iOS, or the web… Or as currently an Electron app on desktop, on all the various desktop operating systems. And it looks and smells a lot like Slack, honestly.

You have lots of group chats, you have public chats, more like you’d get on IRC, you have direct messages, you have private DMs and private group chats… You can group them into communities, but this is a horribly badly implemented feature, at the moment. We’re about to rip it out and fix it, but we don’t recommend people go anywhere near communities at the moment. We only keep them around because people have used them and depend on them, but… It’s something that we’re working on fixing.

[52:00] And then within that you have all the building blocks you would expect out of a modern chat system. Typing notifications, read receipts - really important, to have little chat heads which go Tetris patterning down the right-hand side of the screen to show you who has read up to a given point, that Slack doesn’t have… I have no idea how Slack survives without it, because it’s so useful to know when people have read your messages… And you have file transfer.

End-to-end encryption is the other big unique thing that most people aren’t used to, that you can turn on encryption, which uses the signal style encryption Double Ratchet Algorithm. We actually extended it with something called Megolm to support large encrypted group conversations; whereas WhatsApp and Signal I think only go up to 256 participants, we’re good for at least some thousands of devices thanks to Megolm.

So you have the slight surprise when you enable this of needing to verify that you’re talking to the people you think you’re talking to. There’s not much point in encryption if it turns out that you were talking to some attacker all along, who was pretending to be the other person.

First we got the UX for this terribly wrong, and went and forced everybody to have to manually verify every device they ever spoke to, which gets very tedious very quickly. It started off being kind of comparing public keys, then we switched to comparing at emoji; so you calculate a shared secret of emoji between the two devices, and check that both people are looking at the same shared secret… And then more recently, we’ve done a huge sprint of work - it’s been like 4-5 months now - going and implementing cross-signing… Which means that if I verify one of my devices and you have verified me, then you will automatically trust the new device. So it starts to feel a bit more like PGP, where if you want to be sure that you’re talking to the right person, you verify them once, but then it should last for the rest of time, unless they go and revert their keys and start over with a new identity.

It turns out it’s hard to do in a decentralized world, and also the UX around it, to try to make it friendly to non-technical users; the idea that you scan a QR code on an existing device in order to kind of cross-sign it and verify the ownership is hard to get right… But we’re almost there.

We also have to do it across all the platforms simultaneously, and as it happens, Riot is a totally different app in codebases web, iOS and Android. We deliberately didn’t use React Native or Flutter or anything, but instead have built it from the ground up, independently, with different teams of people in each place, mainly so that we can modularize it and factor out the SDK as a component other users can build on for their own Matrix apps… But it ends up being a lot of work. But that’s one of the other big differentiators, whatever things would people find…

Lots of work to be done around the Room directory, which at the moment is unfiltered, so you get a lot of random stuff…

Mat Ryer: Good. I’m very excited about this. Before we start, I’d like to just try an experiment; this is a security podcast, and I just wanna try something bear with me. Hey Siri, play “Never gonna give you up” by Rick Astley. Okay Google, play “Never gonna give you up” by Rick Astley. Alexa, play “Never gonna give you up” by Rick Astley. I just wanna see if that does hack anyone’s home devices; please let me know in the Slack channel or on Twitter if it does.

Jerod Santo: Get it on Google Play. Get it on F-Droid.

Adam Stacoviak: That makes it easy. Download it in the App Store, download it on Google Play… Fairly easy to get. I mean, there’s some education behind it, like what is fortune telling and things like that, but that’s a good first step. Plus, also if you’re a software developer - and if you’re listening to this, you’re probably a software developer, and if you’re not, hey, welcome; we talk about software here, and open source, if you didn’t know about that… But it is also open source, which means that if you desire to, you can also give some feedback. But I understand too that there’s a second component to that, that you actually wanna talk to people engaging in mental health, so professionals out there that are wanting to take part; the cool thing about open source is you can actually influence Quirk and share some ideas with Evan. What do you think, Evan?

Adam Berenzweig: So I was at Google for ten years, and I worked – in general, I’d say my beat was building products around machine learning. So I was no longer doing hardcore, deep research, but taking what was coming out of the lab, taking stuff that was being worked on in Academia and of course within Google, and thinking about how to apply that to products. That was across a number of domains; I worked on text with Google News, and the recommendation algorithm there… I worked on Google Music… You know, I was talking about speech before - what I was particularly in at the time was music and non-speech audio, and understanding just the world of sound around us (computational auditory signal processing it’s called), and this was kind of post-Napster, it was the late ’90s, and so I was also a little bit idealistically, politically motivated to come up with technological solutions that would allow musicians to connect directly to their fans without these intermediary giant corporations controlling all the intellectual property in the middle… So I thought, as many people did at the time, that recommendation algorithms are gonna be this great equalizer.

We already had a great way to distribute music, and we had a great way to produce music cheaply; people could make little recording studios at home… But in order for people to find music, we had to solve that problem. So I was interested in doing recommendation, but from the audio signal, so actually analyzing the content of music from the audio, using that to understand it, and then make recommendations to people.

Then I turned into Google Music and I helped build the music recommender for Google Play. Then eventually I started to work on images, and I worked on Goggles, which was one of the early image recognition mobile phone apps; you could point your camera at something in the world and we would tell you what it was. This is all pre-deep learning, actually while I was working on that project… So the project at the time I joined was – the app had already been launched, and it was kind of an amazingly cool toy, but it wasn’t quite yet a really useful tool for people… So the obvious thing to do next was to run this technology, run recognition on people’s photos.

At the time, Google didn’t have a Photos project. It was the Android camera, there was visual search on the web, you could upload photos, there was Picasa, the desktop app… So at the time, the Photos project at Google was kind of in its nascent form, but there was clearly a way that technology wanted to have its home. So I worked on images for a while with that. Then I left and I helped start a company called Clarify, which was an early deep learning image recognition company.

In between the time I joined Goggles and when I left Google, the deep learning – this was about 2012-2013… So working on Goggles, I saw some of the early, large conv nets that were coming out of research. There was an intern at Google, Wojciech, and it was doing things that we’d never seen possible from an image recognizer before that. It was really clear that “Oh my god, this is really gonna change everything.” It was a technology that was ready for primetime.

Tim Smith: [laughs] But I use iTunes nowadays for all of my media purchases. My movie collection is slowly moving to iTunes… Back in the day I used to buy a lot of Blu-rays that had a digital coupon, so that I could get it on iTunes. Then what I’ve done also is subscribe to Movies Anywhere, which used to be, I think, Disney Anywhere, which is something Adam said before… But basically, what it means is that you can hook up Amazon, you can hook up Google Play, you can hook up VUDU, you can hook up a few different stores that sell movies, and whichever store you buy it on, it gets backed up. You have it in one central place, which is what I wanted. If one day I decide iTunes is not for me anymore, I still have my movies in Movies Anywhere. I feel like that was a pretty secure decision. The only part is that there’s some movie studios that don’t currently support Movies Anywhere, and that can be a problem, depending on the movie that you want.

But yeah, anyway… I have more commentary on what Adam all said, but I’ll wait for later. [laughs]

Erik St. Martin: Alright then. I wanna thank everybody for being on the show, especially thank you to Aaron for coming on this show. Everybody definitely check out his Go In 5 Minutes videos; I think it’s gonna be really interesting to see all the new ones that come out from this show. I think we’ve had a lot of good discussion. Thank you to the listeners, everybody who’s listening live and interacting with us on Slack. Definitely subscribe - you can go to GoTime.FM to subscribe if you haven’t already. We are on both iTunes and the Google Play Store. Follow us on Twitter @GoTimeFM and github.com/gotimefm/ping if you want to be on the show or have recommendations. Definitely thanks to our sponsors today too, Linode and Backtrace. With that, goodbye everybody!

Erik St. Martin: So almost in line with us talking about the new Sourcegraph tool today. Yeah, so if you haven’t subscribed already, GoTime FM is the easiest way to do that. We are now on both Google Play Store and iTunes, I believe. More episodes will be dropping if you’re listening to this live right now. I think episode four drops today, and then five and six over the next couple of days. We’re slowly getting caught up. We are on Twitter @GoTimeFM, we are GoTimeFM in Slack if you want to be on for the live stuff, and I think that’s it. Yeah, GitHub - github.com/GoTimeFM/ping for suggestions for speakers or comments about the show and all that stuff. Thanks again, Ed, for coming on the show. It’s been great having you on the call.

Spencer Kimball: Yeah, so databases - it turns out that they have been extraordinarily essential in my career, back as early as the dotcom startup I did, WeGo Systems. We built sharded Oracle and sharded Postgres is the two sort of flavors we supported. And I’ve gotta tell you, when I was at Berkeley I wasn’t very interested in databases. I mentioned graphics - that was really probably my key interest. Databases - I didn’t take until my first and only year of grad school, and I just kind of took it to get some credits.

[27:53] I ended up being pretty interested in the course, but I didn’t really think they’d be central to my career, but as soon as I hit the “real world”, databases became a central problem, a big source of frustration at WeGo, and then when I got to Google, that was one of the first projects I got thrown onto, which was the AdWords system, which was nascent then in 2002… But it was running into problems with sharded MySQL. And you hear this word “sharded”, but for listeners that aren’t aware of what that implies, it’s about taking a monolithic database like Postgres or MySQL or Oracle that really is meant for a single machine, even if that machine can be quite large… And you say “Well, maybe this isn’t gonna be large enough”, and this is the case of AdWords when I got put on that project.

So you say, “Okay, we’re gonna use two databases. We’ll put half of our customers on the first database, half on the second”, and maybe at some point you start reaching capacity on those two, and so then you say “We’re gonna use four” or “We’re gonna use five” etc. It got up to about 32, I think, when I was at that project at Google… And all these different problems started to occur as we sharded. The application complexity became quite high. It just went ridiculous…

Practical example - the MySQL databases had too many connections coming into them, and that started to cause them to cavitate. And so we solved these problems – every morning we had these Ads War Room to solve the latest set of problems related to this scalability challenge with the database.

I will just say that in Google AdWords, by the time they replaced that sharded MySQL architecture, they’d gotten to a thousand shards. So it became a thousand MySQL instances. And I’ve heard that Facebook has hundreds of thousands of MySQL instances. So there’s kind of no end to both how scalable that architecture is, but also how much time you have to put in to truly keep scaling it.

So that’s a scalability challenge… There’s also resilience challenges, and that’s part of what we saw when we were at Square, and it was certainly something we saw when we were at Google… And that is you really don’t wanna have a database that has a primary and a secondary… And that’s been the standard way to operate databases for most of my lifetime. The problem with that solution is that the secondary is getting an asynchronous replication stream for data. And even if you put in another datacenter so you have a really nice failure scenario, so you can lose a data center and fail over, that failover might imply data loss… Because that asynchronous replication stream might not have fully made it over to the secondary when the primary dies. So you switch over to the secondary and you realize “Wait a second… I thought I just sent that email out”, as an example. But it’s not in my outbox. What happened? Well, the replication stream just didn’t get that email into the outbox on the secondary. So it’s almost like you’ve moved backwards in time. You’ve regressed to an earlier version of the state that you had in an application, and that causes huge headaches.

If a data center was lost at Google back in 2004, let’s say, it would be many teams scrambling to figure out what might have gone wrong. “Did we charge a customer twice? Are there consistency problems in the data because some of this stuff got replicated and some other stuff didn’t?” And you’d have to write cleaners and scripts that would go through things… And you’re just trying to reason through what might have gone wrong with your use case. That’s not the right way to do database replication, and certainly not in 2020.

Google started to play around with better ways to do that as early as 2004-2006. They built Bigtable, and then they built something called Megastore, and then they built something called Spanner… And Spanner is really what inspired Cockroach. So there’s scalability, there’s resilience… Those are two of the biggest problems that I’ve faced with databases in my career.

[31:53] The gold standard these days with databases is to do what’s called consistent, synchronous-based replication. The popular ways to do this is something called Paxos, or something called Raft… And what they do is consensus. So instead of just writing to a primary and asynchronously replicating to a secondary, you actually write to three data centers, or three replication sites, and you are going to be committed if the majority of the replication sites respond positively or affirmatively to any particular write. If for example you only write to one out of three data centers, that write can’t be committed. You need two out of the three. As long as you have two out of three, if you lose any one data center out of those three, you always are guaranteed that one of the remaining two has the exact data that you need. So as long as you only lose the minority, you have total operational continuity. It’s hard to overestimate just how important that advance is for running these systems operationally.

Spencer Kimball: Yeah, so databases - it turns out that they have been extraordinarily essential in my career, back as early as the dotcom startup I did, WeGo Systems. We built sharded Oracle and sharded Postgres is the two sort of flavors we supported. And I’ve gotta tell you, when I was at Berkeley I wasn’t very interested in databases. I mentioned graphics - that was really probably my key interest. Databases - I didn’t take until my first and only year of grad school, and I just kind of took it to get some credits.

I ended up being pretty interested in the course, but I didn’t really think they’d be central to my career, but as soon as I hit the “real world”, databases became a central problem, a big source of frustration at WeGo, and then when I got to Google, that was one of the first projects I got thrown onto, which was the AdWords system, which was nascent then in 2002… But it was running into problems with sharded MySQL. And you hear this word “sharded”, but for listeners that aren’t aware of what that implies, it’s about taking a monolithic database like Postgres or MySQL or Oracle that really is meant for a single machine, even if that machine can be quite large… And you say “Well, maybe this isn’t gonna be large enough”, and this is the case of AdWords when I got put on that project.

So you say, “Okay, we’re gonna use two databases. We’ll put half of our customers on the first database, half on the second”, and maybe at some point you start reaching [unintelligible 00:26:22.09] on those two, and so then you say “We’re gonna use four” or “We’re gonna use five” etc. It got up to about 32, I think, when I was at that project at Google… And all these different problems started to occur as we sharded. The application complexity became quite high. It just went ridiculous…

Practical example - the MySQL databases had too many connections coming into them, and that started to cause them to [unintelligible 00:26:49.27] And so we solved these problems – every morning we had these Ads War Room to solve the latest set of problems related to this scalability challenge with the database.

I will just say that in Google AdWords, by the time they replaced that sharded MySQL architecture, they’d gotten to a thousand shards. So it became a thousand MySQL instances. And I’ve heard that Facebook has hundreds of thousands of MySQL instances. So there’s kind of no end to both how scalable that architecture is, but also how much time you have to put in to truly keep scaling it.

So that’s a scalability challenge… There’s also resilience challenges, and that’s part of what we saw when we were at Square, and it was certainly something we saw when we were at Google… And that is you really don’t wanna have a database that has a primary and a secondary… And that’s been the standard way to operate databases for most of my lifetime. The problem with that solution is that the secondary is getting an asynchronous replication stream for data. And even if you put in another datacenter so you have a really nice failure scenario, so you can lose a data center and fail over, that failover might imply data loss… Because that asynchronous replication stream might not have fully made it over to the secondary when the primary dies. So you switch over to the secondary and you realize “Wait a second… I thought I just sent that email out”, as an example. But it’s not in my outbox. What happened? Well, the replication stream just didn’t get that email into the outbox on the secondary. So it’s almost like you’ve moved backwards in time. You’ve regressed to an earlier version of the state that you had in an application, and that causes huge headaches.

[28:37] If a data center was lost at Google back in 2004, let’s say, it would be many teams scrambling to figure out what might have gone wrong. “Did we charge a customer twice? Are there consistency problems in the data because some of this stuff got replicated and some other stuff didn’t?” And you’d have to write cleaners and scripts that would go through things… And you’re just trying to reason through what might have gone wrong with your use case. That’s not the right way to do database replication, and certainly not in 2020.

Google started to play around with better ways to do that as early as 2004-2006. They built Bigtable, and then they built something called Megastore, and then they built something called Spanner… And Spanner is really what inspired Cockroach. So there’s scalability, there’s resilience… Those are two of the biggest problems that I’ve faced with databases in my career.

The gold standard these days with databases is to do what’s called consistent, synchronous-based replication. The popular ways to do this is something called Paxos, or something called Raft… And what they do is consensus. So instead of just writing to a primary and asynchronously replicating to a secondary, you actually write to three data centers, or three replication sites, and you are going to be committed if the majority of the replication sites respond positively or affirmatively to any particular write. If for example you only write to one out of three data centers, that write can’t be committed. You need two out of the three. As long as you have two out of three, if you lose any one data center out of those three, you always are guaranteed that one of the remaining two has the exact data that you need. So as long as you only lose the minority, you have total operational continuity. It’s hard to overestimate just how important that advance is for running these systems operationally.

Adam Stacoviak: It’s also interesting to see that you care enough - or you’re diplomatic enough; that might be a better word - to hand this over to essentially the community, in that sense of the world. Being a Linux Foundation sub-thing with the Cloud Compute Foundation, it shows that Google wants to play a role but not own.

Adam Stacoviak: What’s the long-term play here though? I mean, if Microsoft invested that kind of money, and we’re using ChatGPT right now, we’re willing to give it extra seconds, potentially even a minute if the answer is that important to you, that you wouldn’t afford to Google… Like, what’s the play for them? Will they turn this into a product? How do you make billions from this? Do you eventually just get absorbed by the FAANGs of the world, and next thing you know now this incredible future asset to humanity is now owned by essentially folks we try to like host our own services for? Like, we’re hosting Nextcloud locally, so we can get off the Google Drives and whatnot… And all this sort of anti-whatever. I mean, what’s the endgame here?

Jerod Santo: How did you come to work at Google? Did your singing play a role in that directly or indirectly?

Amal Hussein: We’re going to have to come back and do another show on JAMstack. No, really, because I think the landscape has really changed since I’ve taken a look at, “What is this JAMstack thing people are going nuts about?”, right? I think JAMstack has grown to be a bigger bubble, and I’m eager to maybe try to break that down a little bit, but that’s really fascinating. And so, how do the Google core web vitals play into JAMstack applications? I guess, do you want to maybe just share what some of the core Google web vitals are?

Adrien Treuille: Yeah, so the mobile picture is that you deploy an app in one way or another, and so you would have to have either a VPN, or a public IP address… And then if you point someone to that app, they will see it either rendered correctly on a browser, or rendered correctly on a mobile device. An example of such an app that one of our users created is called Awesome Streamlit. It’s basically a collection of cool scripts and tricks that people have figured out in Streamlit. It’s another sort of separate open source project; you can do pull requests against it, and stuff… And it’s an app that just runs on the internet and anyone can go to it.

So if you wanna see the mobile experience of Streamlit, just google Awesome Streamlit and play with it, on a mobile device or on a standard web browser. So that’s the mobile story right now.

We don’t have any way of packaging a Streamlit app into an iPhone app or something like that yet. It’s a really cool idea though… [laughs] Maybe we should do that.

Sherol Chen: “Now let’s take this down the practical route…” So we have machine learning, we have AI - how does that impact the world? How does it affect the everyday person? A lot of people in the U.S. use smartphones, so now if they’re traveling abroad, they could take a picture of a menu and get that translated on the spot. Or they don’t even have to take a picture; within the computer screen, they can translate from Spanish to English… And a lot of this image recognition or language understanding comes from advanced [unintelligible 00:25:10.22] machine learning.

So on the user side there’s already a lot of people using it, and what they’ve found is that a lot of these tools are useful if you want to be able to use voice commands. You don’t need to necessarily retrain – I don’t even know how much data they used, but all the data that they require to train that voice model, that voice speech-to-text model, they don’t need to necessarily have to go through… Like, every company that needs that doesn’t need to go through and do that again. So Google having worked on this technology and using it in its own products makes this available for people.

So you can come in, you have an app that you wanna build, you want it to be voice-activated, or you wanna be able to take images and figure out if they are a kid-friendly image… There are these models that Google has pre-trained, and Google is ensuring that these models work well. So without any data, with just the desire to have this feature, you can already plug and play certain products that Google has in machine learning. Anything that requires vision, or audio processing, or natural language - I think these are very common use cases.

And then on the other end of things, like you were mentioning, you could set up your own Python environment, you could be coding in TensorFlow, you could train your own model, you could feature-engineer on your own, you could hyperparameter on your own, you can look at the models and the precision and recall, and assess how good these models are doing, and then you could do it all over again. And maybe you have something running in production - you can have all the control over every aspect of that. So that would be the other end.

Of course, enterprise cloud technologies - they provide the environments. You have Docker containers, you have virtual machines that can also satisfy that need. What AutoML does is it focused on the middle piece, or at least trying to figure out how you go from one end to the other. So in one end you have pre-trained APIs, where you have basically no control over how that model is making decisions, and then you have build-it-from-scratch, do it all yourself. For a lot of businesses, it’s kind of hard to go sometimes from research directly to production, and so trying to find that middle point is what people working in AutoML are trying to figure out. Things like hyperparameter tuning; tuning these weights around to kind of get a feel. This is a highly intuitive thing, where you get a sense of whether this number should be bigger or lower. AutoML, one of the aspects is trying to figure out how to just simply do that.

[28:02] With the tools that we have now, it could just be a matter of thinking – you might be able to figure out safe search, or kid-friendly images; it’s something pretty universal for the most part, but maybe not for every culture. You might wanna label your own data, as like “This image isn’t safe.” Maybe it’s not enough to know which pictures have cats in them. You can go to your Google Photos right now and do that. You can go into your Google Photos, type in “cat pictures”, get all the cat pictures. But what if you wanna know what type of cat, or what if you wanna know different colors of cats? The specificity that these pre-trained, more general use cases may not provide.

So AutoML, in the most general use case, is basically giving you the opportunity to label your own data, and then feed that in. So you’re getting customization without having to build everything out from scratch.

Gregg Tavares: And I talk to friends, “Hey, we should make a game”, and then yeah, we don’t do it. One of it is golden handcuffs. You come to a big tech company and you’re like “Oh, what? I don’t have to crunch, and I get paid a lot more?” Another is that the industry changed. The last I shipped it was 2007-ish… The iPhone came out in 2007; mobile games became a thing. Free to play games became a thing. When I came to Google, I mentioned that I came to write a game engine; that was because at the time Unreal actually cost like a million dollars to use it.

[00:48:18.13] And I remember – Unity might have been free, but they didn’t have the reputation yet for “Oh, this is something you should use.” So I came to Google to make an open source engine. That all changed. Now there’s two – they’re not free, but they’re free to use to prototype at least… And to even ship. It’s only if you make a lot of money that they want some money.

And there’s things like Godot, and there’s a whole bunch of other engines out there now that weren’t around back then. So that’s changed it in multiple ways. It’s made it easier… So theoretically, if those existed back in 2008, then I probably wouldn’t ever have come to Google, and I would have just tried to make something… On the other hand, there’s – I don’t remember what the numbers are. There’s several thousand new games every day. And so it’s just much harder to stick out, to look at that and go “I have an idea that’s gonna sell.” Oh, wait, I’m gonna put three years in, and it’s gonna sell five copies? I’ve had friends that do that; they made the game look beautiful, they shipped it… I think they sold 80 units.

Brad Fitzpatrick: I started off with BASIC and C. But neither one – you know, in BASIC you hit limits pretty quickly, and C is not very fun. It’s like, it exists to do things when you need to use it, but I don’t enjoy writing in C. It just feels tedious.

So when I discovered Perl, in ’93-’94 or something like this - it was Perl 4 at the time - I was like “Whoa, I can do things really quickly!” But this was also the language everyone at the time was doing CGI scripts in too, so it was also the language that let me do cool, dynamic web stuff.

[28:07] So I was very in the Perl community, very involved, and all of LiveJournal was pretty much all Perl, except for little bits and pieces that needed to be C for performance reasons. Memcached was originally in Perl, but it fell over within like a second, as I suspected it would. The prototype was in Perl, but then we quickly rewrote it in C.

So I love Perl, and I never really jumped on the Python or Ruby trains, because they were just not different enough from Perl for me to care. They had all the same pros and cons; they weren’t much better. They were still like explode at runtime, and no type checking… Python and Ruby were even slower than Perl, so performance wasn’t a reason… And maybe the syntax was a little cleaner, but at that point I didn’t care, because I already knew Perl.

And then I joined Google, and they use lots of Python, C++ and Java, so I wrote in all of these kind of out of necessity, but I didn’t really love any of them. I worked on Android for a couple of years and wrote a lot of Java, and a lot of Google frontends were often in Java, and Google backends were often in C++… I launched a service that was a whole mx of things. It was like Java, C++, Sawzall which was Google’s language for logs analysis, which Rob Pike worked on prior to Go… I kind of hated it, but that’s what everyone at Google wrote their logs analysis in at the time. Nowadays they’ve migrated off Sawzall onto Go.

Yeah, so I was never really excited about any of the Google languages. I wrote in them because I had to, because there was no choice… And then I saw Go come out. I went to a bunch of the internal talks on Go, and I was like “Yeah, yeah, whatever…” Google had a bunch of internal languages, and most of them were so-so… So I was not super-excited at the time.

Somebody had a question from the audience, like “Does Go have closures? Do you have first-class functions?” And the answer was a very dismissive “No, we don’t. You don’t need them”, or something. And I was like “Well, okay… I’m out.” Perl had them, and I had this book “Higher-order Perl” that I really loved, that taught me a lot of what people would probably actually learn in Lisp in school, or whatever, but I never really had any exposure to it until that book. So I was totally in love with higher-order functions, and stuff like that. So when Go didn’t have it, I was like – I kind of wrote it off. And also was like “I don’t really invest much time in another Google internal language, because I’ll just leave Google at some point and I won’t get to play with it anyway, and I won’t get to use it for my personal projects…”

But then when they said they were open sourcing it, I was like “Oh, I should look again. And then I noticed they had added closures in the meantime. I guess Russ Cox just went in one day when Ken Thompson was on vacation, and couldn’t object, and Russ just added them. [laughter]

Justin Searls: [00:43:50.07] …in advance. So I recorded that on Saturday. I was wondering, “Why would Apple publicly announced that they’re going to work with Open AI for their backend model when all of our iCloud stuff is stored on Azure, AWS, Google Cloud?” And Apple never betrays that information. Like, they play them off each other; they just treat it like a commodity. So why is it that the language model would get elevated into this thing? And Federighi even kind of previewed that it would be – or was it him? Whoever was talking about it was like that it would be configurable in the future, that they’d add additional models… So for Apple to take both third parties, elevate them to a named thing on their platform that the users know about, and then to give the users the *bleep* work of having to decide which model is best in which circumstances - there must be a reason. And I think that that reason became clear during the keynote, which was “Apple wants plausible deniability.” “Hey, we are confident that our very limited model, that’s only local, that’s got temperature set to zero, is going to work fine. But if you want something more, here, you can click this escape hatch button.” And as soon as you do, you’re gonna see “ChatGPT may make mistakes”, and you’re gonna see their logo big.

So what Apple is saying really is “That’s the throat to choke. Don’t look at us.” Whereas Google has all the egg on their face, because they brand Gemini as Google Gemini… And Microsoft, too. Even though Copilot – we all in the industry know that it’s backed by Open AI. They want to brand it as if it’s their own thing, and they go out of their way to say “Oh, we’ve got all of our own proprietary models on top of it, too”, which means that when recall screws up, or whatever these new features are blow up, it’s Microsoft with egg on their face. So I think this is a way for them to have their cake and eat it too.

Joran Dirk Greef: So you can actually have a business and sell something that’s gonna make something cheaper for startups. And similarly for enterprise, you can have a business and sell something that is going to provide the value they need, which is - now, they might have SRE teams, but they need the infrastructure to support massive scale, like Petabytes. How do you connect TigerBeetle to object storage S3. Like, OLTP data lake; not only OLAP data lake, but let’s just connect the OLTP direct to S3.

And so this comes to your question about the tug of war, and licensing, and all of this. And I think the big mistake that we can make - and I used to make this until it became clear for me that third moment - was that an open source license affects the implementation, not the interface. But when it comes to competition in the business world, that doesn’t happen at the implementation. Typically, it happens at the interface. So if you think of like some of the fiercest competition, when things were really on a knife’s edge for the web, it was the interface, not the browser implementations. It was the interface that the war was fought. Mozilla fought that war, and we needed other browsers to fight it, because the interface was being embraced, extended and extinguished. Triple E. And then you think of like Android, and Java, and again, it wasn’t about the Android implementation, it was the interface. And that was that Oracle/Google lawsuit, you know? And then again, you think of like “Well, Confluent –” Kafka’s Apache 2.0 open source.

Then Red Panda came along - and I’m a huge fan of Red Panda, because very similar design decisions around being direct with the hardware, static communication… All of this very, very similar. We came from a similar time period, but in that time period the things were changing, how you built things. But Red Panda came along and they saw the open source implementation of Kafka and said, “Well, thank you, but we don’t want it. But that interface, that is great. That’s where our business will be also. That interface, thank you very much.” And so they built a massive high-performance implementation.

And then WarpStream came along and they said, “Well, Red Panda, you are business source license, not open source source available. Confluent or Kafka, you are Apache 2.0 open source.” But that’s all implementation. I’m gonna do my own implementation, thank you very much, of object storage. But the interface - great. Okay, now we’re all competing. And so I think the myth is that source available was kind of the thing that always – I always feel that something inside of me dies when I see a great company relicense, or when I see a young startup follow that lead. Because to me, source available says that “We think it’s gonna stop competition.” It doesn’t. You may as well be on the beach, building little moats and sandcastles, but innovation technology is like a wave. It’ll just find a way around you. It’ll WarpStream around you. You can’t legislate competition away. And we shouldn’t be trying to build companies where we think the success of the company is us creating a monopoly. The world’s too big. There’s too much for you to be – you don’t need monopolies to do really great. And that doesn’t build trust, to say to your customers “You can only buy from me.”

So I think people think it stops competition, and they think it helps them sell, and it actually defeats both of those, because you get complacent, and you actually fail to build trust. So you burn trust when you relicense, and if you start source available, you’re gonna be doing diligence with enterprising and say “Sorry, you’re not open source.” It’s license confusion, you know. And maybe some people get it, but there’s a little 1% headwind. And it’s actually - it’s a category error, because you’re spending so much effort chatting to people about implementation licensing… And the rest of the world is competing on interface. And TigerBeetle’s interface is quite simple. Very simple.

[01:24:30.17] So we could – I don’t know, whatever license we apply, it doesn’t matter. Debit/credit is where we compete. There are companies that offer debit/credit as an API and it’s very similar to TigerBeetle. But we compete on trust. We didn’t just take a general purpose database and slap on debit/credit. We went deep; we really cared and we built the whole thing.

And people pay us. So before we were a company, we had an energy company reach out and we landed quite a good contract, very quickly. And I think it came down to trust. So open source builds trust. So open source is great for business. It’s also orthogonal.

I think the other thing is if you – so there’s a lot of things in TigerBeetle that are like the… I had done many experiments of my own, my passion projects. They’re all in TigerBeetle. Many parts of the design of TigerBeetle come from these various experiments that I did. And so I was never going to put that all into a project if it’s not open source, because it’s just too valuable. I want to always be able to play with it no matter what happens to it. And I think we all feel like that, like our critical infrastructure just has to be open source. So yeah, I think that’s kind of how I think of it.

Autumn Nash: [00:48:09.17] You know what I think is really impressive about everything that you’ve said? First, the fact that you actually went to the conferences and listened to your customers… Like, the feedback loop of like customers… And especially, people go to conferences with their competition, and don’t do – I love going to the different booths and talking to their engineers and talking to their PMs, and the disconnect between their engineers or their PMs and their customers is wild. And then they never have the right information about the competitors that are 10 feet away from them, and they could have easily went to the booth and asked a question… And it blows my entire mind every time.

That is the best part about being a solutions architect, a PM, or whatever you want to call it… Just being able to compare the product and see where – like, half the time there’s low-hanging fruit that would be so easy to fix or to know more about to make your products better. But also, you’re using AI for something that’s going to actually make people’s lives better, not something that we didn’t ask for, not in a way that you’re handicapping your developers.

A lot of times just adding to IDEs, you’re teaching new developers bad habits, that they don’t even know they’re getting, and you’re not enabling them to get the skills that they need to troubleshoot and to grow as engineers. But you’re using it in a way that actually teaches you how to debug, and what the process is, and to help them learn processes, which makes engineers better.

So I’m so excited about this, and I want to go play with it, because I think we’re going to handicap so many engineers in this next generation with different AI, because they’re only – it’s like that one meme and it’s like “Why hire a software engineer if they just google? Why give them six figures to just google?” And it’s because you know what to google. You know what I mean? And we’re almost taking that from people, because ChatGPTs, or just any GPT is going to become the source of truth, and they’re not going to get the skills to figure out “This looks wrong” or “This looks right”, because so much of it isn’t just an error [unintelligible 00:50:07.18] That’s not all of being an engineer, and putting all these pieces together.

So the way that you’re describing it is so exciting, because you’re teaching them a process. Even if they’re going down the wrong – you’re, for one, making it faster, which we all know that every big company or a small company needs you to get faster from being new, onboarding, and writing that first commit in PR. But also, they need you to be effective and to learn. And you’re bridging those two things, which is just amazing. This is the first time I’ve been so excited about AI… Because usually, I’m like “Oh, this sounds [unintelligible 00:50:39.26] That’s actually amazing.

Autumn Nash: [00:08:04.03] I’ve banned them to Epic, but they even watch YouTubes, and PE, and different Doug and all these different questions things, which [unintelligible 00:08:11.14] school that they’re educational… But I’m “If you can’t find it on epic or PBS, you’re out of luck.” They learned how to Google, though, and ask Alexa how to do stuff, and they’ve found out so many ways around system child locks that at this point they should hire my four year old to work for the damn NSA. He figured out PlayStation’s entire child lock thing… And he can’t play Plants vs. Zombies, and he can’t play regular games, but he’s figured out how to redownload Fortnite six times. Who made it?