Justin Searls: So think of a company as large as Apple at this point is – they don’t run different business units as separate companies. There’s only a small handful of SVPs that kind of determine what’s happening… But underneath them is a traditional, big, layered organization of humans. And when you think about the iPhone, the iPhone is awesome, and they’ve never shipped a lemon iPhone. And why haven’t they done that? Because they are proving themselves with excellence, year after year, iterating on that hardware, and learning from their mistakes… And we don’t get to even see a lot of the mistakes. I’m sure they get killed before we’d otherwise see them.

The other department that came to mind - Apple TV. You guys were talking about Severance… Apple TV has got some real good hits. They’re more HBO now than HBO has been for a long time. And the reason is it’s run by competent people, that clearly have a commitment to excellence, and they’ve got the feedback loop of people are watching it, people aren’t watching it. People are subscribing, people aren’t subscribing. And there’s clearly some kind of wind at their back.

Now look at the third division here, of the people who run Siri. Siri has been running in circles since 2011. How do you measure success for Siri? It doesn’t make or save money, there’s not a lot of – because it’s kind of non-deterministic, it’s very, very difficult… No one’s done a Siri benchmark, right? Like, we’ve got all these LLM tests, and stuff. No one’s been tracking the growth and maturity of Siri as a functionality over the years. You know like that story we’ve heard about how the WebKit team at Apple had this build where if you committed any code and you pushed it up, and it made the test suite duration slower, the build system would reject your commit. You couldn’t make web kit slower. And that was just a hard and fast rule. And look around you; I still use Safari every day. And why is that? Because that rule has like a major impact on my user experience, because browsers are so vital. And Siri doesn’t have anything like that. If they have an internal benchmark, like… You look at all the posts over the last year - if anything, since the Apple Intelligence thing, it seems like Siri is getting dumber at very basic facts. So that organization, whoever’s in it and whatever it is, they’ve clearly not been on some sort of loop towards excellence that’s driving them forward in a clear and steady way. And the human organization that’s built around that, as a result, how could it be anything other than a nonsensical bureaucracy of finger pointing, and arbitrary excuses? Because there’s not like some sort of cycle feeding in and reinforcing the good stuff, and trimming away the bad stuff. And that’s the organization that was handed Apple Intelligence here. Go have fun, go do the hardest thing we’ve ever done. Build this thing that is going to just be a moonshot, and also make sure it never says anything racist, and it never generates any kitty porn. Other than that though, I’m sure you’ve got this. We’re just going to go make this video now and start printing the billboards. How do you think that went? Well, we see how it went.

Birk Jernström: And music was in this phase where everyone thought music was dying; everyone just expected it to be free. No one’s gonna pay anything for it. And here came Spotify along, and just like “There’s a new way.” And that became a success story. And they’ve flipped that narrative all the way around. Like, I think Shopify now is doing more than the entire music industry was doing back when they originated and started out. And I think venture capital fundamentally is taking these moonshot bets that can become something game-changing, and define a new market. And I think that’s what Polar really had going for it.

There’s a lot of players out there that are pitching great ideas, but probably great indie ideas at the end of the day… And and for us it was just “You have the developer ecosystem and the open source ecosystem, which are immense and incredibly valuable to the entire ecosystem of technology, but we haven’t cracked funding and monetization yet.” And I want to build a platform that can crack that code. And if you’re able to do that, I think that can have a really meaningful impact for those individuals, as well as for Polar as a business. I think that’s fundamentally what excited VC’s about Polar and what we’re building.

And then there was slides about all of the things that we talked about, like these different monetization tactics that we would offer eventually, and… But we launched and we just started out with issue funding. And so we’ve been just iterating one funding tool, and now soon one monetization tool at a time, to just continuously learn from users and continuously adapt, to hopefully solve it for as many people as possible.

Break: [00:58:24.26]

Jerod Santo: It’s a moonshot.

Dave Farley: Absolutely. And there’s a few things that you said in there that I liked, that you said about teams being kind to one another. I think that’s important. And being tolerant, allowing ourselves the freedom to get things wrong… I think very deeply in my own work, at whatever level, about that… Because as you say, I start out from the assumption that nearly everything I do is going to be wrong in some way. And so how am I going to be able to cope with that, and how am I going to allow myself the freedom to change my mind? Technically, if we’re designing software, that gets back to that stuff about managing complexity - all of the things about managing complexity are only giving us the freedom to make a mistake and correct it later on without throwing away everything that we’ve ever done. I think that’s really important.

I think of myself as a software developer as being defensive in these terms. I’m going to start out trying to design my systems and code my systems in ways that allow me to change my mind about some of the things as I move forward. And this is true about everything. This is true about the goals for the software. If we go and ask our users, they don’t know what we want. If we ask our product owners, they don’t know what’s going to work for users. If you ask the developers, they don’t know what’s going to work for users either. We’ve got to try stuff out and find what lands and find what doesn’t.

I’m occasionally guilty of using soundbites, and one of the soundbites I’ve used in this space is that “A perfect plan is a perfectly stupid idea.” Because a perfect plan has one solution; you’re precisely targeting this one point in time and space that you’re trying to hit. And there’s almost no chance for any – then anything beyond the few milliseconds, there’s almost no chance that you’re going to be able to perfectly hit that target.

[38:03] One of the ideas in the “Optimizing for learning” section of my book is the idea of iteration. Well, two ideas - iteration and feedback. So if we work iteratively, we are going to make progress in small steps. After each step, that gives us an opportunity to reflect on the progress that we’ve made. If we have some kind of fitness– so we’re going to have a target; our plans are in the form of there’s this flag on a hill that we’d like to kind of… “Wouldn’t it be wonderful if we got there?” And those are sort of loose, slightly imprecise, inexact in terms of what it is that they want to do… But it’s the moonshot. “Wouldn’t it be wonderful if we could achieve that goal?” And then, we’re going to start iterating. And as long as we have some kind of fitness function, a way of measuring, “Are we closer or further from that target?”, then even if we just did a random walk and just started iterating, without any intelligence at all, we could sort of try something out, “Does that move us closer to our goal, or further away?” We discard things that move us further away, and keep the things that move us closer. If we just did that, we’d hit the goal. Even if we move the goal, even if partway through we think “That’s the wrong goal. We’re going to shift the goal and move it over here”, but we’d reach out and we’d change our fitness function, we’d still hit the goal. That’s the power of iteration. With an iterative approach there are many ways of winning. So that’s one way in which it improves our chances of success, because now we’ve got more chances of success, because we can find multiple routes to our goal.

So I think planning is used, as the general – and I’ve forgotten who it was; one of the World War II generals, I think, famously said, “Planning is wonderful. Plans are stupid.” I like planning, I like thinking about what’s the future like, but I think people that spend… You know, I once worked on a team and the project manager went and locked themselves away in a room for two weeks writing Gantt charts… And guess what, the Gantt chart didn’t meet reality at any point. Even when she came out of the room with the Gantt chart, we had already moved on from there. [laughs] It doesn’t work. So it’s more complicated than that… And that’s one of those naive things.

So seeing it as these kind of complex, adaptive system, which the whole environment of the software and the people that are building it, and the customers that are using it are changing all of the time, and if you make one change in one place, it changes what other people perceive of it - that’s just where we live. That’s just the nature of the place that we inhabit as human beings and as software professionals. And so suck it up; we’ve just got to find ways of working there. And that takes iteration, feedback, working experimentally, those sorts of ideas to be able to navigate that kind of space. And some of our guesses will be wrong. That’s fine. There’s nobody that knows the right answer. The common refrain of software development teams saying, “Oh, we would have done a good job if only that the requirements had been correct…” They’re never going to be correct. That’s just an illusion. Nobody knows the answer. Even if you have the user sitting in your room, that’s only their guess.

One of the things that I liked that Steve Jobs used to say is “How will the users know what they want until I tell them?” That sounds incredibly arrogant, and he is. I mean, I think he was an incredibly arrogant man. But nevertheless, there’s this truth there. If you’re doing something innovative, you ought to be ahead of the users; you weren’t gonna get an iPad or an iPhone by asking users what they wanted. You’ve got to think ahead. But at the same time, you want a vision that your users are going to love, so you’ve got to be listening to your users.

[42:07] So it’s not as simple as – nobody knows the truth. Steve Jobs got it wrong lots of times as well, and built crazy things that nobody liked… That’s just life. And so and so working in ways that allow us the freedom to make those mistakes and correct for them, to my mind, is the only sane strategy.

Nikita Shamgunov: Yeah, so distributed systems at SQL Server - we always knew that was the future, especially as you go into the cloud transition… And at the time, back in 2008-2010, Microsoft had a flagship product SQL Server, which is a single-node database system. Very good, very powerful; I’m really proud I worked on this one. The main competitor, Oracle, had distributed systems at its disposal, Oracle Exadata and Oracle RAC. And the way the database market is structured is that the top-tier workloads that have high performance requirements, high availability requirements, do require distributed systems, and Microsoft didn’t have that. So that top of the market Microsoft was losing to Oracle then. Actually, I think they mostly caught up right now. But then, architecturally, single-node databases are very hard to change and turn them into a distributed system. That was some of the moonshot projects that my co-founder and CTO Adam worked on at Microsoft, and that moonshot project didn’t succeed.

When I walked into Facebook, the need for distributes systems became apparent, because every Facebook workload is that high-level, high-end workload… And sometimes it’s from the reliability standpoint, sometimes it’s from the scale standpoint; most of the time it’s from a scale standpoint… Because you know, back in 2010 I think Facebook was on the march to cross a billion active users. So that was on everyone’s mind, that was everyone’s goal, “How can we cross a billion active users?” and obviously, history shows that Facebook had blown through that goal quite successfully.

Jerod Santo: [01:24:01.11] Well, I’m glad you remembered that part, because that’s super-interesting, and definitely exciting. So Matthew, we’re hitting up against our time… Thanks so much for joining us today. All the hard work you’ve been putting in for many years on this slightly hamstrung, but much more pure moonshot that you’re on. We hope you have a good landing on the moon, everybody’s safe and sound, and you hit your goals. It looks like you’re well on your way.

Matrix is very interesting, very cool. All the links to all the things related to what we talked about, of course, are in your notes, listeners, so check those out. What’s the very best place for somebody who wants to get involved, from a developer perspective? Maybe they wanna check out this P2P stuff… Is there a waypoint for community members/collaborators, or is it all inside of Matrix’s guardians of the galaxy thing? What’s your guardianship? Is there collaboration points for people?

Evan Sparks: I think it depends on the company and the scale of the application that’s under development. For example, if you’re building a self-driving car, that’s probably not a job for a single data scientist. [unintelligible 00:17:04.01] I’d call it a generational moonshot, if you will. And there it makes perfect sense that you’re gonna have a massive team of people just worrying about data labeling, and data ingest, and ETL. Another set of people just working on the perception pieces of the job. Another set of people just working on maybe a different component around path planning, and so on.

There, in those scenarios, you really wanna think about “Okay, what are the various teams and what are the personas of the users of a broader machine learning platform? What do they care about, and how do we facilitate coordination and communication between those teams?”

In other cases, companies have done a really good job of cleaning up their data, putting it into massive data warehouses, and even making their feature catalog self-serve, and the kind of thing where a data scientist who says “Hey, I’m looking for a fraud model for mobile purchases in South-East Asia. We’ve decided we’re losing enough money on that particular area that a specialized model on this particular part of the world makes sense.” In those cases I do think that proper infrastructure can enable a data scientist to go from start to finish all the way, and ideally you wanna get with that person to the point where they don’t have to work directly with a data engineer to get the features flowing through the system, and so on.

In my view, almost more importantly - or a place that we see people get tripped up - is around deployment and monitoring of those models. We see people often taking models that are built in PyTorch or TensorFlow or whatever, and completely rewriting these things in C++ or Scala or whatever, because that’s what fits into the production serving environment. That side of things - we see these deployment engineers… That’s a job I would love to see go away in the common case, if the infrastructure gets better. You want data scientists to be able to get to the point where they’re confident that it works well enough on test sets, and maybe even start to A/B test it, and then hit a button and deploy it more broadly [unintelligible 00:19:09.26] is definitely the central thing we need as an industry in order to make these technologies more viable and successful.

Chris Benson: No, I guess I’ll go back to something that I know we both have said several times in this podcast - I would love it if the White House would go back and bring us something a little bit grander, and take a leadership position. For what it’s worth, I say this completely in a non-partisan way - get out there and lead this, and lead the world, and show the amazing things that we can do with this new technology that’s here to stay. I hope there is a round two of executive order that gives us that AI moonshot.

Todd Gamblin: Lawrence Livermore is one of three labs in the National Nuclear Security Agency. It’s part of the DOE, and we’re responsible for a whole lot of different national security missions. That includes things like nuclear non-proliferation, making sure that people don’t sell nuclear materials… We’re also responsible for the U.S. stockpile, making sure that it remains viable, and then we do work on a lot of just basic research projects. We do some climate simulation, we have people working on that, we’re part of the – I don’t know if you remember Obama’s Cancer Moonshot, we have people working on that initiative - there’s a lot of interesting data analysis and machine learning there - and just general computation.

[03:55] I think all sorts of missions at the Laboratory are based on computing and our Livermore Computing Center, which is where all the different clusters are that we run. I work both with the research organization and with the computing center, and also with the code teams, who are from all different parts of the Lab.

Jerod Santo: Well, VCs are fundamentally about the moonshots. They’re trying to ride the rocket ship, and then have an exit before it blows up. I mean, they’re not about a sustainable lifestyle. So of course, for them lifestyle - that’s an insult, like you say. But for open source maintainers, it’s all that we have, because the rocket ship - I mean, you’ve gotta rug pull to get your rocket ship, right? I mean, some people started in open source and made money in addition to it, but not I don’t think because of it. But for most of us, we’re just trying to –