Charles Martin: Right. So let’s say we’re training a model to generate text. There’s no test set, right? You have to read the text, and ask, “Okay, does it look human or not?” And that’s sort of where the first problem came, is that there are many problems in generating things. Another would be - let’s say you’re doing search relevance. So I’m trying to predict what somebody wants to click on. I have clients like Walmart, for example; we built these systems for them. It’s very expensive to run an A/B test. So you can test things in the lab, and you can like make a model, like an SVM model, to predict what people would click on. But you don’t really know how it’s going to perform until you put it in production. And there are all sorts of biases that exist in the data… Because there’s like presentation bias; people tend to click on things that are in the first element, and that screws the model up.

So there are many cases… Another good example is in quantitative finance, when you’re trying to predict the stock market. And you have models where you would like to train some neural network to learn something about how the news predicts the market, but if you train it directly on the market, you’ll overfit it. Always. So you have to have some way of evaluating whether your models are converging properly or not, without just looking at the test sample; a lot of data is out of sample, or you can’t really evaluate it without human judgments, or it’s very expensive.

Gergely Orosz: Very much so, in many ways. Yeah, I think that was an unexpectedly quickly heating up market. I was also very surprised to see just how quickly things have changed… And I guess now, hindsight is 50/50, but we should have known that if things can heat up quickly, they can also cool down quickly… And it has cooled down very surprisingly quickly. And we’re hearing news – there’s obviously news about layoffs, but it’s not just about layoffs. There’s the hiring freezes, there’s – looking ahead, some companies, some of big tech is not looking to hire, and a lot of the news is focusing on VC-funded companies and big tech; there’s also a silent majority that we’re not really hearing too much, but there’s the more kind of traditional companies, or companies that were not paying the top for the market… And there’s actually some good news there as well; it’s just, as with any news, bad news usually trumps out good news or no news, really.

Autumn Nash: Do you feel like we’re still going to be able to achieve that for people in this market the way that tech is like changing? Because I’m really worried that it won’t be that easy for people that didn’t go to college, or people that are philosophy majors; this new market, people with 20 years of experience are having a hard time to get in. And when I go to talk to like college students or military spouses, I don’t even know what to tell them right now.

Autumn Nash: Ooh, that’s an interesting upskill and pivot, because I think a lot of people right now are scrambling like “How do we upskill to make ourselves marketable in this new market?” That is a really interesting pivot.

Daniel Whitenack: Well, Chris, I have maybe a related question for you, which - it’s not exactly related to the hype cycle, but I was at my local co-working space for a fundraiser on Friday night, and shout-out to Matchbox co-working if anyone’s listening… But yeah, so that was fun. But I got into a number of AI-related conversations, as I usually do, and what one of the guys I was talking to mentioned was, you know, there’s a lot of people talking about how this sort of wave of generative AI, this wave of AI and what people are referring to AI now is being compared to kind of like the surge of – it’s like the new internet. When the internet was brought about, and the type of change that created. And his point was sort of “Well, it definitely created a kind of new market, this space that was an is the web…” And it wasn’t just about creating efficiencies. And his point was it seems like most people are using AI to create efficiencies in the enterprise, whether that’s helping write reports, or automate certain functionalities that interns were doing before, or analyze a bunch of documents, summarize those, answer questions, get quick access to information. And from his standpoint, these are all kind of efficiency gains, and not necessarily creating any sort of new market that would be comparable to the huge shift that happened when the web came about. So I was curious on your take of that. Maybe it’s slightly related to the hype cycle stuff, but…

Birk Jernström: And music was in this phase where everyone thought music was dying; everyone just expected it to be free. No one’s gonna pay anything for it. And here came Spotify along, and just like “There’s a new way.” And that became a success story. And they’ve flipped that narrative all the way around. Like, I think Shopify now is doing more than the entire music industry was doing back when they originated and started out. And I think venture capital fundamentally is taking these moonshot bets that can become something game-changing, and define a new market. And I think that’s what Polar really had going for it.

There’s a lot of players out there that are pitching great ideas, but probably great indie ideas at the end of the day… And and for us it was just “You have the developer ecosystem and the open source ecosystem, which are immense and incredibly valuable to the entire ecosystem of technology, but we haven’t cracked funding and monetization yet.” And I want to build a platform that can crack that code. And if you’re able to do that, I think that can have a really meaningful impact for those individuals, as well as for Polar as a business. I think that’s fundamentally what excited VC’s about Polar and what we’re building.

And then there was slides about all of the things that we talked about, like these different monetization tactics that we would offer eventually, and… But we launched and we just started out with issue funding. And so we’ve been just iterating one funding tool, and now soon one monetization tool at a time, to just continuously learn from users and continuously adapt, to hopefully solve it for as many people as possible.

Break: [00:58:24.26]

Paul Orlando: [01:07:38.27] Oh, that’s a great question, too. My hope is that smart, creative, talented people will be able to better devote their time. So they’ll still do things that they’re passionate about, but if it’s a question of “Hey, I’ve got a lot of options. I’m going to think about timing as a way to make a decision, and then I’m going to do something that I think will benefit from timing, that’s just going to help me along the way.” So related to that, I hope that our cousins in these large corporate environments, where maybe they have in-house labs, or they’ve got like 50 different potential products that they are trying to figure out how to put resources behind, or what to keep, what to kill, I hope that timing will become another way they filter opportunities and decide where to put resources.

Similar things for startups that are pitching for capital, or investors that are evaluating startups. We recognize timing is important. How could I actually step through that with some type of a process that’s helpful, and then get to a place where if I’m the startup, I either represent what we’re doing appropriately? This is why the time is right, in a Why Now slide. Or on the investor side, lots of things we can invest in, but we should be in this area. I don’t know exactly who’s going to win. I’m not predicting which team, if there are 100 different startups doing something similar… But we should have some type of place in this new market.

So I hope it’s things like that, that come out of this. I would really want people to apply this in what they choose to work on, and what they choose to invest in, and what they choose to do. But absolutely, founders do influence what happens in the world… So you can take this book and say “I’m still really passionate about this topic. I’m going to pull the future forward. I’m just going to do it anyway.” Or “I’m willing to wait the 10 years that I think it’s going to take for it to have a supportable business model. I’m gonna get through that in some other way.” But yeah, to your question, I really hope that it’s going to change the way people actually build or invest in companies.

Cory Doctorow: Cheating, labor market cheating, attracting subsidies on false pretenses, sucking up billions in subsidies and false pretenses, violating all kinds of laws in every way… Using subcontractor gambits to avoid having to pay for like maiming and killing your workforce… Yeah, Amazon did a bunch of historically contingent things to get to where they were. But – so let’s look at a couple of different possibilities. So one is Jeff Bezos got lucky… Or the way Jeff Bezos got lucky is he was born with a genetic mutation that made him Jeff Bezos, that made him the guy who was going to do this. So that’s one.

Another one is Jeff Bezos got lucky because he was born into a world with a Jeff Bezos-shaped hole in it, and he filled it. And without Jeff Bezos, there would have been another guy just behind him who would have done it.

A third one is that Jeff Bezos is descended from a line of would-be kings, and he drew the sword from the stone. And Jeff Bezos’ son, and his son and his son confound a dynasty, that from now on rule over us with the divine right of kings.

So we’ve had all those different theories of history in the past. And the question is, which one do you think is like more plausible? I think that Jeff Bezos is good at some things, I also think that he got really lucky… Because there are lots of people who are good at the same things as him. I think that if there hadn’t been Jeff Bezos, there would have been someone else who would have filled much of that niche. When you look at what it is he did, the tactics aren’t all that different from other people who did similar things, but he prevailed, often by things that amount to coin tosses… And what I reject is the idea that he’s King Arthur. And yet, the dynasties, the kind of wealth that Jeff Bezos has established are self-perpetuating. In the absence of some competitive force that allows for new market entrants, in the absence of like periodically burning down some of the forest to open up some space in the canopy, so that new shoots can develop, he lasts forever. And his children and his children’s children govern forever. You get this dynastic wealth. And not only do I think that’s not true that he is King Arthur, and not only do I think that even if he’s good at his job, I don’t want his kids, and his kids’ kids running the business, I also think that when you have a society built on those lines, it becomes extremely unstable. Because the legitimacy of a society, the reason we buy into its institutions, and its norms, and we accept its rules, is because we think it’s fair. And unless you’re prepared to accept that you are a peasant whose destiny was to be bound to the land, which - the peasants didn’t; they had uprisings. Unless you think that that is your destiny, then living in a world in which that is all that you can hope for, is a world in which you will cheat any chance that you get, in which the system will fall apart. And so either we have a fair system, or we have no system. I don’t think we have an intermediate system; or the intermediate system that we get is an authoritarian system that is intrinsically brittle, and that collapses in the most awful way imaginable.

Jeff Geerling: And profits are down, but they’re still profits. And you’re saying, “Oh my gosh, oh, please don’t hate us, because we’ve got to pay our developers.” Well, then maybe cut your profits a little bit. But they can’t, because it’s a for-profit corporation, and they have to please their investors, and the board and all that. But it’s like, you can’t use that line; if they were a not-for-profit, sure. “We want to pay our developers in it.”

In the Drupal community, we have had that conversation, because the Drupal Association, which is a not-for-profit, has had hard decisions. When times get tougher, how do we keep these developers who are working on the Drupal infrastructure employed, and pay them? And so we all band together, and the community contributes, and we get that. But you don’t get that when you’re a for-profit company advertising billions of dollars of profits in your new market segments that are leeching off the Kubernetes ecosystem. Sure, you can contribute back, but you didn’t build all of Kubernetes yourselves.

Paul Hünermund: [06:09] Yeah, so maybe it helps if I’ve defined first what I mean exactly with causal inference, because there is actually a neat definition by James Woodward, which is a philosopher of science, who said that causal inference or causal machine learning is a special kind of prediction problem, so in a sense, it is a prediction problem, but here we are predicting the likely impact of an action, intervention or manipulation. Really this idea of I do something, like I increase the chocolate consumption in a country; will that produce more Nobel Prize winners? …in this context.

I think if we approach from that perspective, you immediately see the value for business. Because in business, we are always asking these kinds of questions. What if we do X? What if we implement this new HR policy? What if we enter a new market? Should we invest in this product, or another product?

So business always involves actions, interventions, and we want to forecast, predict the likely outcomes of this. That would be what causal inference people call the interventional level; then one level above that is the counterfactual level. So counterfactual meaning we’re reasoning about two states of the world. Had I not taken the aspirin this morning, would my headache be worse today still? These kinds of questions. And they also very relevant in sort of hindsight retrospective, like “Was it, again, HR policy that we implemented that improved employee satisfaction?” and so forth. So immediately relevant in all sorts of domains in the business world.

Specifically in AI, we’re talking about fundamental problems in AI, which is fairness, robustness, explainability… And I believe that causal AI has to say something in all of these domains. So there is also an immediate practical value in this.

Beth Rudden: I have a couple. I do think that with the technology that I have, and with an ontology, I can take like a paragraph of your text and understand something about your existing mental model. So if I can relate new information to your existing mental model… Like, I found out you live in Indiana; if I say something about Indiana, that makes you not only trust me, it makes me give you new information in a way that reduces your period of disequilibrium. We can make people learn faster, and that’s truly exciting to me. And I think that human beings, given trusted, evidence-based, tested artificial intelligence - if they had that, I think that it opens up this entire world of visual thinkers.

And shout-out to Temple Grandin and her new book, Visual Thinking… All about how we’ve been living in a verbally-dominant society. Well, guess what? Words just became very cheap, very much of a commodity. So the engineers, the tradesmen, the plumbers, the people who are doing the things with their hands, the artists, the fuzzies - this is our time. And I think that you’re opening up an entirely new market for everyone to be able to create. And that to me is super-exciting.

Kenneth Auchenberg: I actually think of infrastructure in the same ways, in terms of it being like pieces of components that you can leverage to kind of build the experience that you want to provide. But I probably don’t have a tight connection between infrastructure and hardware, in the sense that I see that as an abstraction ladder. Like, let’s say that I want to build a PaaS today. You will probably rely on another computing abstraction that you’re using, that is so far away from hardware anyway, that you have a virtual machine of some sort, and you have another runtime. So even as you kind of compose kind of like a new kind of computing abstractions as infrastructure, hardware for you anyway is abstracted away.

[30:12] So when I think about infrastructure in a DX or in a software context, is that we think about how I’m composing a modern, let’s say software as a service business, I probably need some sort of compute layer; I probably need like a Vercel, or a Netlify, or AWS Lambda, how I’m putting my stack together. But at the same time, I probably need an authentication piece. That is not a hardware piece, but it’s a set of services I’m using to provide auth, right? Or maybe even I’m using a combined provider that does all that for me, like Firebase.

So what I mean by that is that to me, infrastructure is the business components that you’re leveraging to kind of put your product together. And the argument I’m making for DXI as a category is that we have reached this maturity point where if you have to provide all the aspects of a great modern developer experience covering your product, docs, content, community, you cannot build it in-house anymore. You have to rely on these new infrastructure pieces that are emerging from this new market landscape of companies that are saying, “I want to be the best in providing SDKs. I want to be the best in providing documentation infrastructure. I want to be the best in doing community management.” And then I as the founder, I as the C-level at a company, instead of sitting with my budget and saying “Should I allocate 100 engineers to build internal developer experience?” No, I would rather use a different part of my budget, and use a vendor for these kinds of things.” So it’s through that lens that I’m seeing infrastructure.

Will McGugan: I think of these Textual applications not as replacing what people use web applications for, but I think because it lowers the barrier, those applications which simply weren’t being built because it would take too long with a web application, or they didn’t have the skills - I think a lot of these could be built with Textual, because you just need Python skills, which a lot of people have, and it’s just easier and quicker to build them. So I think there’ll be not cutting off a part of the market, but creating a new market for applications which just were never built in the first place.

On Freund: By the way, countries are one of the biggest hurdles, and one of the biggest needs for technology. So just being able to bill in multiple countries is super-complicated, especially when you have customers that are cross-border. you have Amazon in Germany and Amazon in the US, or something like that, and you need to bill them… It’s super-complicated. And what - we actually went to a lot of billing companies, and we asked them, “Let’s say we decide on a new market we want to open. How long would it take you to add that country’s billing to your system?” And they said, “Well it would take us about two years to add a country.” And we’re like, “We’re going to open a building in six months. Actually build it, and do everything around it… And it’ll take you two years just to do the billing?” And we ended up having to write our own billing system because of that.

Paul Orlando: Sure. So I’ve worked in tech my whole career; I have been just about always on the business side. I started out early on doing voice-over-IP work… This is a little before Skype was around, so it was about connecting telecom carriers to each other, and routing traffic differently… And that’s certainly when I got a great appreciation for how, in that case telecom networks, can lead to these unusual outcomes. We can dive into that maybe later on, if it’s a fit… But I had the experience also of starting a startup, and making - you know, not 100%, but maybe 90% of the possible mistakes in doing that… And just really getting interested in how people figure things out, and how things end up being differently than you expect.

In our case, it was also a telecom-related business. What we ended up doing was connecting patients who were going through a serious health situation, connecting them to another patient, and that was part of their recovery process. So you get to talk to another person who has kind of gone through some difficult recovery, and we protect the patient privacy, and we would kind of push out whatever their doctor, or support group leader wanted afterwards. It could be a survey, it could be “Here’s some health goals for you this next week.”

But getting to that point was this jumbled – certainly not systematic process for us, where we had all these different ideas of how people were going to use it, and then were surprised in what ended up emerging, and what people were happy to pay for later on.

[07:52] But I got interested in that process, like how founders figure things out, or often don’t figure things out. I was in New York at the time, and I ended up visiting a bunch of startup accelerators, just to mentor, meet people, and kind of talk and (I guess) coach back then… And I realized I wanted to make that a bigger part of what I did.

New York back then - this is like 2011-2012… New York had probably ten startup accelerators, and I didn’t think it made sense to try to start the 11th one… So I was looking for a new market to enter. And earlier in my career I had actually worked in Hong Kong, I had worked in China a lot, other parts of Asia… So I did a scouting trip, just to try to suss out, “Okay, maybe Hong Kong, or maybe some other location in Asia might be a good fit for something like a startup accelerator.”

I had landed, with a bunch of meetings set up, I met a ton of new people, I got introduced around in Hong Kong, and ultimately determined, “Okay, yes, this market is ripe for something like a startup accelerator.” I kept the conversations going when I went back to New York, but I ultimately spun up a pilot program, and then raised a small fund to support an actual startup accelerator, and ended up building the first program in Hong Kong.

From there, I ended up – I got pulled in to run this unusual startup accelerator that was based in Rome… Now I’m helping this big non-profit build a community health-related accelerator, and day to day I’ve been at USC (University of Southern California). I teach there, and I also run the university startup incubator program.

So I’ve kind of been in this early-stage venture space for a little while, and… I don’t know, for me there’s a lot of overlap with the systems or unintended consequences interests that I have. It’s certainly a side-project, but that’s kind of my process for how I got here.

Adam Stacoviak: Yeah. It’s a whole new market for Depends.

Danielle Morrill: [51:48] I made this spreadsheet that I think really helped me… Basically, it lays out what are all the possible things that could happen, like big picture - like, sell the company, turn this thing around, go in this new market… Whatever the buckets are. And then what would the impact be on the core constituents that I care about? So my employees, my investors, myself, maybe my co-founders, my family… Whatever you feel are the core constituencies that you are responsible for representing. I hope I said customers in there; if not, the customers should definitely be in there.

And I just kind of filled in the grid, and I started to look at like what would just be deal-breakers? What things am I not willing to do? Maybe one of the scenarios is just so bad for my investors that it’s not acceptable, or it’s so bad for my family that it’s not acceptable. And then I think from there, kind of getting a sort of stack or make sense of “What are the outcomes or the scenarios that I’d be okay with and how okay would I be?”

The reason to do this is I think this is actually where there’s a lot of fuzzy thinking… I feel like it’s not okay or it’s not kosher to talk about selling your company, and I think that’s why this conversation with my co-founder and husband about what would life be like afterwards was so powerful. So I would just say rather than waiting till you’re in a really bad spot, laying these scenarios out now and maybe seeing new scenarios you weren’t considering because you just took the time to think about it is very powerful, it lets you feel like you’re in control, and that you’re choosing what’s happening… Even if maybe your choices get crappier and crappier as things don’t work, you’re still actively choosing and maybe you’re adding new options to the list.

I ended up making that and I ended up sharing it with my investors when we got to a certain point, and they actually pointed out some other deal-breakers or possible scenarios or twists on the things that I had put there. And I think just giving yourself permission to play out all the possibilities in the privacy of your own spreadsheet is something I think for me made me feel like “Hey, I don’t like this, but at least I’m pretty much seeing the full picture, as well as I possibly can.” That’s sort of tactical, but I wish somebody had maybe encouraged me to do it 18 months even earlier, because I’d probably had more freedom than I realized to change the way things were playing out.

Justin Searls: You know, it’s interesting, too… I think Google is always – Google, Meta, and to an extent Microsoft, if you think of them more as a b2b company, although I think they’ve done a good job transitioning to being a consumer-oriented company… Google and Meta – you get good at what you do, and they don’t make their money taking dollars from Jerod and Adam, and Justin. The people paying them are somebody else, and all of the reinforcing feedback loops inside of their broader system. Marketing is like an accident that happens sometimes, and products or things that come and go almost as a way to keep engineers retained as much as something that’s vital to their business… So of course, it’s not shocking when they’re finally faced with an existential threat to the thing that makes them money, that they’re completely caught flat-footed and flailing… Because they don’t know how to ship a product. Sorry, if you’re a Google fan, or something. It’s like their organization’s lifeblood, that is new revenue and profitability, has nothing to do with the normal cadence of bringing something new to market for actual users to use and buy. Gmail and Google Maps, the two most famous things that everyone loves, were total accidents.

Daniel Whitenack: Yeah. And I think this just represents a sort of wider rift in the AI research community as well, where you kind of have a race to dominate the market with new models kind of at odds with this really zealous promotion of like trying to prevent AI from advancing beyond our control… And so you see this playing out not just at OpenAI, but elsewhere. And I think that led into some of the Q* craziness.

Well, in terms of retrospective, as we kind of – you know, we’ve told the whole story, we’ve stepped back and looked at OpenAI… There’s definitely, from my perspective, some takeaways. Of course, I’m actively leading a company that is providing hopefully trustworthy LLM APIs in enterprise environments, that can be self-hosted… So my big takeaway, and I think what I was busy doing all week was answering people’s questions about “Hey, if I don’t have OpenAI, what is there?” And it turns out that there’s a lot. So it doesn’t necessarily have to be our APIs, but there’s a lot of options around using Enterprise models in a safe, secure environment, that can be deployed under your control… And yes, there’s kind of this balance to take around like “Hey, OpenAI’s APIs are really good, because they’re a managed service”, but they also have downsides. And I think you’ve seen this over time with other managed services, right? Like with running your own database. There’s a trade-off between relying on a – especially if it’s a one-off unique hosted service, that is a single point of failure, versus kind of hosting your own, or maybe having your data spread out with different databases across your infrastructure… You see some of that kind of infrastructure concern, and the concern around the resiliencies of these systems really playing out… And I think that we’ll see other companies rise to that as well. There’s going to be a new wave of these companies that play off of what has happened over the past week, to provide enterprise solutions.

Chris Coyier: No. I just wanna – you know, you can’t help but think about… Like a product release - a product has a release; you do a new feature, and then you go “I’m gonna market the new feature!”

Adam Stacoviak: Maybe even more particular with Jeff, who would buy them? If they are to market his new product, who’s going to license that? I don’t know… Maybe a small handful.

Alya Abbott: I mean, some of them are kind of standard things. Paid advertising, going to conferences and various kinds of events, and sharing Zulip that way… One thing that – another direction is kind of content. So we’ve had blog posts on various topics, we’re starting to – you know, one of the things that I talked… You can probably see my excitement about it - it’s this kind of side of community management, and getting folks engaged in an open source project… For example, we’re working on partnering with some organizations on blog posts around that kind of thing… And so just kind of getting the name out there in whatever way… Because I think, as you were saying, kind of the brand recognition and just kind of awareness matters. People aren’t constantly in the market for a new team chat, but we want to be top of mind when they are starting to think about it, and when it does come up.

But yeah, I would say we don’t necessarily have kind of like something unique other than we do have this open source angle, and so engaging with community and like the open source community more broadly, and sponsoring open source projects is definitely one angle for us that we’re investing in.

Rob Mee: [01:03:43.25] e do use it. [laughs] Interestingly enough, we tried not to… So I’ll just say that one of our portfolio companies at Geometer is called Kohort, with a K… And sometimes I forget that cohort is actually spelled with a c. It’s in my brain now… But that team is led by Dan Podsedly, who led the Tracker team for many years. So he’s the CEO of Kohort. And he said, “You know what, I don’t want to make the team use Tracker just because I ran Tracker all this time.” And I think he was challenging himself to do something else. He said, “We’re gonna go use something else.” And so they picked something that was getting a lot of traction, that was relatively new in the market, as the best thing that they could find. They started using it, and after a month, the team said, “Can we please just go back and use Tracker?” So reluctantly, Dan went back to using the thing that he built. So yeah, it’s still useful.

Steve Jobs: One of the things you learn when you start building factories is that warehouses are really bad, right? Warehouses are bad, because you tend to put things in them. And inventory is really bad; inventory is really bad because if it’s defective, you don’t find out about it for a while, and you don’t close the quality feedback loop with a vendor, and correct the problem till they’ve made a zillion of them. What you want to do is find a problem, the first one that comes in the door, and stop them from making more until you fix the problem. So warehouses also cost money, because you put all this stuff in them, and the stuff - you have to go borrow money from the bank or use money that could be used in a more productive purpose. So warehouses are bad, and you want to go to JIT; I’m sure you’ve studied this all, and studied examples.

I was walking through the Mac factory one day, and the two biggest pieces of automation we put in were a giant small parts storage and retrieval system; there were these totes that ran around. And the second one was this giant burn-in system at the end; a few tens of millions of dollars worth of equipment. And I realized, unfortunately too late, that both of them are warehouses. They’re just high-tech warehouses. So when we looked at NeXT, we said, “No warehouses of any kind. We have a true JIT factory. Stuff comes in and is delivered right to the point of use on the factory floor. There is no warehouse. Deliveries are made daily, sometimes more frequently than that. There is no outgoing warehouse. Everything is visible.”

And the reason that we were able to do a lot of what we’ve done is because when we were learning about manufacturing Mac, we hired a Stanford Business School professor at the time named Steven Wheelwright… And he did a neat thing, he drew on the board a chart. The first time I met him, he said, “You can view all companies from a manufacturing perspective this way. You can say – there’s five stages. Stage one is companies that view manufacturing as a necessary evil. They wish they didn’t have to do it, but damn it, they do. And all the way up through stage five, which is companies that view manufacturing as an opportunity for competitive advantage.” We can get better time to market and get new products out faster. We can get lower costs. We can get higher quality.

[47:58] And in general — you can sort of put the American flag here, and put the Japanese flag in here… [laughter] And that’s changing, however. That’s changing. And it’s changing because people like you are going into manufacturing. Big companies are starting to realize that we were great at this one time, and then we took it for granted. Ad people are starting to pay good salaries now and get good people. So we want to be one of these and we try very hard.

By the way, just going back to software for a minute… I often apply this scale to computer companies and how they look at software. See, I think most computer companies are stage one - they wish software had never been invented. I think there’s only three companies here, and that’s us, Apple and Microsoft in stage five. We start everything with the software and work back. But anyway, going back to manufacturing… We started looking at the factory as a software problem. And the first people we hired in the factory were some software engineers; we convinced them to move from R&D into software, which was not easy. We had to give them bonuses, we had to cajole them, we had to promise them they could come back if they hated it… And they went over there, and we said “This is really just a software problem with interesting I/O devices called robots, that’s all it is.” And so we started building the software first.

And our first robots that we got, we spec’d them out, and we bought them completely turnkey, with the robot arms on them and all the electronics, and the software to control them. And we spec’d it out, but we didn’t write it. And they worked okay. Some of them are still in use, but they weren’t great. And being software folks - we weren’t real happy. They weren’t elegant. We couldn’t do what we wanted with the robots. We couldn’t tie in a quality information system to them, and all this other stuff we wanted.

So the second generation, we spec’d out the hardware and had somebody build the hardware for us, but we wrote all the software on our own computers. We’re object-oriented, so we started writing robot objects, quality objects, all sorts of objects to control this factory. And we found out our computer was great for it. And so our whole factory now runs on this object-oriented factory and quality system. The last generation, our latest generation of robots, which we’ve deployed this year, we actually built the hardware.

I’ve been to Japan a lot of times, maybe 30-40 times, and I loved to tour factories over there… And they always amaze me, because they built everything themselves, they weren’t afraid of anything. They needed a robot—they’d try to buy one, but if they couldn’t, they’d actually engineer it and build it. And you’d think this was really expensive, but we found out it’s pretty cheap. It’s actually cheaper than buying them. And so we’ve actually now designed and spec’d our own robots; we don’t mill the metal or anything we get them all made we put them all together, and we do the software top to bottom, and we have now some extraordinarily advanced robots in the factory. And our computers are built start to finish on the key components, completely untouched by human hands.

Michael Malyuk: Yeah, I think it’s – if you look at the whole ML market, it’s pretty new, and there was a lot of really good developments in the infrastructure world. And because of that, the infrastructure for ML kind of becomes a commodity more or less right now… And where are your actual strategic investments [unintelligible 00:06:45.10] how you process the data and how you prepare the data, especially if you’re working with data that requires some sort of expert knowledge to be labeled… So everything around how you capture those expert knowledge into the way of labeling data - that’s a very liquid asset that you have on your hands after you are done.

Kyle Daigle: Yeah, there’s all kinds of very interesting things that you have to do when you have a website like ours, that is under a tremendous amount of traffic, but is also a 10 or 11-year-old Rails application. GitHub has from the beginning always really cared about the product and the user experience of what was happening, and in a lot of ways sort of slowed to the microservice application that we all kind of experienced over the last 4-5 years, when that became very en vogue; it’s been around forever, but it became very en vogue.

[43:55] So for us, the thing that we’re always sort of figuring out is “How can we make a technology choice that lets us move quickly, in service of bringing a product/tool/experience to our customers?” So you end up doing these tricky things where you think, “Okay, maybe I pull this service out, so that way while we’re iterating very quickly, we don’t have to deal with the entire application while we’re doing the launch editor, or the parser, or the scheduler?”, all these little things. But in a lot of cases, there’s things that should be in the app, and for what powered actions there’s some pieces that are in the main GitHub application. There’s all kinds of little tips and tricks as we bring new features to market, that we have to make that choice. Because I don’t think it’s as simple as saying that there’s a single technology approach that’s just gonna solve all our reliability problems, or solve all the customer problems, or product problems… And this gave us a tricky way around that when we already had a solution like it for all the 3D and image rendering.

Adam Stacoviak: It’s the new search for me, too. I’ll share a shorter version of it… I do want to concur though, a big shout-out to Min Browser. That’s super-awesome. I recently used ChatGPT, and this was actually the 4.o model. It wasn’t the 0.1 model. I think I was just, you know, silly for a day… Didn’t swap out to 0.1. Anyways… I’m in the – I think I’ll be in the market soon to get new tires. And I know that when you buy truck tires, it’s always like “Well, you want to look cool, you want to have the MT, right? You want to be a cool dude.”

Chris Shank: [00:09:49.23] The component paradigm has gotten us really far. It really enabled us to – like, if you go back to 2012, 2015, whenever that idea started and started to spread, it helped us build more complicated user interfaces on the web. There’s no doubt about that. And I think because of that, we’ve hit this wall… And part of that wall is the limiting factor of a lot of these web applications that we see today, I’d argue are not complexity at the rendering level. They’re complexity at the behavioral level. One of the things that I think we’re really lacking is a set of “Here’s all the different types of web applications you can make.” The ones that I’m thinking about are partly what I worked on; we’re streaming real-time market data, we’re contextualizing new stories that are also streaming in real time with that market data… There’s an endless amount of APIs and data that you want to integrate. Outside of FinTech you have real-time spatial canvases, multiplayer experiences… The type of applications that we’re trying to build today are a lot more complicated than they were 10 years ago.

I’ve had anecdotally experiences of “What the hell is happening in this application? Why is this data coming in wrong? Why are our streaming services being canceled and restarted?” And then there’s also this other aspect of - the UI is staying the same, and all of the bugs and the additions that we’re adding to this application are behavioral. It’s just “Oh, we’ve got to multiply X, convert this currency”, or whatever. And that doesn’t happen in the UI, or how we render things. It’s happening in the behavior, 20 yards away. And I think we really lack primitives and approaches to understanding applications better. We really struggle to see how “Oh, a user interacted with something, and there was a click event. What is the application doing?” And part of that stems from this tension of co-location and components sort of being really encapsulated things. Though Reason React aggressively wants you to co-locate things is “Oh, it helps you think reason about the application.” It’s “This button or this table is just its own thing”, and you just need to think about state inside of that thing. And it’s like, well, what happens when this table is talking with the drawer, and affecting other parts of the application? The behavior that we want to co-locate is getting hoisted up and up and up. And then it’s like, well, when now we don’t have co-location, we’re just passing all this data down. And so when you want to say “Oh, this cell in a grid was clicked”, it’s not just the side effects and the thing that the user interface needs to do in those cases is not encapsulated to the table.