Jerod Santo: Yeah, I mean, everyone’s different, but… Okay, let’s not say Jerod Santo. Let’s say I’m a mid-level engineer at an insurance company who writes Java nine to five, and I’ve got a backlog of JIRA tickets. You know, I’m a typical software engineer. Do I wait? Do I dive in? Am I vibe coding in my free time? Do I have agents working underneath my desk, at work?

Bert Hubert: Yeah, they’re still rocking it over there. So it’s not necessarily – you could come at a point in your life where you say, “Look, I don’t need all these new things anymore, and I have only so many years left in my career, and I’m going to spend them in a basement, in some kind of insurance place.” So it’s not all bad, but it is something that I think you would have to just ponder for yourself, and say “Am I going to invest also?” Because there comes a point in your life - and I think we’ve touched on this in this presentation also, where I’m just saying “Hey, these 2000 dependencies, they’re all bad.” And there is a risk that at some point in your career you go “Look, things used to be better, and all this newfangled stuff is just wrong.” And when you hit that point, you have to have a real good think, and think about “Am I leaving the industry behind? Am I just old now? I’m just sort of a sad person that sits there and say “Well, in my time…” And you should at least – when that happens, it should be a conscious decision, where you say “Okay, I love this old stuff more, and I’m not going to pretend that my old stuff is better. I’m just going to admit that I’m really good with this old stuff, and I enjoy doing this old stuff. And I’m not going to tell you that Rust is wrong, or that Swift is wrong, or that whatever new thing is wrong. I’m just going to enjoy my old stuff.” But it’s really sad to see some people see new things happening, and they just get stuck in telling you how great their old stuff is.

Elecia White: That was really cool technology, and had some amazing applications. I’m kind of glad I left… I mean, I am comfortable with the level of privacy associated with it, but I am less comfortable with how the company started being run.

Children’s toys… That was fun. I got to take some of my toys to a kindergarten, just volunteering. And one of them that I hated - it was a karaoke, and it was the most annoying toy. It would just sing constantly. And there was a voice changer that did not make anybody sound better. And I thought it was a dumb toy. I was way more interested in this other toy that helped kids learn to write.

And I would take the toy to the kindergarten, and kids would play with them, and I’d watch, and it was fun. And then one of the boys, who was kind of behind, asked if he could take the karaoke home, and I was like “Good riddance.” And a week later, he had caught up to his class. He had been really behind, but because singing was the way that worked for him… Yeah, it totally blew my mind that something so different than what I could have stood, worked.

I’ve worked on DNA scanners, twice now, actually… Once pre 1995, ’98… Well, anyway, once when DNA scanners were really cool, and very hard. And more recently, when – like, you can just make them on your desk. That progress has been just phenomenal and amazing.

What else have I worked on…? I worked on inertial measurement units, which used to be the way, when I worked on them, that I would make anybody I was sitting next to in a plane just stop talking to me. “Oh yeah, I work on inertial measurement units.” I could have said “I sell home insurance. Are you interested?” No, inertial measurement units was so good. People were just asleep by the end of the words.

[00:16:06.03] But I didn’t tell them what I was doing on this airplane… I was going to a racetrack, so that we could test them on high-speed vehicles, or airplanes, or any of the other places where inertial sensing becomes really interesting.

This was the day before Wiimotes. And so the whole idea of accelerometers and gyroscopes as being interfaces was very different then. Now you flip over your phone, it turns on. That’s what you expect. But how does it even know? I think sensors are probably my favorite area of embedded systems right now.

Yann Lechelle: …we released the very first official Scikit-learn certification program. And what’s amazing is that we – so this is the first time, so we’re doing it step-by-step, and the system works, people can register, they can pass or fail the test… But without advertising, we had like within a couple of days 600 registrations, all over the world. A lot from India, actually, because people in India, they do work also remotely for other clients across the world, and so they do need a stamp of approval to showcase their ability to provide a service. So very interesting that this brand almost instantly can promote a sort of service that is value-adding. So that’s the one thing.

But then on a more technical level, I fell in love with one new feature that came out with 1.5 of scikit-learn, developed by another co-founder and core developer, Jeremy… And that is the callback feature. Why? Because Scikit-learn in fact is a platform - it is a platform - and the callback feature allows us to provide extensions, if you wish, where people can hook into the inner workings of Scikit-learn as they are building new models. And in fact, I find that to be essential, because we are entering an age of liability with regards to AI. Companies need to be able to introspect. They need to actually find out why the model is producing such and such results. And so introspection is critical.

And as I said earlier, deep learning is sort of a black box type of approach, which I love, by the way. Again, in 1992 I was building deep learning models in the middle of the winter of AI at the time. But Scikit-learn is actually quite introspective, quite transparent, frugal, as I said… And so callbacks are yet another feature that provides actual introspection into how we build models. Because - talk about insurance companies, fraud detections - you’ve got human beings at the end of the spectrum being handled by algorithms. And so that is critical. And I think we fulfill a very important need with these features. So again, Scikit-learn, the gift that keeps on giving, and I’m impressed every day, with a bit of an imposter syndrome, because that team is just so powerful with this tool.

Adhithi Ravichandran: Yeah. He’s been super-supportive. A lot of the risks I was able to take was also because he was in a stable full-time job, with health insurance and everything. So I was like “Alright, I’m going part-time now that we have the baby.” And then I was like “Now I’m going to go to a startup.” Or – I was able to do all of that stuff because I knew I had like a support system. And then once I got the stability in that business, he was able to take a risk too, and start.

David Berenstein: I think we recently had this company from Germany, [unintelligible 00:22:17.21] over at one of our meetups that we host, and they had an interesting use case where they collaborated with this healthcare insurance platform in Germany. And one of the things that you see with large language models is that these large language models can’t really produce German language properly. They’re mostly trained on English text. And that was also one of their issues. And what they did was actually - they had a huge classification and generation pipeline combining a lot of these techniques where they would initially get an email in that they would classify to a certain category, then based on the category they would kind of define what kind of email template, what kind of prompt template they would use… Then based on the prompt template, they would kind of start generating and composing one of these response emails that you would expect for a customer query request coming in for the healthcare insurance companies… And then in order to actually ensure that the formatting and phrasing and the German language was applied properly, it would then, based on that prompt, regenerate the email once more. So prompts in LLM to kind of improve the quality of the initial proposed output. And then after all of these different steps of classification, of retrieval-augmented generation, of an initial generation and a regeneration, they would then end up with their eventual output.

So what we see is that all of these techniques are normally combined. And also, a thing that we are strong believers in is that whenever there is a smaller model or an easier approach applicable, why not go for that, instead of using one of these super-big large language models? So if you can just classify “Is this relevant or is this not relevant?” and based on that actually decide what to do - that makes a lot of sense.

[00:24:11.20] And also, one of the interesting things that I’ve seen one of these open source platforms, Haystack, out there using is also this query classification pipeline, where they would classify incoming queries as either a key terminology search, a question query, or actually a phrase for an LLM to actually start prompting an LLM. And based on that, actually redirect all of their queries to the correct model. And that’s also an interesting approach that we’ve seen.

Autumn Nash: Yeah. Well, it’s about real life, and it’ll be like crazy; it’ll be like “This guy BSed his way through being a surgeon, and now he’s like operating on like patients, like as a neurosurgeon.” And then this dude put a huge life insurance policy on himself and decided he wanted to leave his family and crashed his like plane and showed up like 10 years later. It’s wild. But it’s like all these really crazy, like, things. But that sounds like that, but with technology, and the FBI, and I’m just like “I would totally read that book.” That sounds wild.

Adam Jacob: Yeah, so initially it was going to be a hosted SaaS. So this was also the moment – the rise of SaaS. So Salesforce had launched a few years earlier, AWS had just begun, so like S3 was just happening… EC2 had just happened… And so like SaaS was just beginning. And so our idea was that we were going to run a hosted service, and then if you wanted to run it on-prem - which you’d be a dummy to do - you would be open source. And we were just way too early for hosted configuration management. Like, way too early, you know?

And so yeah, we launched this hosted SaaS product, big multi-tenant SaaS service, that did okay, but didn’t do great. And all of the large customers, banks and insurance companies and stuff would just run the open source, because they weren’t willing to run a hosted product. A decade later, they would all come to me and be like “Can I just buy a SaaS?” And I was like “F**k you guys.”

Johnny Boursiquot: Somebody, somewhere – it’ll be like your old dentists doctor’s office, or something, you need to send some proof of insurance, blah, blah… And they’ll be like “Can you fax that to us?” I’m like “What?! What do you mean?”

Daniel Stenberg: Sure. It’s actually pretty easy. So the curl project is completely separate and standalone. So I don’t do business with the curl project. I support curl stuff. But I do that separately. So I sell curl services. There’s an American company called wolfSSL. And my primary curl business is just support on curl. A little bit an insurance thing. I sell a number of issues per year, and I have a guaranteed response time. So my customers, mostly actually big American tech companies, they have paid me a yearly subscription, basically, and they file issues, and I help them when they have issues. So I make sure that their curl use is uninterrupted and works well in their products.

[52:25] Usually, companies where curl use is deemed important enough for them to do this. Even though I usually, of course, try to tell a lot of my potential customers that it’s much better if they pay me to do that, rather than spend time for their developers to try to figure out how to fix curl… Because I probably do it much faster and much cheaper than they having to waste engineering time on figuring out how to do things, figuring out even just how to, or just finding or fixing bugs even more.

So that’s what I do. And then of course, in addition to just supporting, it’s also more feature development and contracts, more working closely with the product development teams really, and how they use curl, and debugging their applications using curl. Because very often it’s not a problem with curl, but maybe in how they use curl, or in the area between; it’s hard sometimes. And having an NDA and contract in place makes them feel safe to share their code with me. They wouldn’t dream to sometimes submit extensive explanations in the public bug report, because they’re scared that they’re never really special source code would be something special for the rest of the world to see. So it works pretty good. Still a challenge to sell support, so something that is free, because it’s free. So why would I pay you when it’s free…?

Denis Yarats: Yeah, that’s definitely been very fascinating almost two years, I guess. I think in general web search – so what do you want to ultimately get as an answer to your question, right? So the current iteration that we have with Google and all the other classical search engines is an approximation of to get to this point, right? So you your question, you ask it as a formal query, and then you get a bunch of documents that are very relevant, but you have to still do additional work to get to the bottom of it. So you have to scan through the documents, you have to for yourself understand what “Is this a true answer? A not true answer?”, you have to trust the information. And as you can imagine, it’s a lot of work, especially if you’re trying to search for something very complicated, maybe things that are not so obvious.

And wouldn’t it be nice to avoid that step, where you ask the question and you get an answer right away. So that’s the ultimate destination where we’re trying to get. Obviously, it’s been tough to get there over the last decade or so, there has been a lot of work, but it never quite worked. There is this much higher level of hallucinations, much higher level of maybe not perfect synthesis of the information… You basically get a Frankenstein… So instead of a coherent and nice, easily parsable and readable answer, you get some just basically extracted pieces of the information and just concatenated together, so not very pleasant. And it’s funny that when we started – so one of our angel investor was Jeff Dean; he requires no introduction. And he was saying Google actually wanted to always build something this, but because they had such high expectations for accuracy, because millions and billions of users are using Google… And if you hallucinate 1% of the time, you’re gonna get a lot of unhappy people. And so they were never able to – because the models were not as strong as they are right now, they were never able to get to just 99.9% of accuracy. And that’s why like this work never panned out.

[00:06:20.00] But something great happened in 2022. When we started our company, both myself and Aravind, my co-founder, we come from academics, so we’ve been doing a lot of research in language modeling, reinforcement learning and stuff like that… And he was actually at Open AI at that time. We’ve been very literally following improvements of GPT models, GPT 2 and then GPT 3; that’s where it actually got very interesting. And it became obvious there was going to be something there. And this was primarily the motivation for us to start a company.

We wanted to build an answer engine from the get go, but it was very ambitious. I remember we would go to the investors and say “Oh, we’re going to build a search engine”, and they’re looking at you like you’re crazy, which makes a lot of sense. They’re just like “Oh, there’s Google ready.” And they had a fair point. But we still weren’t very discouraged by that. We knew there is something there, and we started prototyping.

So the first version of Perplexity we actually created as a slug board, or Discord bot, where it was a very primitive combination of a search engine plus - at that point it was DaVinci 2 models; it was still pre-ChatGPT. And it worked much better than I expected. It was very quick; we put up this demo in a couple of days. And you could already see that in certain cases it is very helpful.

Because this was a very early company, we were trying to hire one of our first engineers, and we didn’t know how to organize the insurance for him… And so we actually used this bot to ask those questions about insurance. Because if you go to Google and start asking the questions about insurance, you’re gonna get a lot of ads, and you’re just gonna get very quickly disappointed.

Ramin Mohammadi: Sure. I can do that. So first to give the introduction, what’s the PHI, or personal health identifiables… So based on the HIPAA rule, they are 18 identifiable, which can lead to identify an entity or a person within healthcare organizations. And these informations are valuable, and being targeted by hackers. One of the reasons is that they have a high value, because they contain your sensitive personal information, such as your medical history, social security number and insurance details, which makes it very valuable on the black market. They also use this for monetary gains. So a hacker can sell this stolen PHI to criminals who use it for identity theft, insurance fraud, or other legal activities.

They also use it for exploitation and extortion, basically. They use this stolen health information to use for blackmailing individuals or organizations. So 133 millions healthcare data was breached in 2023, which means one out of three Americans’ life was affected. This means about 160% increase compared to 2022, and about 240% increase since 2018.

Eric Boyd: I mean, that’s absolutely the vision of where we want to go with this. Imagine, you had a personal assistant who just helped you get everything done in your life. This morning I had to like print out a new car insurance form, because my old one expired, and I didn’t remember how to do it… And you’re just like “I don’t want to think about this.” There’s mental load. It’s a minor task, it was something I had to do… Can I just ask an agent to go and figure this out and print it, and then can I stick it in my car and just be done with this thing? So yeah, I think that’s sort of this dream of “Can we have these assistants that just help us with so much of our lives?” I think it’s really exciting.

Bryan Cantrill: After season one’s debacle, their proponent at the insurance company is only looking to save money. That’s it. It’s not really any strategic value. So there’s the episode where there’s the uncharacteristic cold day in Silicon Valley, which of course is like a warm day in most other places; it’s like 50 degrees, or whatever… And it’s causing a bunch of data transfer that Richard’s not anticipating, and it’s like a financial crisis for Pied Piper. And it’s an interesting object lesson in not being transparent. Melcher’s got no idea what’s going on behind the scenes, doesn’t have any appreciation or doesn’t care at all about the technology, and is really only interested in money. So we’ve actually wanted to avoid – I’d actually used that example a couple of times, of like… Because we have had customers that come inbound, that are not a good fit for us, because we’re not sufficiently strategic to them. And we don’t want them to be in a situation where you want to take us in a direction that is actually not where we want to go as a company. And you want to be in a position where you’ve got some discretion over the costumers that you take. Now, I mean, I think what’s also great about that particular kind of storyline is they don’t have that discretion. They’re desperate.

Amal Hussein: I think people should also generally have car insurance, but you know… [laughs]

Adolfo Ochagavía: Yeah. And additionally - there’s benefits and drawbacks to this, but the kind of welfare system in a country like the Netherlands means that if you have, I don’t know, a very serious illness, you won’t be bankrupt by that. So everyone has mandatory insurance, and you would never, let’s say… Under normal circumstances, you will probably never pay above 1,000 euros for anything in your healthcare.

Jon Calhoun: I feel like it’s one of those things that if somebody’s new and getting started with a new application, that some – like, I’ve found myself doing logging as an insurance policy; not as a long-term plan, it’s more of like I know mentally this is a short-term fix, knowing that as this thing scales, it’s not going to be sustainable. But I think you kind of have to – it’s like any other code, you have to kind of go in knowing that that’s the case… Because when you get to like Cloudflare scale, all of a sudden things that worked at 1,000 users or 10,000 users doesn’t work at billions of users. Like, it’s just a completely different world.

Adam Stacoviak: If you want the market to grow for the camp you want to hang out in, and not ever write JavaScript again, like you said, then you’ve got to put some insurance in place, and the best insurance is your wisdom, and directing folks to the right – to what you think is the way.

Kyle Wiens: Yeah. Well, and it’s frustrating, because you have a lot of really fantastic environmental organizations and local waste management organizations trying to do the right thing and trying to set things up, and they run the buzzsaw of opposition to new rules.

[00:46:08.08] Industries, the general stance is “Hey, let us do our work. Let’s try to minimize regulation.” And I can empathize with that as a business owner absolutely. But when you get situations, when you get externalities – there was a recycling center in the Bay Area that caught on fire and burned to the ground a handful years ago, and they have video of a consumer electronics device - it kind of looks like a tablet - going through the shredder, and then you have a battery that goes into a shredder, it’s already got maybe a bunch of suspended metal dust in the air, the battery provides a spark and you get an explosion… And in this case, the whole facility burned to the ground. And they have been told that if they have another fire, they will never be able to get insurance again. So now you’re in a situation where you have an entire Bay Area affluent community that might not be able to have municipal recycling anymore because they can’t get insurance to operate their facility.

Autumn Nash: [00:09:54.07] I remember when I was pregnant with my son, I was so sick… And there was a medicine, it was the only FDA Medicine approved for morning sickness that wouldn’t cause a miscarriage… And it was $700. But it was literally B6 and Unisom, which is like you could mix it at Target for like 12 bucks… And it was $700. And it took three weeks; three weeks of me not being able to hold down the water, for insurance to approve it. And I was like “This is ridiculous.” Like, some dude in an office is just turning this down while I’m so sick. It was just like – it blew my mind, that whole process of insurance and stuff.

Shaundai Person: Absolutely. So that sparked the idea that “Oh, maybe I could do this engineering thing.” But shortly after that, I got pregnant with my son, and I was like “I have to go get health insurance, so let me go back into the workforce.” I went back into sales. But then I started kind of coding as a hobby. I was using Dreamweaver… This is 2019. So this is bad. Like, I’m using Dreamweaver… I didn’t know any better, I didn’t know what people used to code… But it got the job done.

Roland Shoemaker: They hire people whose specifically job is to do like insurance billing…

Cory Doctorow: Well, we are segueing very smoothly here into the Internet Con, my last nonfiction book, which is a book about how platforms went sour, and what to do about it. And my thesis is that basically, there’s three things that discipline companies that might do something bad to you. And I think all companies might do something that to you, right?

[00:31:49.21] There’s always like a product manager who’s like “My bonus depends on figuring out how to extract a few more points out of this feature, or whatever; or hit some KPI. And there are lots of ways that I can do that. And some of them come at an expense to my users. And if I can, I might be tempted to do it”, especially if it’s like “Well, either you do that, or we’re gonna cut the headcount in your department”, and that guy who’s got a kid that was a preemie, and is in the [unintelligible 00:32:16.20] is gonna get fired and won’t have health insurance, and they’ll take his preemie baby out of the incubator, and… You know, it’s very easy to talk yourself into doing bad things.

So there’s three things that stop companies from hurting you. One is the fear that you might go to a competitor. And one prerequisite for that is there have to be competitors. And the competition has to be meaningful. So we talked about how Audible doesn’t really have meaningful competition… If you’re like an app developer, there’s no real meaningful competition between Apple and Android. Like, what is the commission they charge you for in-app payments in Apple? 30%. What’s the commission their dire competitor Android charges you? 30%. Like, if Burger King and McDonald’s charge the same price for a hamburger, are they really competing? They’re certainly not competing on price, right? So you need to have meaningful competition. And these firms have gobbled up all of their competitors.

So Google is a company that made a really kickass search engine a quarter of a century ago. And I don’t want to undersell it. It was magic, compared to like Ask Jeeves, and Yahoo, and AltaVista. It was amazing. And then nothing else. Everything they try in-house almost without exception crashes and burns.

Jerod Santo: It was at the World Herald, okay. There’s a group in Omaha that puts public data out, because you know, we own the data, because we’re the public, but sometimes it’s really hard to actually do anything with it… And so this was like to get citizen journalism kind of going, I think, was kind of the idea… And so they had these different datasets that they had released from government data, and they had this hackathon where you were supposed to come together and do something cool from this set of datasets that you have. And Zach and I took the crime dataset, and we decided to build a game where we’d put up the picture of what was stolen, and then it’s like The Price is Right, you have to guess the value. Because we had both the description of the items stolen, and then the perceived value or what they decided the value of that was, for insurance and whatnot. The funny thing is, we didn’t have any of the pictures, we just had the descriptions. And so we like literally did Google Image search, I’m Feeling Lucky, with the description, and we put that picture up, which was super-dangerous…

And we also decided to use Meteor.js. So you know this concept of innovation tokens, like “Don’t spend too many innovations on one project.” We hadn’t built a game before together; we hadn’t really coded much together. We’d never use Meteor.js. And we had two days to get it out there. And it was a disaster.

Adrian Hesketh: [13:55] And by that point, we had real users, right? Ourselves and our team. So we were using an early version of Templ to build HTML for PDFs, that produced insurance documents. So we had real users of it, and they were telling us things like “Hey, I really don’t like having to type two characters”, because the initial version was a curly brace, and then a percentage symbol, and then another percentage symbol and a curly brace to close it off. And they were like “This isn’t great, because it’s too much typing.” And I could totally – and that was kind of the syntax we inherited from quicktemplate.

So we started to sort of go “Okay, well, what would a second version look like?” And we did exactly what you were thinking there, of like “Well, what should it look like?” So you’ll see that a template looks like a function, because that’s what it is. It’s a function that returns a component. So we tried to make it sort of the most obvious thing you could think of.

Daniel Whitenack: Yeah, I definitely get the concern around this sort of cyclical thing, and where does a human actually pop in. Yeah, I think there’s a lot of systems that will operate at speeds as well, and with complexities, that it’s going to be hard for a human to debug these things anyway. It’s an interesting point.

One of the other things that they link in there is this - it’s a NIST AI risk management framework, AIRC. Part of me wonders as a practitioner - like, if I’m developing a new software product, or I’m offering software in an enterprise setting, there’s probably going to be an expectation on me that I go through some process to maintain GDPR, SOC-2, type 2, whatever the specific compliance, HIPAA compliance… Monitoring you can put in place, there’s third party audits, etc, etc. So part of me wonders, is this sort of AI framework kind of morphed into – maybe not this one specifically, but will there be a sort of risk management frameworks that filter into not necessarily policy… So I think we’ve been talking about like the White House and governments; there’s certain things that could be put into law. But also, I could very well see a scenario where one enterprise says to one of their vendors, “Oh, are you AIRC-compliant? And how do you prove to me that you are?” Well, maybe it’s a third party audit, maybe it’s a monitoring system, like it’s done with HIPAA, or other things… I wonder if we’re gonna get into some of that as well, where whether or not the policymakers make laws, I suspect we’re gonna get into some of these scenarios where we’ll have some compliance frameworks put into place, that certain enterprises start forcing on other providers… Because they’re accepting some level of liability for the type of AI reasoning that they’re integrating into their applications.

So if I’m an insurance company, and I’m hiring X vendor to provide some of my AI logic, or something like that, I’m making calls into their system… Do they have to be compliant in some way beyond the compliance structures that are already in place, like HIPAA and others?

Jerod Santo: I thought insurance is a good answer, but it’s also – it’s financial if you just break it down, right?

Steve Tuck: Insurance, and manufacturing, finance… Yeah.