Amal Hussein: Yeah, that definitely makes sense. Thank you for that great explanation. So I guess maybe taking it from the top, just to drive this message home - you know, Workers, still supported, they were the original thing… There’s a lot of evolution in terms of even just debugging support, and there’s a dashboard etc. Can you give folks some context around when you would even use a worker, especially in the context of a static site? It’s a little confusing and counter-intuitive, right? We’re mixing a lot of concerns here. How is this different than like an application server somewhere else?

Adam Stacoviak: I think just recapping really the how it works, from the site, really. Warp is a fully native GPU-accelerated, I think – you know, talking about GPU acceleration, why that’s important… Those are part of user experience; obviously, there’s a reason you chose Rust, but the reasoning for being 60 fps friendly on 4k screens, why no Electron - you’ve kind of answered that. Why web tech obviously being web-native… But the GPU accelerated and the 60 fps I think can use an explanation in terms of the user experience.

Kevin Ball: Excellent. I also have a five-year-old, so I’ve planned my explanation for yours in terms that I think he might understand.

Daniel Whitenack: Yeah, Chris, so coming from Arnold Schwarzenegger and large action heroes, to large action models… I was wondering if this was a term that Rabbit came up with. I think it has existed for some amount of time; I at least saw it at least as far as back as June of last year 2023, I saw Silvio Savarese’s article on Salesforce AI Research blog about “LAMs, from large language models to large action models.” I think the focus of that article was very much on the sort of agentic stuff that we talked about before, in terms of interacting with different systems, but in a very automated way. The term large action model as far as Rabbit refers to it, it’s this new architecture that they are saying that they’ve come up with - and I’m sure they have, because seems like the device works… We don’t know, I think, all of the details about it; at least I haven’t seen all of the details, or it’s sort of not transparent in the way that maybe a model release would be on Hugging Face, with code associated with it, and a long research paper… Maybe I’m missing that somewhere, or listeners can tell me if they’ve found it. I couldn’t find that.

They do have a research page though, which gives us a few clues as to what’s going on, and some explanation in kind of general terms. And what they’ve described is that their goal is to observe human interactions with a UI, and there seems to be some sort of multimodal model that is detecting what things are where in the UI… And they’re mapping that onto some kind of flexible, symbolic, synthesized representation of a program.

So the user is doing this thing - so I’m changing the payment on my Uber app, and that’s represented or synthesized behind the scenes in some sort of structured way, and kind of updated over time as it sees demonstrations, human demonstrations of this going on. And so the words that they – I’ll just kind of read this, so people, if they’re not looking at the article… They say “We designed the technical stack from the ground up, from the data collection platform to the new network architecture”, and here’s the sort of very dense, loaded wording that probably has a lot packed into it… They say “that utilizes both transformer-style attention, and graph-based message passing, combined with program synthesizers, that are demonstration and example-guided.” So that’s a lot in that statement, and of course, they mentioned a few, in more description, in other places. But it seems like my sort of interpretation of this is that the requested action comes in to the system, to the network architecture, and there’s a neural layer… So this is a neural symbolic model.

[00:36:01.18] So there’s a neural layer that somehow interprets that user action into a set of symbols, or representations that it’s learned about the UI; the Shopify UI, or the Uber UI, or whatever. And then they use some sort of symbolic logic processing of this sort of synthesized program to actually execute a series of actions within the app, and perform an action that it’s learned through demonstration.

So this is sort of what they mean, I think, when they’re talking about neurosymbolic. So there’s a neural network portion of this, kind of like when you put something into ChatGPT, or a transformer-based large language model, and you get something out. In the case of - we were talking about getting JSON structured out when we’re interacting with an external tool, but here it seems like you’re getting some sort of thing out, whatever that is - a set of symbols, or some sort of structured thing - that’s then passed through symbolic processing layers, that are essentially symbolic and rule-based ways to execute a learned program over this application. And by program here, I think they mean – they reference a couple of papers, and my best interpretation is that they mean not a computer program in the sense of Python code, but a logical program that represents an action, like “Here is the logical program to update the payment on the Uber app. You go here, and then you click this, and then you enter that, and then you blah, blah”, you do those things. Except here, those programs - so the synthesized programs are learned by looking at human intentions, and what they do in an application. And that’s how those programs are synthesized.

So that was a long – I don’t know how well that held together, but that was my best, at this point, without seeing anything else, from a single sort of blog post…

Carlisia Thompson: [08:07] Somebody gave a talk about that one at the Go Language team members, I’ve forgot who, and there was a good explanation. But yeah, it’s not easy; you don’t find this information right away when you start learning. Even now…

Chris Benson: So I was wondering that myself… A big topic that I spend my time at work is around multitenancy in your workflows, and the accessibility of compute in those… And I was taking it in that way. But I’m not sure, because they’re a little bit ambiguous in the way they use some of the terms.

Another one that I’ve noticed - they talked about the need for no code changes, and I’m assuming CUDA code changes in this case… But they’re not always as specific as they might have been in terms of their explanations here.

Douglas Raymond: Sure. So it's true that we do use models to build our knowledge graph, and there are some efforts going on there to increase the quality and coverage of it. In terms of what are the areas that we think are most exciting to us from a research standpoint, we're focused on this discovery experience, in terms of how do we help you identify what's new and relevant. We have several research efforts in terms of creating a personalized representation of what's new, of creating explanations or recommendations that are actionable in terms of how we explain to you why we've recommended particular papers. We have a number of other research areas there that I'd love to talk about, that I think are quite exciting.

Paul Hünermund: So there are many names - causal AI, causal machine learning… Causal Inference, I think, is the more traditional term. But I think the basic idea is pretty intuitive for everyone who works with data, is that if we look at correlations and patterns in the data, sometimes they can produce quite surprising, and probably nonsense results. I mean, we all know the story about ice cream sales and shark attacks that are highly correlated over the course of the year. Chocolate consumption and Nobel Prize winners in a country, probably driven by of Switzerland predominantly… Stocks and the babies - the stock population and fertility rates are correlated. With these examples, we usually use them in the classroom as sort of like a caveat, right? “Wait a minute, correlation is not causation.” People have heard this term. But then causal inference and causal machine learning is really this idea of taking causality seriously, and trying to build tools, algorithms that allow you to draw causal inference from data to distinguish cause and effect, and weed out the kind of nonsense correlations. And yeah, that comes with a different tool set. You can approach this from a purely algorithmic point of view, and you would probably apply different tools than standard machine learning… It goes even deeper. There’s a whole epistemological point about it. Well, if you want to do causal inference, you cannot do this in a purely model-free way; you actually need background knowledge, expert domain knowledge in order to do this, in order to, for example, distinguish between possible alternative explanations… And that is a whole paradigm shift in terms of how we approach data, and how we approach machine learning.

Well, and then my last word on this is what’s the difference to standard machine learning is that - well, standard machine learning, the bulk of it is really correlation-based. I mean, all the tools that we’re having - deep learning, support vector machines, and so on, these are predictive tools; prediction means correlation, finding/detecting patterns in data. And so they’re not suitable for it. I mean, there’s a branch of AI that is reinforcement learning - maybe we could talk about this later - that goes more into the direction of actually intervening yourself; the learner intervenes itself in the environment. So that goes in the right direction, but it’s not getting there the full way. This is the main difference to standard machine learning.

Wally Quevedo: Okay, so NATS is a high-performance messaging system, and you could use it as a transport to be able to communicate with clients that connect through it. It’s like a pure publish-subscribe messaging system, but on top of this basic pub-sub you could do also basic request-response for RPC type of use cases. That’s how I would describe NATS.

Actually, one of the really nice, concise explanations from NATS is from Ivan from the NATS team, describing that someone was complaining that a NATS server was sending duplicate messages. Ivan describes in the issue that basically it receives a message, and once it receives a message, it send it directly to any subscriber that has shown interest into something. It keeps it in a buffer for two seconds, and it’s going to be waiting for some client to be able to [unintelligible 00:11:25.20] these bytes in order to communicate by draining the socket. That is, in a very small explanation, basically what NATS is… To communicate to these many clients very fast.

Adam Stacoviak: I think you probably do it because you can, and because obviously you create content around the things you do in terms of exploration… So to me, it totally makes sense. Me, I don’t make money doing what you do, or maybe you don’t make – I don’t know.

Jerod Santo: So just on the homepage, VisiData.org, it shows off some of the things it can do. It’s a terminal tool, a visualization tool inside your terminal… And you talked about the duct tape analogy. Right now you have as the tagline “Data science without the drudgery.”

The way you present this is pretty interesting, I think. You mentioned that it’s duct tape to kind of glue stringing data in, getting something out. And you have this double-select. On the left-hand side when you have a blank, but you need a blank… and on the first one it’s the typical data formats that we all know and love/hate (in many cases) - CSV, JSON, HTML… There’s lots of formats supported. You can throw a SQLite database at it; if you’ve still got some Excel 97 files laying around, VisiData can handle that.

And then on the right-hand side you’ve got the outputs that are possible, which I think probably most of them, in terms of data exploration and really the kind of data sciency sleuthing that you’re talking about - probably they’ll use their terminal interface. But you can clean up text, you can output plain text, JSON, these kinds of things. Has this been a feature set that you’ve just built up over time? Did you have a first use case where it’s like “CSV in, and terminal interface out”? What was your initial concept of what this thing should do?

Adam Stacoviak: I want to share something, because you made me think about the history of this podcast. I met Wynn Netherland at the Ruby meetup in Houston. Houston RB, right? And it was funny, because he and I really were in the same place at the same time in terms of curiosity, and exploration of content, and what’s different, what’s new, because we were both in transition… And the early days of this podcast began because I met the original co-founder, Wynn, at a meetup. And we were friends. I met him and I liked him. He spoke well. It was instant friendship. It was too easy. And I didn’t really consider that until this moment, that the beginnings of this thing we’re doing right now began at a meetup.

Adam Stacoviak: It’s not, “Follow us.” But also, we don’t know what teams out there are facing. They may have ultra resilience problems. They need to have two CDNs, because they cannot stay in – I think there was some sort of tweet out there recently like one minute of downtime for Amazon is just an enormous amount of money, and why they spend so much effort into basically not making that happen.

Now, obviously, Amazon’s an extreme case, because they’re the largest distributor of products worldwide probably, but not everybody has that problem. We don’t know what teams out there have. So follow us in terms of learnings and exploration, not so much take our advice on having two CDNs. It may not be necessary for you.

Adam Stacoviak: You know, what’s important to mention here is that this may not be what every team should do. In many cases, this is exploration on our part. This is not so much what every team out there should do in terms of redundancy. We’re doing it in pursuit of 1) knowledge, and 2) content to share. So we may go force new ground on the listeners’ behalf, and hey, that’s why you’re listening to this show… And if you’re not subscribed, you should subscribe. But we’re doing this not so much because 1) our service is so important that it must be up at all times, it’s because the pursuit of uptime is kind of fun, and we’re doing it as content and knowledge. So that’s I think kind of cool, not so much that everyone should eke out every ounce of possible runtime.

Adam Stacoviak: You know, what’s important to mention here is that this may not be what every team should do. In many cases, this is exploration on our part. This is not so much what every team out there should do in terms of redundancy. We’re doing it in pursuit of 1) knowledge, and 2) content to share. So we may go force new ground on the listeners’ behalf, and hey, that’s why you’re listening to this show… And if you’re not subscribed, you should subscribe. But we’re doing this not so much because 1) our service is so important that it must be up at all times, it’s because the pursuit of uptime is kind of fun, and we’re doing it as content and knowledge. So that’s I think kind of cool, not so much that everyone should eke out every ounce of possible runtime.

Jerod Santo: Yeah. Well, let’s start right there. First, give us a definition or a general explanation of what they are, just for the people who are completely uninformed. Maybe they’ve just started programming, they listen to this show… Mainframe is probably a term they’ve heard, but have never even come across a real one, or the concept. Define it, and then we’ll talk about why it’s still such a big thing, even though they seem like they’re ancient, but it’s still a big thing. There’s still students signing up for new classes.

Kris Brandow: This is actually something that I’ve been thinking about recently… Because I saw a thread on Twitter where someone gave a really good explanation of what – I think it [unintelligible 00:27:30.18] continuous integration, but what extreme programming and pair programming and all the [unintelligible 00:27:36.00] what was the genesis of that, and how it was more about synchronous, team-based software engineering, versus what we’re more used to, which is the asynchronous, individual-based software engineering.

[27:52] And I think a lot of the time when we think about writing disciplines we think of that kind of asynchronous, individual-based – if you talk about novel writing, most novels are in that way. Screenplays, plays - most of those things are written that way. Sometimes you’ll have a couple of people… But as you pointed out, TV shows are absolutely not written that way. TV shows have a writers room, and everybody is in that room and everybody is working together… And those are two extremely different methods of writing, and you can’t really take the techniques of, say, the individual way you write a novel and just pull them over to a writers room. That won’t work. And that’s sort of the situation that we’re in right now as an industry. Everybody was excited and talking about this writers’ room style of doing software engineering, and we just started plucking the tiny little things that we saw were like “Oh, that sounds nice. That looks nice.” Like, “Oh, yeah, we can sort of do pair programming. We can sort of do test-driven development. We can sort of do continuous integration.”

I think continuous integration is a really interesting point here too, because it’s like, we’ve taken the term continuous integration and left what it means. It’s like, continually integrating. That means you’re literally continually putting all of the code together, which implies you don’t have gates where you have to stop and check. It’s like, the code goes in, and you can check it afterward… Which, ironically, is like some of the ways that novels are written - you put it into form and then you send it to the editor. You don’t say “Oh, can I add this paragraph here?” before the editor allows you to put it in. But we’ve taken these terms and then we’ve just tried to put them into a different domain, and I think what would help here is if we recognized that and separated them, so we understand “Okay, these are the techniques you use if you’re gonna do an asynchronous, individual-focused software development process, and these are the tools that you use for a synchronous team-based development process”, and have a good understanding of that, and not try to co-mingle the two.

Johnny Boursiquot: I’d like to perhaps briefly touch on the fact that you consider the term “idiomatic Go” to be exclusionary, in some way… I kind of take a slightly different view of it, in the sense that I think it’s an aspirational goal almost. So when you join a community, there is going to be an idiomatic way of writing, be it Go, or Rust, or Ruby, or whatever. You want to fit in, in a way, and to write the programs like most others are doing… At least the correct way that most others are doing it.

So to me, it’s an aspirational, less so an exclusionary term… But I can definitely see where you’re coming from when you say that. To me, when I juxtapose the Go proverbs and the Zen of Go, I see the Zen of Go as something you understand and take to heart on your way to the Go proverbs. Once you reach the Go proverbs, you’re like “Oh, okay. Now so much starts to make sense”, because you’ve been working with Go for a while, you understand the layered meaning in the proverbs. The Zen of Go is more explicit, it’s deliberately an explanation of how you should approach your Go work.

So I see writing idiomatic Go as aspirational, something that everybody should be striving to do… Not necessarily shutting the door on folks who don’t quite know how to do it yet, whether they know if they’re doing it right.

Janis Klaise: APIs may be a bit of a grandiose term. It literally refers to the way we’ve structured these various kinds of explanation algorithms within the library. You could think of it in terms of the scikit-learn API, which we all know, which typically has two steps for any given model or estimator. You have the fit step, which takes in a training set, and then you have the prediction step, so you can make predictions on new instances.

With the model explanations it’s a reasonably similar process, actually… Some explanation methods require access to the training set to be useful, so you would have a similar fit step; not always, but in some methods. And then really at explanation time, explanation almost exactly maps to the predict step, so you would have rather model.predict, you would have an explainer.explain and pass in a single instance that you want to be explained why the model made the prediction that it did.

Carlana Johnson: Yeah, that’s a good question. So I’m not somebody who understands compiler internals really well, but the explanation I read was that the way that the language grammar is right now, it accepts either zero, one or two values in front of the word “range”. So [unintelligible 00:18:33.12] So in terms of the way that the grammar already worked, it only accepted either zero, one or two values in front of range. And so then they were like “Um, that’s enough. We don’t want to extend it so that you could do three or four or a thousand, or whatever. We’ll just say that it’s enough just to let you do zero, one or two.” And then if you do need something where it’s like “Oh, I need to unpack five values” or something like that, then just return like a closure, and then inside the loop say value A, B, C, D, E are equal to, calling the closure, or whatever it is.

And yeah, that’s one of those ones where it’s a little bit inelegant too, that – so because of this, if you want to accept a sequence, there’s different kinds of sequences you can accept. You can accept a sequence that just has one value, or you can accept a sequence that’s a pair of values.

Jerod Santo: Yeah. I mean, for me, it seems like some of our most ingrained premonitions coming to fruition in terms of being down in the mucky muck, as a developer, we just know – and I’ve said many times, it just feels like we’re building a house of cards… Because it’s so complicated, and it’s so intertwined. And it’s effectively - especially with web development, we’re talking about a worldwide distributed system, which has things that happen… I mean, of course, there’s an explanation in retrospect for all of these things, but when you build a house of cards, eventually it’s going to just topple. And sometimes it topples in ways that you don’t know why, or when, or how, and what will be the downstream effects… And of course, this isn’t web development in this case; this is operating system code. But still, networked machines being able to remotely update. Every once in a while, the house of cards topples, and we have to start over to a certain extent, rethink things, try to adjust, and clean up the mess, and move forward.

Johnny Boursiquot: The size matters in in-person workshops differently than sizing in remote workshops, especially where the remote workshop either has video, when you can see the people you’re teaching, or it doesn’t. And for better or for worse, I’ve had experience with all those different modalities. Let’s pick on the remote one, because that’s its own special brand of challenging… Because like Kris is saying, if – so let me just use a concrete example. A few weeks ago, two weeks ago, I taught a live class on O’Reilly, and the platform that they use, basically you can’t see any individual. So there’s some compliance thing that they do… I imagine that in some certain enterprises, part of the compliance thing is that you can’t use the chat, can’t have video on… Things that I find odd. Some [unintelligible 00:24:17.08] I’m like “What, you can’t have video on?” When people have meetings remotely, they all have their cameras off… It was like odd to me, but for some folks that is their reality.

So if you have basically a virtual room filled with hundreds of people, like I did - I had like 150 something odd people in my class… And some of them could even – those that could use the chat, that was the only way of communicating with them. I couldn’t even see their full name. It was like abbreviations. It was like first initial, last initial, something. So I couldn’t even address, say, a Ryan, or a Dave. I didn’t have any of that. It was just letters. So it makes it especially challenging in those situations, where you can’t see anybody, you can’t pick up on sort of the facial expressions to know that “Is this person getting what I’m saying?” Or “Does this person look like they could use some additional explanation?” Things like that. Things you can easily do, or rather more easily do, like a live workshop situation. But I didn’t have any of that.

So to me, in terms of the quality of the experience, or rather the challenge level for teaching, if I can’t see or hear people - I’ll still do as good of a job as I can possibly do, and depending on what I see in the chat, if people say “Hey, can you explain this a little more?” or whatever it is, then I’ll kind of know where to focus my attention a little bit. But that is by far the worst kind of experience for me. If I can’t see anybody, hear anybody, it’s just – I can’t even tell how many people in the room… It’s hard.

[26:01] But then you can have remote that is better, far better, because if you have video, you can see everybody, you can see smiles… When you make a joke, you imagine people on the other end smiling, you can actually see them smiling back at you on the the video call. People can unmute, talk, and ask questions, and everything else. So it’s a lot better experience for them and for me… But ultimately, the best – if you can get it, the best possible, highest fidelity experience that you’re gonna have is the face to face. It’s when you’re in the classroom.

And I would say even if I had four people in a classroom, we’re still gonna have a good time. Past 40-50, then I’d probably want to have TAs. At this point, I can manage a room of 50 people, depending on their level. If more people need more hands-on attention, then it’s hard to sort of bounce around the room a little bit. But the face to face experience - I mean, there’s nothing quite like it. I mean, it’s better for the students, it’s better for me, because I can interact, there’s body language, I can run around the room, I can do things – like, Bill Kennedy, if you’ve ever been in Bill’s classroom, he’s all over the place. He’s bouncing off the walls… I admire the way he teaches; I can sort of pick up some hints from him when he’s like keeping everybody engaged through his movement. I steal some of those ideas. I’m like “Hey, you know what? Let me go to the back of the room, and pull a Bill Kennedy. Let me go to the back of the room and start talking to this individual, start asking that question. So let me pace the room a little bit.” So things like that, I pick up from other instructors. What’s the best way I’ve seen somebody deliver this particular kind of topic? These things you get to experiment with when you’re in a live, face to face classroom, that you just can’t do on a remote call. So the sizing there is definitely gonna matter… But yeah, it’s tough. There’s no formula for it, but I’d say past a certain size you want some help, you want some TAs.

Jerod Santo: HashiCorp strikes back

On April 3rd, Matt Asay published a piece for InfoWorld titled OpenTofu may be showing us the wrong way to fork. In it, he says:

that OpenTofu may have illegally taken HashiCorp’s code to keep pace. At least, it’s hard to avoid that conclusion, perusing OpenTofu’s GitHub repositories and comparing them to HashiCorp’s.

The code in question is a new feature in Terraform 1.7 that also landed in OpenTofu as the fork maintainers work to maintain parity. Asay claimed:

OpenTofu took this BUSL-licensed HashiCorp code, removed the headers, and tried to instead relicense it under the Mozilla Public License (MPL 2.0).

As a beleaguered boxer might say, “Them’s is fighting words.” So, it naturally prompted many armchair software copyright lawyers to analyze the code in question and determine whether or not it was actually copy pasta. Smart people have landed on either side of this issue. Dan Lorenc from ChainGuard says:

I did my own audit and the samples bear no resemblance despite implementing similar functionality, which is honestly hard to do in Go where there are so few ways to do things. I can’t possibly see any validity to this claim.

Meanwhile, Joe Duffy from Pulumi concluded:

There are three major kinds of taint, from worst to least worse

1. Copied the code directly
2. Read the source and was influenced by it
3. Copied the functionality

I’ve seen this game enough to know that if the file, function, and variable names, plus non-zero number of statements, match, you’ve probably got at least level 2 (if not level 1). And that’s a problem. That’s pretty clearly true of at least remove_statement.go.

Asay later issued this statement about his article:

I regret how strongly i expressed myself (force of habit ),but grateful for those who expressed support against mob dog piling. Two are execs my post put under fire. They didn’t agree w/ my conclusions but responded w/ kindness. They’re the kind of OSS community I want to join

A few days later, OpenTofu posted this on their LinkedIn page:

OpenTofu Project was recently made aware of a letter by HashiCorp’s lawyers, alleging that OpenTofu was not respecting the terms of its BSL license governing its Terraform codebase. OpenTofu vehemently disagrees with any suggestion that it misappropriated, mis-sourced, or otherwise misused HashiCorp’s BSL code. Indeed, it seems that HashiCorp may be conflating code that it had previously been open-sourced under the MPL and more recently developed code it published under the BSL. OpenTofu’s maintainers have investigated this matter, and intends to issue a written response providing a more detailed explanation of its position in the coming days.

I’m excited to read OpenTofu’s written response, but I have a feeling it’s only going to get uglier from here. If I’m running the OpenTofu project, I’d be seriously considering a change in strategy from “feature parity” to “differentiation” from now on…

Miriah Peterson: I’m gonna steal that explanation now. I’ve always explained garbage collection as only that sweep phase… I always say, it’s the marking for garbage collection which happens in parallel, and then garbage collection. I’m now going to change the terms, whether that’s more or less confusing, I don’t know, but it is more correct. And that’s what matters. So thank you, Bryan, for teaching me today.

Jack Shanahan: I think so strongly about this idea of human machine teaming or human systems integration, that if we don’t get this right, we’re in trouble. And what I mean by that is, there are people have been writing about human-machine teaming in some form for 50 years. No different in terms of the concept of “How do you get the most out of human machines?” But to me, we’re at a period where it is changing. And it’s changing because the machines are going to be so much smarter than we’re used to talking about, that it will be a different view of human machine teaming than we’re used to. So I think there’s a whole new research field. There are already people working on this, and I’m amazed at some of the work that’s being done. But we just don’t know how good some of these technologies are going to be.

Let’s just take the example - very brief, the example of sort of ChatGPT, or the equivalent. There’s this idea of prompt engineering. I look at that as almost like the elevator operator in the ’30s or ‘40s, whenever it was, and it goes away eventually, because people get comfortable with it. But they’re not as comfortable with it today. You need help figuring out what that interface looks like.

But the other thing I’d say - and this is where I come back to - I really do believe there will be situations… I call this the bell curve of military operations. Some cases, you really do want that human making that final decision. Maybe to launch nuclear weapons as the President of the United States, and nobody but the President of the United States. On the other end, you need the machine to do what the machine does best. There is no time to get in the way. Everything else in between - I don’t know what that is. 80%, 90% is human machine optimized for both. That Centaur idea. And that we need a lot more work working through. And here’s the example I would give from my time in the flying world… There were a lot of times the machine did not operate as intended. The human was there to make sure that everything went fine with the mission. “Oh, that broke again. We’re going to have to pull that circuit breaker…” I started in the F4. Circuit breakers popped all the time. [unintelligible 00:37:12.27] hydraulic fluid out the backend, whatever. So the human would have to make up for a lot of the machine’s mistakes. I’m not sure that’s going to be a luxury in the future. You’re going to have to let the machine do what the machine does, and the human do what the machine does very well. What do humans do well? They do reasoning. They do inductive reasoning. They do deductive reasoning, they do inductive reasoning. They put things in context, they deal with other human emotions. Machines don’t deal with emotions, they don’t understand emotions, and I’m not sure, despite what some people claim, they ever will.

So what does that look like? I think there’s a lot that has to be done in this area in experimentation. Play around with it, see what works. Because if you don’t get it right - and there’s a couple of examples where it’s went dramatically badly, and I think the 737 max is one of those examples. “Well, we’re gonna put this software in. Trust us, you don’t need to be retrained as a pilot on this. Just listen to what it says in the cockpit.” That was wrong, and it has proven that it was dramatically the wrong thing to do.

[00:38:11.12] So I think this is an area that needs a lot more explanation. There are people writing about this, and really good writing, in various places… But we have to figure out how to get this part right. Because the machines are going to be smart enough that they have to be allowed to sort of run when they should run.

Ryan Metz: What always stands out is when somebody – the Jupyter Notebooks, which [unintelligible 00:33:18.20] had put together, I think they’e really well designed as learning activities, and to like get something done at school just by going through them. And so people use those to do a lot of amazing work. And I was stunned by the quality.

But what always stands out to me is when somebody takes the concept, takes what’s in there, and then runs with it. And we saw people setting up AI agents for some of these challenges, like the comic book generator… And the fifth challenge, where code explanation comes in, it’s like “Oh, explainability - listen, we’re going to do things like we’re going to do the explanation, and the model will then cite the sources it’s using.” Which made me think - you know, in years gone by, people were very concerned with explainable AI, and what they always meant is like “Well, if the model is making a recommendation, or classifying something in such a way, we should be able to figure out exactly why.” And so there were all these discussions of how best to do that, like “Oh, you can use [unintelligible 00:34:19.17] values”, whatever.

And I think what it turns out is like, well, now that we have Gen AI, and we have retrieval-based methods, it’s like, “Just ask.” Okay, so this is your explanation. Where did that come from? And we see the setting of sources.

So that creativity, not just in application, which was astounding, but then also in people bringing in methods, cutting edge methods from outside of like what we even included in the notebooks - that always blew me away. And there were some people that just always ended up in the top five. [unintelligible 00:34:52.29] I don’t know if I’m pronouncing that correctly, but I actually reached out to him, because it was like “What do you do for a living? Your work is incredible.” Because there are so many. Simon’s team…

Daniel Whitenack: So Prashanth, you kind of alluded to this, and I think that explanation was amazing, of how this vector-based semantic search really exploded around the time that transformers and large language models did. I think even in this past, let’s say year, there’s been this huge explosion of interest in vector databases. Could you maybe describe a little bit – so we know that you can search a vector database to find similar statements, let’s say, or similar chunks of text, where the similarity is based on semantics… How are people using them with regards to their AI workflows, and how does that kind of correspond to what’s sort of popular right now in terms of what people are exploring with AI?

Bryan Boreham: Methods are slightly worse with generics than they were before methods on interfaces… So it’s a little bit disappointing. So I guess an example which is a little bit like what you’re talking about is the heap; not the heap that you allocate memory on, but the one which sort of keeps the smallest element at the front. So that’s expressed in the Go standard library as an interface that has a less method, but also has a swap method and a pop, or a push; I forget exactly what it has. Anyway, I noticed that there wasn’t one of those in the slices package, and around about Christmas time I was off work, and I thought, “Oh, I should be able to fix that in a couple of hours…” And this ballooned into something that took weeks, because it’s just not that nice to try and express a thing which is a container, and has operations on the objects being contained in terms of Go generics.

I’ve got a talk at GopherCon in San Diego coming up, where I sort of explain where all this landed, which is with a completely different data structure called a loser tree. So I won’t go into the whole explanation of that right now, but anyways… The short version is that it leaves a lot to be desired right now, trying to express a kind of a generic thing which operates as a container on other things.

Chris Benson: [00:37:46.27] I appreciate the explanation. It’s a very robust-sounding pipeline that you have on that. Let me ask you… And this could be whether it’s Prediction Guard, or whether it’s the larger field. One of the challenges – it’s certainly something I’ve been playing with, but I don’t have a good rhyme or reason to it yet… With the proliferation of these models coming out, and evermore coming; we know this space is going to get larger and larger. How does a user, or how would a system like Prediction Guard be able to determine which is the right way to go in terms of which model you want to choose, or which group of models? And you talked about the comparisons a moment ago… How do you structure the input and know that you’re gonna get what you need from an output by putting the right model or collection of models together, and then knowing how to evaluate them against each other? Does that make sense?