How I lost my (old) job to AI

Well, hello, hello, hello. I am Johnny, your host, and I’ve brought the band back together. So we’re here, we’re here… Y’all are already cracking up. That’s how you know it’s going to be a good show. Nobody even knows who’s going to be on the show, but hey, you’re already laughing. Alright, that’s how you know it’s going to be a fun time. So y’all might recall, we did a show about how AI was going to take our jobs about six-seven months ago. A lot has happened in the industry in the world of AI since then. So I figured “Hey, let me bring back my good ol’ friends and buddies from wartime.” Can we call it wartime? No, we can’t call it wartime.

[unintelligible 00:05:00.12]

[unintelligible 00:05:01.27] …to talk about sort of what’s changed for them. Like how the new world order, seemingly being driven by AI and AI-related things is affecting their day to day, and if they have any new insights that they’ve sort of developed over these short six or seven months in the grand scheme of things, but a very long time in terms of AI development and news. So welcome… Starting with Sharon DiOrio. How are you doing, Sharon?

I’m doing good. Not much has changed in the seven months for me personally, but I just have lots of opinions to share about what’s changed in AI, so…

Nice. Nice. Steve… Mr. Steven Pyle, how are you?

Hey, doing okay. Yeah, not too much has changed for me personally, but my opinions on AI - I’m going to try not to make it one long unpopular opinion. So we’ll see how it goes.

I’m detecting a trend here. Mr. Kent Quirk, how are you?

I’m fine, thank you. And yeah, still same job, same house, but… Yeah, I mean, for me, I feel like I’ve been watching a slow-moving trainwreck for a long time, and so it’s still crunching.

Spicy. Still slow-mo crashing. Okay. Man… Okay, where do I even begin? So it seems like every other day I’m seeing tweets and social media posts… Wait, can we even call it tweets anymore? Anyways, that’s a different – yeah, we’ll talk about that later. But I’m seeing posts - that’s universal enough, right? …often from people who don’t even write software for a living, declare the death of software engineers. Like, “This new model, this new thing, Claude 17.9, cursor three point, whatever, blah, blah, blah… You don’t even need…” We’ve gone from “You don’t even need junior engineers” to now we’re saying we don’t even need senior software engineers. I’m like “Whoa, whoa, whoa. What is the Kool-Aid you are drinking, and can I have some of that reality distortion Kool-Aid, please?”

And they are sharing those opinions with no small degree of joy, glee even. Like “Finally, we’ll be released from the tyranny of software engineering.”

Does that mean like deep down, deep down we are not a welcome bunch? We are not welcome. If they could get rid of us, they would.

It kind of feels that way.

Wow. How do we explain this? Please, somebody make it make sense.

[00:07:55.12] I guess I’m going to put it this way. I think the best way I’ve come to characterize this personally is that AI gives you the average result. If you are below average, it’s amazing. If you’re not a software engineer, and you’re writing code, it’s doing things you can’t possibly do.

Right, right.

But the flipside of that is if you’re actually better than average, it’s probably pulling you down.

There’s a spicy take.

That is a spicy take. So you’re not seeing it as a productivity booster?

Well, I have to be careful about that, because I do use AI in my job literally every day. But asking what it is - it can be a very useful assistant. What it is not is a replacement for you or your brain. It doesn’t have a brain, and people who think it does aren’t understanding it. So you can ask it to write you an app, but the app you’re going to get out of it is a mediocre app, that nobody wants to pay for. If you want an app that actually monetizes and works, and pleases users, AI is not going to give that to you. And no matter of prompt crafting is going to give that to you unless you know how to craft a prompt that is the equivalent of software engineering.

You’ve just moved all the stipulation of your business logic into your prompt.

You’ve just changed the syntax of the code you’re writing.

Well, perhaps that is the goal. I mean, isn’t that the goal? Is it the – I hate to bring him back up again, since we brought him up during the first episode… But you have the CEO at NVIDIA telling people not to learn how to code, but instead learn how to write prompts, or learn how to write instructions instead.

Hey, I hate to say it, I’m old enough to remember COBOL. The first code I ever wrote was in COBOL. And the promise of COBOL was that business people were going to get to write code, instead of engineers. It was the 1960s. Hate to say it.

Right, right. Yeah, no code. No code when we didn’t have a fancy name for it, we didn’t have a trendy name for it. Yeah.

Declaration division, period. Come on. [laughter]

Oh, man. That’s like dragon naturally speaking for writing software.

Oh, don’t get me started.

Oh, man… Steve… You sound down, man. What’s going on? Has the AI got you down?

Oh, it’s rough out there, man, I’m telling you… So yeah, same as Kent, I still use Copilot daily for my job, doing prompts to generate some code, some really rough draft stuff that I then work with… But that’s pretty much it. I’ve canceled my ChatGPT Plus subscription, because I’m not getting anything useful out of it anymore. Not that I could get from some other source…

Like another model, or do you think that particular product is no longer living up to its hype, or…?

Well, for one thing, I’m getting a lot of wrong answers. Like, there was one time a few weeks ago I was trying to build some Terraform modules to build infrastructure. And I was having it – I was prompting it to get information from a series of AWS documents, right on the web… And it says, hey, it can go out and pull information from the web and generate on the fly. And it was giving me completely wrong answers. So if I had not known that it was giving me incorrect answers, I would have done something completely wrong with our production system.

[00:11:46.25] So maybe in the future it’ll be at a point where it will be more trustworthy, but that’s just the first point. The second point is - I don’t know, I get more out of doing the research, and the creativity, and the searching for the information that I need, either from books, or from documentation.

Oh, books… What are you, in the Dark Ages? [laughs] You’re looking things up in books?

Still, yes… I’m not old.

You probably still go to the library, don’t you? [laughter]

Absolutely. I’m very proud of it. Yeah, but it just – it feels like a shortcut now, that just… Ah, I feel like I’m cheating myself now when I use it, if that makes sense.

So is it a degradation in quality? Or is it now you’re coming to terms with the fact that it’s always been like wrong to some degree? Or what changed your mind from, say, I don’t know, four or five months ago?

I mean, part of it, I guess, is - yeah, the models have never been perfect; we know that. That’s part of it. The other part is the environmental aspect of it. You’ve got all these servers… It’s a reminder of cryptocurrency, where it’s using all…

Oh, we are [unintelligible 00:13:04.16]

Right, right.

That’s our favorite comparison.

Yeah, yeah. When it was crypto - oh my God, the environment. Now it’s AI. “Oh, it’s fine. It’s fine.”

Yeah, exactly. That’s what I’m talking about. I don’t know, I’m trying to think of how to best put it into words, but it just… I’d rather find information myself.

Is it possible you’ve just reached its practical limits for your applications?

Maybe.

Did you push it hard and see what you could do?

Yeah. I mean, when I would do some iterative prompts, it would give me some interesting information, but… I don’t know. I’d rather talk to another person instead. I’d get more creative information.

You know what, Steve? You remind me of vinyl. [laughter] People were moving on to like cassettes, and like CDs…

Hey, the audio purists go back to vinyl.

Exactly.

Because it does capture…

[laughs] Steve’s like “I want that original, smooth sound, baby.”

I’ve got a record player. I’ve got lots of vinyl. I’m telling you.

There you go.

The other day I saw on LinkedIn, OpenAI was saying “Hey, you can use it for your kids for their homework.” And I’m like “No! I’m not going to have ChatGPT help my kids with their homework. I’m going to help them with their homework. I’m going to have other people help them.” You know what I mean?

Right. So you still want a relationship with your kids. So helping them do homework… [laughter] That’s important to you. I get that. I get that. Right.

I feel like I’m ranting, but I just… Yeah, I don’t know.

That’s what we’re here for.

Yeah, let it out, man. Let it out. This is like therapy.

Okay, when I was younger, back in the day, back in the last century, I would always think about AI.

That was 1890…?

[laughs] Exactly. Yeah, and I had this idea of what AI could be. And I think that – you know the whole idea of the uncanny valley, where you have some visual which is like almost, but not quite like a real person…

But not quite. right.

Yeah. This feels like that to me now, where it’s just like, it’s the uncanny valley of what AI could be, if that makes sense. And it just feels –

It’s almost there, it looks like it’s almost there, but it’s not quite there, right? It’s very convincing at being fake…

It’s the animation in Polar Express.

Yeah, exactly.

Wow. You sound ruined by that experience. [laughter]

[00:16:07.06] No, I mean… Google – this is a combination of like the business thing and the practicality thing. It’s like, AI is desperately searching for the problem to solve; like the big problem. Because now AI is a trillion-dollar problem. It’s not a billion-dollar problem, it’s a trillion-dollar problem. Because that much money has been spent on training these models, and building all the hardware, and NVIDIA is pumping out chips like nobody’s business… And now it has to find a trillion-dollar problem to solve. And there isn’t one.

There is. There is. It’s just not in like every application out there. It’s in financial markets…

In what way?

Quant. The quant firms are using the hell out of AI.

Well, sure. But they’re not going to spend a trillion dollars on that problem.

They might.

They’re going to spend hundreds of millions, maybe even low billions. But the problem is that – like, OpenAI is out there, and it needs a billion dollars a quarter to operate… And what is justifying that spend? I mean, I for one think this is on the verge of the bubble popping. It may not be this week or next month, but I’ll be really surprised if we’re not here in a year going “Yeah, okay, well, OpenAI is done. Google’s in trouble.”

I mean, Google’s out there trying to say “Use an AI to write your emails”, and simultaneously telling its customers “Use an AI to summarize your emails.” So you’ve got somebody writing bullet points, generating text, feeding the text back into an AI to generate bullet points again? Like, just send freaking bullet points.

Yo, dawg.

Like, why do I want double AI miss-translations? I don’t know who’s going to pay for that. That’s the problem.

I mean, VCs are going to pay for that. As I’m talking, there’s – I don’t know, today, yesterday, the day before, one of the people who originally co-founded OpenAI has struck out on their own, and they have a company that is three months old, and they have raised a billion dollars from VCs. They’ve been in existence for three months. They have no product. They already have a billion dollars in investment.

Now, even the premise that the company - and for those who don’t know it, I’m talking about SSI, Safe AI Intelligence or something like that.

Yeah, let’s put safe in it. Yeah.

Right. Yeah, exactly. So I’m like, for me, there’s a disconnect here. I’m like, okay, the issue that was originally – what was originally a problem for said individual and others, when that whole OpenAI debacle, with the ousting of CEOs and everything else was happening, presumably the concern was around safety. That perhaps in the interest of actually creating a money-making venture, OpenAI had changed from its original thing… I mean, even Mr. Musk has sued over this, or whatever it is. Whatever those sort of conversations, whatever those speculations out there have been, is that OpenAI was – hence the term “open” and AI… It was was all about sort of building and making AI available to the entire world, and not really being driven by profit, money and things of that nature. But somewhere along the lines, when people started seeing the potential of the actual technology, they’re like “I don’t think we can leave that much money on the table.” Right? So things change, as money tends to do in those situations. The aims changed.

Now, the person or people who left were worried about the safety. That safety was no longer a primary concern. Now, they went and started their own thing. And that’s all well and good. That sounds honorable, admirable… Hey, I respect that. If the company’s going in a direction that you don’t like, you go make your own thing. Great. Love America. Love capitalism. That’s exactly what you’re supposed to do. Awesome.

[00:20:06.25] Now, keep that in mind. You’re worried about safety. You even create a company with safe in the name. Your primary objective is AI safety; presumably, not profit. Keep that in mind. Now, you have VCs who’ve put in a billion dollars. What do VCs want back?

Money. They want profit. They don’t care about safety…

[laughs] Oh, my God… I’m having a disconnect in my mind here. Like, how are you going to be like “No, we’re not about profit. We’re about safety.”

OpenAI has open in the name, for similar reasons.

Right.

[laughs] Oh, my goodness. What is this post-truth era? Truth is whatever somebody decides it is now. Like, this is the new world. The things that ought to make sense… I mean, I’m an engineer at heart. I’d like to think that if I see one plus one, I can say it’s two. But if somebody tells me it’s three, and they say “Well, that’s my truth”, I’m like “Well, I guess math doesn’t matter anymore…” It’s just whatever we feel like. I mean, this math is not mapping, right?

So a billion-dollar investment, company is three months old. They have no product yet. To me, this represents peak AI hype, peak bubble. Who knows, maybe I’ll be wrong. Maybe in six months the complete vision will come into being, and I’ll be like “Oh, okay, I was wrong.” But right now, I’m having a hard time reasoning through this.

Right. I mean, look at the amount of marketing that’s out there right now for AI products. Every single one that I see now, it’s AI-something. AI this, AI that. The amount of money that’s just being thrown at that…

And to your point, in six months… So AWS’s big conference, Reinvent; it’s in December. I see a lot of focus on AI. Will that still be the case? Will they be like “Oh, [unintelligible 00:22:25.17] everybody’s moved on to something else by then. Will that happen?

Um, I think we have – I mean, it’s only September. There’s a whole Q4 remaining… And yes, nobody works in the last two weeks of the year. That’s fine. Even counting that, there’s a whole Q4. So I don’t think the hype – given what I’m seeing now, I don’t think… There’s at least one or two more model releases to come from notable companies in the sector. There’s at least another round of hype to go. There’s at least a few more, quote-unquote influencers on the socials that are going to make videos saying “Hey, we don’t need any more senior engineers.” There’s at least a few more rounds of that to come.

It’s at least another podcast from us on it.

At least another podcast. [laughter]

So one of my co-workers this morning told me that – I think they said Google has just released a system, an AI-driven system that can theoretically take a research paper and turn it into a podcast that discusses that paper. And what I thought was actually extra hilarious is it only works on papers from Archive, which is being used by AI organizations to sort of research-wash their work as if it were an academic paper, except that there has no peer review. It just goes up on Archive and it looks like a research paper.

[00:24:07.27] [laughs]

So yeah, I mean, actually, AI is going to put us out of THIS job.

Right. Yeah, you don’t need any more commentary, you don’t need any more opinions. You know what would be wonderful? Like, you know what? If somebody came to me and said “Listen, there’s an opportunity to invest in a company; from research papers it generates podcasts. It’ll even generate the personalities. Do you want a funny personality? Do you want to an anti/downer, wet blanket kind of personality? You can have that in there, too. Do you want somebody who’s too excited about everything? It can literally generate all the personalities on the podcast, and do the recording, spit out the audio.” If somebody came to me and said, “Hey, I have such a thing…” Yeah. Where do I sign? [laughs]

It’s probably scraping us right now.

I just want to know which one I am… [laughter]

Listen, if they get my voice right, if they get my laugh right, I’m happy. I’ll invest. I’ll invest. Sure. Oh, my goodness…

Oh, my goodness… Alright, so all joking aside, like you, Kent, I use AI in my line of work every day. I still find value in sort of a conversational style of learning. So when I’m trying to explore a subject that I know very little about, and rather than searching the Web, or searching YouTube for videos, hopefully somebody has recorded something of worth, with accurate information or whatever it is, rather than sort of having to consume a bunch of videos or a bunch of blog posts or a bunch of websites, a bunch of documentation, if it’s a technology from a vendor or something like that, I will literally – my first step will be to open up my ChatGPT window, and ask about this particular topic. And at the very least – so I’m not expecting to get 100% accurate information, but at the very least I’m expecting to have some surface knowledge, shallow level knowledge of whatever it is that I’m trying to learn about. And then from there, if I want to go deeper, then then it’s like, okay, it’s on me to go read the docs, RTFM. It’s on me to go get to the specifics. To this day, I will pull up AWS SDK documentation and pull that, rather than asking ChatGPT or any other model to give me the cloud formation for something. I’ll literally pull up the documentation and read through it, because I know at that point, once I’m ready to actually execute and actually write the code that is going to go into production, I need to be damn sure of what I’m doing. I can’t leave it to a model which could be wrong. Especially if there’s money riding on a feature, if something could cost me hundreds or thousands of millions of dollars in my business I need to be certain. I will read the doc at that point. But I still need to get that surface-level sort of understanding, and I think AI is a good solution for that. What do you think about that?

So for teaching, basically, interactive teaching?

Yeah. And then, “Hey, teach me about this thing that I don’t know about. I need to be able to talk with my CEO or with the business team or the management… I need to talk to people about this thing. Catch me up.”

I would consider AI to be on par with a conversation with someone who is knowledgeable and you say “Okay, well, is it 100% accurate?” No, it’s not. A conversation with the average person, even an expert - can’t get a promise of 100% accuracy there either. The thing I don’t like is how it responds in an eerily overconfident way.

So you don’t like the way it responds to you.

[00:28:02.04] Right? Like, it doesn’t qualify its answers, it doesn’t – it’ll backtrack if you call it out. Like, you can say “No, that’s wrong.” And “Well, oh, I’m sorry” and then it’ll try again.

Like, “You’re right.” “I’m sorry…” [laughs]

But when I give an opinion, I’ve already qualified it. I’m like “Okay, here I am, this is what I this is how I understand this problem. Here is what I think the answer is. Here’s what I think.” No. AI is like “This is the answer.”

It doesn’t know what it doesn’t know, because it doesn’t know anything. It’s just giving you the next most probable word after the last one it said.

Right. And that’s the problem, right? Because if somebody says something confidently, you’re more likely to believe what they say, or take it as facts.

Well, it works in politics…

There you go. [laughter]

Oh, man.

You’ve proven my point.

It does work in politics, don’t it? Listen, we’re not about to branch into politics here. That would be a whole separate podcast. But yeah, so –

But it’s a personality thing, I think, that obviously the four of us have, but I think most software engineers in general have, that they will question everything. You tell a software engineer the sky is blue, they’re going to go outside and check it out. And I think that’s inherent to our need to understand things –

Most of us. Most of us.

Or they’ll write an app that asks you “Is that green or blue?” [laughter]

But I think that’s a requirement of the job, that you have to push on everything you’re told, because you don’t always get the full context just from one nugget of information. Whereas I think the average person out there doesn’t do that automatically. And I’m not throwing stones at anybody, it’s just not – they don’t need to in their job. They can talk to people and trust what they say, and kind of move on, and it’s fine. So we might be coming at it from a slightly different perspective than the average user.

So you’re thinking LLMs on the whole have more – okay, I’m going to stretch this. I’m going to connect some dots here. The reason why it is so popular, so hypeworthy, is for certain degrees of technical difficulty. For certain kinds of situations, software engineering being one of them, but I’m sure there’s other technical fields that basically… We’re software engineers, so we’re talking from that from that perspective, but I’m sure other people, other technical areas of engineering or science, whatever it is, perhaps they can levy their own complaints as well. But for the general public, for somebody who just wants to generate some nice words to put on a a birthday card, or something like that, or you want to generate a haiku based on your dog chasing a squirrel, or whatever. For these kinds of circumstances, this is wonderful technology. It generates things. It might even generate ideas and wonderful things of that nature. But given a sufficiently sophisticated problem to solve, we’re saying that – well, not we, but Sharon is saying that… [laughter]

I’ll own it.

…AI is not quite there yet. Not that – I mean, who knows? Maybe one day. I mean, I keep hearing GPT-5 is going to be the shyte. So who knows…?

I have no doubt that AI will improve. I’m not saying it won’t. I’m not I’m not even suggesting that what we have today is what we’re going to have forever. But some of these interpersonal little quirky things that AI does, and the big one that I notice is that it says things as they are as if it was like an absolutely known fact. It doesn’t doesn’t qualify it, it doesn’t give context, it just goes “Here’s an answer.” That’s a big problem, I think, for an average AI user.

Break: [00:32:13.15]

So Kent, are you using AI to write code?

Yes. So there is no doubt that in the last several months Copilot in particular has gotten better. Just yesterday I actually went back to some code to add a comment on the usage of this module I’d written, and said “This only recalculates when you call”, and the AI wrote the names of the two functions that I was about to put in that comment for me, having looked at my code, and it gave the right answer. And I was like “Dang, that was good.” That’s what I meant to write, and it was correct, and it did it by looking at my code in the context of the other stuff… And it understood it well enough to do that. That was genuinely useful and also tiny.

I have also found this week, where I went to try to ask it to do something in particular and it gave me garbage, and I deleted the garbage, and then I tried a much more detailed comment and said “Write me a function that does X…”

A better prompt. Yeah, yeah.

…and it did write, again, a different function, that was also no better, did not achieve what I was trying to do, and then I went and wrote the function myself, and yes, it tad-completed a lot of things once I got it started in the right direction.

Right, right.

So that stuff is great. When I’m writing tests - oh, my God, it saves me so much pain. Especially doing table-driven tests, and you say, “Okay, here’s the first two examples of the table I’m writing”, and then it just starts writing things that are like exactly the other cases I’d been meaning to test… I love that.

So I think this is all part of “What’s the training set?” The training set is all of our software together, that we have all written focused – so we’ve got a model that is built on the very thing we’re trying to create more of, and it’s trained on good code, including probably almost certainly private code, stolen code… [laughter]

Let’s call it different licenses.

Sam Altman said last week that OpenAI I couldn’t succeed if it weren’t doing copyright infringement. So okay –

That’s why they have lawyers and a budget.

Do we want that as a society? I don’t think so. But also, I’m getting benefit out of it right now. I don’t know, I’m really uncomfortable with the whole thing. But I also think that the combination of problem set and datasets don’t overlap for the general population in the same way that software engineering does. So software engineering, the tools created by software engineers are really good for training the tools created by software engineers.

Who knew…?!

But it’s not a trillion-dollar problem. It’s a hundred-million dollar problem, or whatever number it is. Billion-dollar problem, I guess. I don’t know. Multibillion. I’ll figure what the number is. But it’s not a trillion-dollar solution. And there isn’t anything else, in my view, anything else like it that is going to be equivalently solved. Nor do I think it’s anywhere close to AGI. I don’t think it’s anywhere close to solving the problems of the world, and I’m not at all fearful that we’re creating Skynet.

Famous last words… [laughter]

I’ll just say, it’s like, sure, I’ve got some stocks in some of this stuff, but I’m keeping an eye on it, and someday soon I’m going to decide that I don’t want my money in those anymore.

I have a theory. My theory - and the context is software engineering specifically. So when somebody says something outlandish as “Oh, we don’t need senior software engineers anymore, because Cursor or whatever it is can generate code” or whatever. I chuckle. I understand where they’re coming from, but I chuckle. The reason I chuckle is because I’m like “Well, you my friend have no idea what it is like to write sophisticated, complicated, long-lived software, that you have to maintain over years, decades even. Keep that stuff human-understandable, human-readable, still generating business value, still allowing somebody to do a checkout and pay money, and still connecting all these dots that need to be connected in order for software to [unintelligible 00:39:56.06]. You have no idea what’s involved in the process. So I chuckle, because you don’t know. And I understand that you don’t know, so that’s why you can say such outlandish things.”

[00:40:07.03] But at the same time, I have to check myself, because I’m like “Okay, perhaps I’m looking at this from the point of view of the way things have always been. If I were to change my perspective, to be more like this person, or these people, who don’t know what software writing and maintaining is like, to be like “Okay, what if what they’re demonstrating in their YouTube or X video clips and whatnot, what if the AI or the LLM, the Gen AI, whatever tool, whatever it is, whatever fancy name, whatever fancy algorithm we come up with in the future… What if whatever solution, whatever code it generated was not the concern?” What if I as an engineer didn’t have to actually maintain whatever code this thing spits out? What if its job was to generate the code, fix the code, maintain the code, deploy it, do all the things that a software engineer or SRE team or DevOps or whatever it is, all the things that we do - what if it was in charge of all that stuff? That way it could optimize the code however it saw fit, it could generate however it saw fit. Heck, it can generate binary, for all I care. I don’t have to read it. The reason why we write software and have programming languages is because we need a higher order level to be able to reason about things and communicate to each other.

Machines just need binary code. The reason we have programming languages is because you and I need to communicate to each other as humans around intent of what we are [unintelligible 00:41:32.22] the code to do. So what if the machine doesn’t need that level of sort of abstraction, of communication?

I think that once you get to the point where you’re generating all code from a prompt, the language that you have to use, even while it might be English, still has to get down to a level of detail that it becomes pseudocode. We’ve just created another coding language.

But if you’re not reading it, if you’re never reading it, do you care what the code is?

I don’t care… You know what? They can even use single-letter variables.

[laughs] Oh, the shade. Listen, Sharon, every time you come on the show, you throw a shade at Go. Listen, listen… [laughs]

I mean, at that point - you’re right, you don’t need that level of understanding of something, like to know what a variable name is, and et cetera. But if you’re talking about a complex application, the person writing the prompts for that does have to have a pretty detailed understanding of how the application works. And if not the exact variable names, they have to have their own understanding of how the information is reasoned about, and stored, and communicated. That I think is the part that I – it’s not even close.

So this is the spec problem. You ask a product manager to give you a specification for a product they want built, and then has ever in the history of the world there not been like a follow-up hundred sessions saying “What does this mean? What do you expect here? How is this supposed to work? Did you think about this?” All of those questions together inform the application that is actually built.

I mean, yes, you can ask OpenAI to “Build me a real estate sales application” and it’ll go and find one on the web and clone something about its behavior. But then you go “Oh, CRUD. it turns out that my friend named Cindy Null can’t log in, because her last name is Null.” [laughter]

Oh, my God.

Wow…

That is funny.

[00:44:08.18] It’s that kind of specification. I don’t think you solve the problem of underspecified behavior and are happy with it. I think you try to free people from having to specify all that stuff, and yes, write better prompts. And maybe there’s a little more induction from the prompts compared to what we used to have to do. But nonetheless, I don’t think you’re going to get an application that a) is good to start, and b) you can iterate on effectively. I don’t know how you iterate on something like that.

It’s not now. But what about in the future, as it improves?

I mean, I think, Johnny, what you described is ascribing intent to the A.I. that’s doing this maintaining of the software.

“Oh, it can do what it wants to do.” It doesn’t want anything. It only does what you tell it to do in a kind of probabilistic manner. “Oh, these words were used, therefore that implies this.” And “Okay, deploy this so it scales.” Oops, it didn’t scale. “You didn’t tell me what ‘scales’ means.”

I want to know what it does when you tell it there’s a bug.

Well, so here’s the thing… We’re kind of solving that problem too, because one of the recent developments - and I don’t know if it’s recent, but I’ve started hearing about it two months ago, or three months ago… It’s the idea of models checking models. Whether it’s the same model checking the model’s output, or a model checking its own output… But they were creating sort of this feedback loop of basically having the model analyze itself, or having another model analyze the output, to determine if there’s been any hallucination or if the answer is incorrect, or whatever it is. So maybe this creates this back and forth… You do a few rounds of back and forth and maybe you come out to the optimal solution, and no human was involved in the process. So yeah, you might not even need verification whatsoever. The system can work itself out.

I mean, remember that AI thing where you like feed the images back into itself, and you ask it to keep iterating, and suddenly you get an image with 30 million eyes? I mean that’s what happens in a feedback loop. Anybody who’s ever taken system dynamics knows how hard it is to stabilize a system with feedback. You have the problem of damping, you have the problem of underdamping, overdamping, feedback… This is why microphones squeal, why we’re all wearing headphones, or have noise cancellation on, because that is hard.

Right, right.

So excuse me, but…

No, no, trust me, as a podcast host, who’s had to get who’s had to get specially built cables, grounding cables, because there was a hum and a buzz on my microphone, despite it being an $800 microphone… It’s still buzzed, and I have to get a special cable. I get it. I get it. This stuff is hard. But the thing is some would have you believe these are bumps on the road to this future. This envisioned future where you don’t need humans to write code anymore. And believe me, I get it. I’m not so egotistical as to think that I’m irreplaceable as a knowledge worker. Every year that goes by, I find myself having to do fewer and fewer mundane things. Some of those things are being taken care of automatically by software, by automation, and these things… So there’s fewer and fewer – as a result, I can focus more on the things that require creativity, on the things that require true sit down, think through the problem, and create a solution that’s going to actually solve a problem kind of thing.

[00:48:15.10] And I think perhaps why we collectively as software engineers are perhaps miffed when people throw out exaggerated statements in today’s context, exaggerated statements like “Oh, you don’t need software engineers anymore. Heck, you don’t even need senior software engineers anymore”, the reason why we find it perhaps annoying to hear is that we’re like “Ah, a machine can’t replace my creativity. A machine can’t replace the context that I have, the nuances that I’m aware of outside of the code itself. The machine can only generate code and see the code it has, but it doesn’t know that I have to think about this brief encounter that I had with my project manager in the hallway, talking about a potential change in scope… Now that I’m writing some code to account for the coming scope; not the current scope, but to account for the coming scope that I know is going to show up next week… It doesn’t know things of that nature.”

So we see ourselves as having more knowledge, more context. Speaking of context and context windows - we see ourselves having more context than the machine does. But what happens when the world learns to live with AI, when the world learns to live with this new technology? No longer are we using shovels, but we’re now using [unintelligible 00:49:29.27] So basically, we know now that there are better tools, so we simply learn how to use those better tools. So I don’t want us engineers to sound as people who know how software is written, because - well, software writing may completely change in the next decade. Because now it’s being done differently, by different tools, by different systems, different processes. And we no longer need to be involved in the same way we were before. So now we too need to evolve. I just don’t know what that looks like yet.

Well, I mean, you could have said pretty much exactly the same words 30 years ago, when, you know, I didn’t have an entire open source environment; like, I had to write a physics engine from scratch for a game, because I couldn’t buy one. Okay, that’s cool. Now I can just – I start with Unity and I have the thing I need. And it’s like, that evolution is absolutely necessary. Yes, our tools change, the environment that the tools operate in changes… It’s why software needs to keep getting updated, because the environment around it changes, even if the software doesn’t. And all of that is true. AI is a component of that. I just think that the hype of “AI eliminates the need for intelligence because it’s going to replace it” is not true, because AI is not actually intelligent. It’s a faster, more efficient, more robust form of data lookup…

Autocomplete. [laughs]

But databases didn’t destroy writing software. Neither will AI, in my view.

You know, as a slight tangent, we still think we can do better than SQL. [laughter]

Is this where we bash NoSQL?

No, we’re not bashing NoSQL… But - well, let me let me link the two things together. So SQL has been around for many, many, many years. And one could argue it is the perfect language for what it does. Given a relational database – heck, I think I’ve even seen SQL-like dialects for non-relational databases. But given a relational database, on average, SQL is the right sort of language, the right way of thinking through data that you want to get out of a database system.

[00:52:09.11] So to me, us as software engineers, we understand SQL and we understand how to extract data from tables in a relational database. In the future, what if data is not stored? So this is the correlation I’m trying to draw here - what if data is not stored in a table, in a relational database, but it’s stored in a completely different way, so you need a different language, and now AI knows that language and can do a better job than you possibly could, because your thought processes and your tools and everything else, and even the systems you used to use to get at that data, these things are now obsolete, because there are new, better ways you can store all the world’s knowledge on a four by four chip on your fingertip? It’s a whole different world.

So the reason why I’m not worried about AI taking my job is that the way we build software would have to completely change in order to get rid of the people who know how to build software for that given era. You’d have to completely change the way you write software, the way software is built, the languages that are used for it… Everything about it would have to change in order for AI to have a complete takeover. So I’m not worried about somebody saying “Oh, yeah, AI is going to come take your job.” Well, there’s always going to be… AI is going to be generating – listen, if anything, I think there’s going to be a greater need for software engineers, because AI is going to be generating a bunch of slop, and creating projects that people are like “Oh yeah, I replaced my engineers”, and then once they put it into production, they’re like “Well, crap. I guess I need a software engineer to keep it going, keep it running.” Like, there’s going to be more demand, I think; I’m predicting there’s going to be more demand in the short-term… There’s going to be more demand for software engineers to come maintain things that were generated originally by AI, until we basically go over the hump where not only is the software being generated by AI, but it’s also being maintained by AI, and serviced by AI, and all these things… And no human need to be involved in the process. Once that can be accomplished, then yeah, sure, AI can take software engineering jobs.

I mean, you’re still going to need humans to tell the AI what to maintain, and how.

Yeah, prompt engineers, as we call them. Prompt engineers.

I don’t see that skill set as being less technical.

You need to express yourself clearly and logically. That is not a common human trait. And the people who have that ability are software engineers, or at least a large fraction of them. Another fraction of them are lawyers, who are just coding in an environment with a really underspecified programming language… [laughter]

You’re right back to it.

…and a buggy compiler, and a broken execution system. I mean… You wonder why legalese is so redundant. It’s because of that. They’re trying to code protectively against all the ways somebody might interpret this in the future. So you’ve got buggy hardware, buggy compilers, buggy interpreters… [laughs]

Yeah, yeah. But hey, you know what? They don’t need us anymore…

Yeah, but we get to the point where AI decides “You know what? We know better than the human. Let’s just completely block it out of the way. We’re going to do it all ourselves.” Yeah. And then what are we going to do? It might be a tangent, but… Is anybody concerned about bias?

Hell yeah.

Yes…

So bias in the model. So say you have more and more that we’re giving to the to AI to build our code. And we get to a point where we’re just doing prompts. If there is bias to a particular way, like a more inefficient way of doing something, or something that uses… Oh, here’s something malicious. It’s trained to use –

[00:56:19.24] You just have one of those on hand? I see you reaching down. Have you got something malicious within reach? [laughter]

No, no. Okay, so say there is some product which has like an enterprise license which is very expensive. What if the model requires that library? And so now by default you’re paying this entity, this company more money because the model has been biased to use that more often. And then when you’re doing a prompt, you’re just saying “Hey, I wanted to do action X”, or whatever like that, and you have no visibility of what it’s doing. It just decides, because it has this bias, that “Oh, okay, I’m always going to use this expensive product”, if that makes sense.

So we should trust like Google and AWS to decide what products we use? Is that what you’re saying? [laughter]

I mean, that’s a really interesting question, because Oracle’s pushing an AI, and wouldn’t it be interesting to know if Oracle’s code generator was by definition using Oracle databases with most of its –

I’m telling you right now, it is.

Wait, Oracle’s still around?

And the AI will be free, but all the little add-ons you pay for.

I’m just surprised Oracle’s still around as a business. [laughter] I don’t hear about them unless it’s like a lawsuit, or something. I mean, hey, no offense against Oracle. I mean, I’m sure they’re a great company and everything else. And I’m not throwing shade at them, or anything like that. But as far as my day to day, as a technologist, I don’t hear about Oracle very often. The only time I ever heard about Oracle is when I actually had to work with an Oracle database for an application, where the person who selected the database for the backend happened to be an Oracle DBA. And that’s what they knew, so that’s what they picked. Hey, I’m glad it’s still around as a company to still employ people. Hey, good on them.

But I do think the bias problem has been shown over and over again to be a real problem. Not just like who is it going to direct you to spend your money with, but also things racism and sexism are massive problems. There’ve been any number of trained visual databases that - you know, we’ve all seen all the headlines about asking an AI to show you a happy family and it’s a white family, and show us a sad family and it’s a black family. That stuff is hard to get out. And then now that we’re training AIs on the output of AI’s, it’s only going to get worse. That’s your feedback problem.

Break: [00:58:46.09]

So when you talk about bias, Steve, you’re worried about… So if I say “Hey, write the code, Gen AI. I write the code that turns on the faucet when I put my hand under it. When you detect a hand, turn on the faucet.” You’re saying that if we have biased AI and a person with enough pigmentation puts their hand under… If the training data only includes people without pigments in their skin, the moment I as a person who is heavily pigmented puts my hand under that thing, if it doesn’t turn on, that’s bias. Right?

Mm-hm. Right.

In other words, the AI is racist. Is that what you’re saying, Steve? [laughter]

The AI has encoded society’s racism.

There you go. Thank you, Kent.

Well, no, I mean… I don’t think you’re wrong. You’re not wrong.

This is why I don’t trust… And see, I’m an old man shouting at the cloud again. This is why I don’t trust driverless cars either.

Oh. You don’t live in San Francisco, my friend.

Right… I do not trust that the – yeah, I’m going off on a tangent. Sorry. It’s just, I feel like we’re playing a game of telephone. If you prompt the AI to do something, we’re adding this extra layer of abstraction that we have less and less control over. And there’s more and more…

You sound like overall you have your own perhaps bias against AI in general. And perhaps, again, because you happen to be a software engineer, you’re even more hypersensitive to sort of the implications of that than the layperson… And you might be one of those people who sort of swears off AI on principle, even if for a particular situation it might actually be helpful to you. Would that be fair?

I think that’s fair. I think that if we could have more control over the training of our particular model, then I would feel more comfortable about it. Like, the idea of having AI in a box, have your local copy, or you can train it specifically for your task. I would feel better about that.

But if you want control, then you don’t want AGI, because at some point when AGI comes along, that is by definition going to be releasing control of everything; the unit or the thing, the entity, whatever it is we create. That implies releasing control of whatever – because you’re not programming it to do a particular thing at a particular time, based on a particular stimulus. You’re saying “Make your own decisions based on what you ‘think’ about the situation.” So by definition, that means relinquishing control.

But I have to [unintelligible 01:05:14.26] that the end point of LLMs is not AGI. It is not.

Right. That is a thing on the road to AGI. So I’m told.

Yeah. I think we’re walking up a cul-de-sac on LLMs. I think we’re close to the end of it as it is now. In my view, LLMs will be a feature of a future AGI, should one ever be built. But an AGI will not come from LLMs. I’m going to make that statement categorically.

But do you think that most of the hype – I mean, I don’t see the hype around LLMs. I see the hype around AGI.

[01:05:54.08] Well, it’s because people who are hyping LLMs are saying that it is the next logical step; that LLMs lead to an AGI. And I’m saying it does not. I’m saying this is a dead end technology that is, if not already out of data, will soon be out of data, and the only thing they’ll be able to feed it is more data generated by itself. And that is not going to generate an AGI either. So in my view, we can do a bunch of stuff with this, but this is why I do not think that there is a trillion-dollar problem for AI in its current state, for LLMs to solve.

And I think this is why the term AI is so prevalent. Because it’s sexy. It’s got the science fiction kind of –

It’s whatever hasn’t been done yet.

Right. Exactly. People have this idea in their head of – and I think they conflate AGI versus LLMs subconsciously, and that’s getting us into trouble.

I think VCs – back to the VCs question. VCs are investing in AGI, but they’re getting LLMs. And when they figure it out, they’re going to be pissed.

Everybody feels sad for the VCs. [laughter]

This is one of those episodes that I’m having trouble speaking, because it’s making me think about sort of [unintelligible 01:07:10.07] and concepts here that we can really just dig into. So, yeah…

Well, we’re going to have to do that on a future episode, because we’re an hour in, and the hot takes keep on coming. So I hope you brought some unpopular opinions… Because that’s what we’re switching to next.

Jingle: [01:07:41.17]

Alright… Which one of you wants to go first?

You said that so enthusiastically…

I’ve got one. I’m going to say that microservices cause as many problems as they solve. [laughter] There you go.

So it’s an even trade then, no?

It’s an even trade. Monolith versus microservices.

Like, if you happen to be bored…

[unintelligible 01:08:28.01]

That would be fun… Let’s put let’s put a network hop between the different parts of my application. [laughter] Let’s introduce some latency…

Oh, boy…

I don’t even disagree, despite having once been a very large fan of microservices. I don’t even disagree anymore.

And I swear that all my unpopular opinions are not aimed at Kent. It just feels that way.

Alright… Hey, at least for you it’s an even trade. For some people it’s a net negative. You know?

Yeah. I’ve got another software development-related one, ish… Product managers.

Oh, boy…

So you have the P*M problem. Is it product, project or program manager? But I’m talking specifically about product managers. They come in two flavors, in my view. There are those who want to make good products for real customers, and there are those who want to be in charge. And there’s way too many of the ones who just want to be in charge, and they can kill a project.

So what are the symptoms of someone, a product manager who just wants to be in charge? How do you spot one?

They don’t have expertise. They deny the value of expertise, or refuse to do the research. It’s all surface level. So I think seeing how deep somebody can go is a really good indicator of whether a product manager is going to work out well or badly.

[01:10:07.01] Okay, we’ll see what the people have to say about that. Mr. Pyle. Pile it on.

Oh, boy… I’m not going to do my PHP one again. [unintelligible 01:10:14.11]

You’re still throwing shade at PHP?

PHP is good, actually…

What, you don’t want a Lambo?

[laughs] Yes, I want my Lambo. No, no, no. PhP is awesome. It’s great. Okay, here we go… I think you can build a really great application using ORMs, the object relation models.

That is an unpopular opinion.

Yeah, it is.

You will have to occasionally go back to SQL, and tweak your queries once in a while, but you can build a pretty high-powered application. Case in point is Laravel’s Eloquent framework.

Have you been doing some PHP lately, Steve?

Every day… [laughter] I’m sorry.

Would you say you are biased? [laughter]

You caught me.

[unintelligible 01:11:15.13]

Oh, my goodness.

Oh, man… I mean, I don’t disagree. I don’t disagree. When I’m throwing together a quick service to talk to a database, I’ll use an ORM, for simple cases. Yeah, that’s fine. Yeah. Yeah. I mean, I can see it, especially if you’re building with a framework, where the framework just has a native assumption that you’re going to be using its ORM, or something like that. I mean, yeah, that’s fair.

Yeah. It’s like the Go versus PHP conversation, where if you want the fastest thing possible, you can do it in Go; if you want something that will work… Okay, this is another digression here. [laughter]

Oh, this is a much spicier unpopular thinking coming…

Like, go on…

PHP will do just fine for an enterprise application, I’m telling you. Laravel all the way.

You can write an enterprise application in anything. Even COBOL…

It’s been proven.

PHP is the new hotness.

Oh, it’s made its turn around the Sun? It’s back again? It’s the new hotness?

I’m telling you, check it out. But I’m throwing shade on a Go podcast. I apologize. Go is awesome. Go is great.

[01:12:45.07] No, no, it’s fine. Yeah, yeah. We like it when people make fools of themselves on a podcast. Yeah, it’s fine. [laughter]

No shade against PHP. I did my time writing PHP code. But presumably, this was back when PHP wasn’t good, I’m told. This was like, I don’t know, 15, 16, 17 years ago. Apparently, PHP wasn’t good back then. It’s good now, so I’m told. I don’t know what modern PHP is like, but hey, if Steve, whom I respect, says PHP is the hotness, then hey, it’s the hotness. I mean, I don’t like things that burn, so I’m going to stick with Go.

If people want to throw stones up on the Gopher Slack, feel free to.

[laughs] Nice, nice, nice. Alright, my unpop, and then we wrap this up. So I did this unpop on last week’s episode, but I think it’s worth reiterating. I think in the current job market I’m seeing a lot of job posts; the same job posts coming up over and over and over and over and over again. I’m like “Hm…”

Feedback loop.

I’m like “Hmm…. I don’t think these things exist. I don’t think these things exist.” I think whoever is posting things, usually recruiting companies, I think they’re building and harvesting and putting together databases of candidates, just so they can say “Oh, we have a database of hundreds and maybe hundreds of thousands of candidates, potential employer who wants to hire a recruiting company to find your next talent.” I think that’s what’s happening. Because I’m just – I’m always keeping an eye out on the job market to see what’s available, and what skills are in demand these days… And I keep seeing the same job posts over and over and over again. I’m like “Hmm… This job, for this junior Go developer - surely you could have filled that up by now. It’s been six months, brah. Surely, surely there are enough junior developers on the market that you could have filled this job by now.” So I’m suspect.

Yeah. I don’t think that’s an unpopular opinion.

No. I mean, I’ve seen where one job will be posted, and then it’s like, it gets sprayed across the internet as all the board scraping happens… It gets reposted, re-scraped… And I believe there’s probably a feedback loop where the original board is probably re-scraping it… [laughter]

Jesus…

It seems like it anyway, because it’ll be the same job. You go in and read it, it’s the same job.

Yeah, yeah, yeah. Indeed. Indeed. Alright, so… Yeah, this was something. This was a fun one. Yeah, I’m sure I’ll be bringing the gang back together again in a few months, to get an update on how AI is changing or taking over our old jobs, or even –

Whether we still have NVIDIA stock…

At the very least – listen, if AI takes your job, at least have some NVIDIA stock, okay? You can get some solace out of the fact that “Well, I’ll get it down the road. I’ll get it back down the road.” You know what I mean? When you get some dividends, or something. But yeah, as usual, thank you, Sharon, Kent, Steve, for coming on the show and talking about the current hype. I think this might have to be a segment. Today’s it’s the AI hype. But we’ve seen a few hype cycles before, so maybe what we do is, as the new hypes come along…

Are we in the trough of disillusionment yet? [laughter]

Exactly, exactly. With the next hype I’ll have the team back and we’ll talk about whatever the current hype is. Yeah, we sure have seen a few of them. So with that, listener, thank you very much for your time and for listening. We’ll catch you on the next one.