Understanding what's possible, doable & scalable

Welcome to another episode of the Practical AI Podcast. This is Daniel Whitenack, I am founder and CEO at Prediction Guard, and I’m really privileged today to have in the Prediction Guard offices one of my good friends, Mike Lewis, who I’ve known for a little while now in the AI space, and I think have been really intrigued to see how Mike as an architect has been able to solution certain AI tools with his customers, with his partners… He’s currently the chief AI architect at Synthony down in Cincinnati… And yeah, it’s great to have you here, Mike.

Thank you, Daniel, and I’ve been looking forward to this for a long time. I’m obviously a huge fan of the show, and have learned so much by listening to you and Chris… But I just am so excited to dig in here.

Well, one of the things that I think is really intriguing is that I’ve heard a lot recently that there’s some disappointment a lot of times, and people trying to find value with AI, especially in enterprise corporate environments. I’ve heard things like “The tools aren’t there yet.” “We have yet to kind of fully realize the value of AI”, and there’s some disillusionment maybe. But I’ve known you for a while now, a couple of years at least… And even before we had GPT-4, GPT-3, you were creating and architecting solutions for people that actually led to value, and eventually led to your business getting acquired by a company that’s engaged with Fortune 50 companies. And you continue to prove that you can find solutions with this latest wave of AI technology… And so there’s a disconnect for me here. From your perspective, as that sort of architect, people are talking about this disillusionment, but you’ve also been finding these solutions with your partners… So yeah, what’s your perspective on that? Are we not there yet? What is the state of these tools? …that sort of thing.

Frankly, I’ve done a lot of head-scratching on this topic… There are astronomers and there are astronauts. And the astronomers get out their telescopes, and they look at the stars, and they tell us about all these things, and the astronauts get up there and work on satellites. And I think you and I both have been up in orbit for a while now. I tend to not pay too much attention to all the people who tell me the things these tools can’t do. Most of the time I’ll go to a conference, or I wind up in an environment where there’s a lot of talk about the tools… And I’m just astonished at the use cases where I will hear that there’s no value currently, and we’re waiting on that. And then at the same time, it feels like the same person will tell me all the things it can do, and I’m scratching my head like “Well, how’d you get it to do that?”

And so the reality is, it’s just – I mean, there’s a whole universe of problems and solutions, and we were talking about it earlier, that to be involved with even just generative AI, not even counting all the stuff going back a decade, or a decade and a half, or since the fifties, I suppose… But even just the recent developments - golly, I mean, I don’t think any human could really understand it all. And so we all tend to carve out a niche… I know you have a phenomenal niche carved out, and mine tends to be, you know, get to know the client, find the low-hanging fruit, prove out value, and then start making decisions about “Is now the time to put something in production?”

That said, I understand. I mean, if you hadn’t been tinkering and you haven’t built 150 tools already, then it could feel like they don’t do a lot… Because frankly, I worked really hard to figure out how to solve all those problems, right? And it’s not like I’m running around, telling everyone how I solved them all. I hope to do that today, a little bit of that… It can’t do it until it can. And sometimes it’s not that the tool can’t, it’s just that the user hasn’t been creative enough yet.

[00:08:11.04] Yeah. And maybe this might be good context for certain people, because even before the show, I think it was interesting to talk to you about perceptions in this space… There’s sort of a perception maybe by larger organizations or teams that don’t have kind of data science, AI expertise, that we sort of don’t have the know-how or the expertise or the skills to be able to adopt this technology in a meaningful way. And I think it’s interesting that your background - although you have some technical background, it’s not like decades and decades as an AI researcher at Meta, or something like that. Can you describe a little bit of your background, and kind of how that’s influenced maybe how you’ve come into this space, and how you’ve come into this technology? I think that might be relevant for people.

Yeah. As quick as I can, I’ll say - initially, I aspired to be an industrial designer. And so even in high school, when I was 17 years old, I worked as an industrial designer. And that might not seem possible, but the reality is I got an internship at a medical device company, and I went from like printer paper/coffee boy to designer within like six months, because I knew AutoCAD, and the company was hurting for like warm bodies to like draw up, sort of…And I had this amazing mentor who taught me problem-solving, from just – he just had a great way of explaining how to tackle problems.

And so that was the framework that I kind of initially formed professional aspirations. I went off to college to become an industrial designer, and I got an internship at NASA… Aerospace was where I wanted to go. And so that was all highly technical. And then my career took this massive turn. We won’t go too deep into it, but essentially, I was presented with the opportunity to be a full-time professional portrait artist, and I just got really lucky in that my art was taking off.

I spent the next 20 years essentially just traveling all around the world, painting portraits, all commissioned works… I did about 2000 of those. I stepped away from that in 2016. We just completed a successorship, and I really didn’t know what I was going to do next… And I decided to just work with a couple of nonprofits, just to keep myself busy. One was a homeless shelter, the biggest one in Cincinnati, and then another one was a publishing company that operated a homeless shelter. So I was just kind of – I just kind of wanted to do that.

And the publishing company ended up being bigger than I knew. They sort of had a global presence. Aside from several book lines, they had a magazine that had a similar subscriber base to like Newsweek or Sports Illustrated. So that’s how many tables it was on; sort of a niche publication.

I wound up being asked to be the director of innovation for them, which - suddenly, I’m the director of innovation for a publishing company. You can kind of start to see some overlap. I took that job probably in 2016 or 2017. Yeah. And and it really hit around ’19, I think with the AI stuff, when we started reading about “Oh, these tools are coming for us.”

It might’ve been ‘20, around COVID I think is when some of this stuff started to – you know, the language models were surfacing. Not only was that going on, so I had that incentive… That alone would have been enough to really familiarize myself with the tool. At the same time, I’m hearing that there’s a tool out there that you just type in words and it can spit out images. And that’s interesting to me, because I sell images. In fact, we charge design images, and we charge a lot. In fact, I’m uncomfortable with how much we have to charge. I feel like it’s a deal-breaker in a lot of scenarios, because I know it takes us a month to land on. So I just wondered, is this something that we could leverage as an efficiency in our fine arts business?

[00:12:13.26] So Open AI at the time was – you could apply to become a beta tester just to get early access to DALL-E… And don’t hold me to any o this; the way I remember it, there were three criteria. You needed to own a commercial arts business, you needed to have a large online following, which we did, and you need to have plans to commercialize. And I could demonstrate all that. I think I was approved within a week or so, to get early access to DALL-E… Which wasn’t super-impressive. Actually, I kind of remember being a little disappointed. That got me in sort of their ecosystem really early. So whenever the API hit, I was in there. I already kind of knew how to code a little bit… Python pretty much looks like plain English. So here we are.

Awesome. Yeah. So I guess as you were kind of first starting to develop those solutions, first kind of your interest from the fine arts side, but then as you started to get into architecting these solutions with smaller businesses, and then medium businesses, and larger businesses, what have you found? Because I often – well, we sell infrastructure for this sort of thing at Prediction Guard, and so oftentimes we hear people that they’ve found maybe the right infrastructure, but they might not know where really the value at the application layer is unlocked in these models, in the functionality that they have.

I know you spent a lot of time kind of categorizing certain things that you’ve seen as trends, and commonalities among the solutions that you’ve developed over time… And I really like how you frame these in – you know, earlier today we were at lunch, and talking about some of these less from the functional level, maybe a knowledge-retrieval or information extraction, and more from the kind of task or goal level. So like generating new ideas with AI is one of the ones that you mentioned to me, preserving important know-how. I think these sorts of ideas that can connect with a less technical audience. So how did you come on these trends, and anyone that maybe – we’ll go through a couple, but any one of these that you would want to highlight?

Yeah, so you’re talking about our solution archetypes. And initially, I was asked by my team to identify all of the different archetypes of solutions that I’ve tackled and sort of solved for. So none of the ones that we couldn’t figure out; only the ones that we were able to get the client to sit forward in their seat and say “Holy smokes, it can do that.” And I identified 35. And I brought it back to the team and they said “35 is too many of anything. Get that down.” And so we got it down to 15. And so inside of each of the 15, what you’ll see is like three or four or five examples of like either full-blown applications, or just a simple script that runs in the background…

Here’s what I’ll say - for the people who look at these tools, who’ve spent any time around them and will tell you “Nah, they’re not there yet. They’re still baking”, whatever. They’re probably people who have really only interacted with them in a chatbot kind of environment like ChatGPT. I realized ChatGPT is more than a bot. But I can definitely appreciate how you could come to that conclusion if you’re a business leader and you derp around on ChatGPT for an afternoon, and you kind of go like “How would I use this?” It isn’t until you start to bake personas or agents, or if anyone’s listening and you just don’t fully understand what that means, just to say an AI that has an awareness of a discrete task and purpose. So it’s kind of like “I do this specific thing.”

[00:16:04.20] When you start to learn how to program very good personas and park them in the middle of code, where you can kind of automate them and have them “Oh, this file hits this folder, trip off this chain reaction.” And then this is the moment that used to be unsolvable or would have cost 10 million dollars to figure out, because there’s so many if, or, either ands… Forget all that. Let’s just aim this persona at it. Poof. Now we know which direction to go.

And that’s really like the simplest version of what to say. I will say, for those of you who have perked up and thought, “Yeah, but how can we trust it?” It’s like “Hold on. Hit the brakes. How do you trust your coworkers?” Like, we work with people all day, and the same concerns that we have about these AI agents, I think most of them you could map onto your coworkers. And the reality is - how can we trust it? The answer is because we worked really hard to make it very good.

Or maybe you can’t trust it a hundred percent of the time, but maybe there are jobs that 90% of the time is good enough. 99% of the time is good. 30% of the time is good enough. There’s all sorts of work where it’s just kind of like something is better than nothing. Or “No one was going to do this anyway.” Or “This just doesn’t even make sense for a human to do, because you’ve got to do it 10,000 times a minute, and we’re not going to pay that many people.”

Break: [00:17:27.12]

Yeah, I know one of the things, and it’s super-interesting to me, but I think also it’s unlocked a little bit in terms of what you’ve talked about in our meetings, is that there’s a certain element of operational efficiencies that can be gained with these tools… If you look at a human process and think “How could we use a gen AI tool? How could we use a language model to improve the efficiency of that process?” So you have like five things in a sequence… How can it help me do two or three of those things faster, or something like that? That’s certainly true, but I love how you’ve talked a little bit about finding something upstream, or going upstream of that to really think about - and maybe this is part of the thing that people are missing when they’re trying to find the real value that AI unlocks in their organization, is that they have a set process in their mind, and they could see “Well, yeah, we could do this 10% faster.” But is that worth the money that they’re investing to an LLM system at a certain scale? So yeah, could you help maybe talk us through that upstream value and how you think about honing in on that?

The reality is that when you are solutioning, it is often a barrier for the client to visualize a solution, because we tend to map sort of a biomorphic representation of the process onto the problem and the solution. And we’re trying to imagine a robot doing a human’s job, and just instantly you give it arms and legs and a head, right? So for instance, if I were to say “Daniel, how long do you think it’ll be before robots replace cashiers at the grocery store?” So we have Kroger in Cincinnati. So at Kroger, how long? And instantly, where does your mind go? It’s kind of like “Well, I’m just trying to imagine, okay, the Tesla Optimus bot”, or one of these, just trying to fumble around with your delicate groceries, and not crushing it, and scanning it… And it just seems so hard to think through and pull off. And you would just map that so far out into the future. And what you don’t know is that my little brother, who works with the robots for Kroger, look nothing like what has popped into your head. They’re like these little square garbage can-looking things that run around on tracks; it’s like they really operate on the z-axis, so it’s like they’re stacked on top of each other, and these grid systems, and they’re dropping products from one to another… And in the end, in the bottom, a bag of someone’s order comes out and it loads right onto a truck, and a human doesn’t touch it.

And this is an example of how it’s just really hard, when you’re staring at a problem, trying to think of a solution, it’s really hard to think way upstream. But the reality is the solutions tend to live up there. We start so late. We often start at letter Z, when like we should be starting with “What’s an alphabet?”

And yeah, why are we using an alphabet?

Yeah, why are we using an alphabet? And I know that sounds “Oh, that sounds expensive. What are we going to be, talking for days?” No. All I’m saying is like – the example we used at lunch was “Oh, I think that for a developer these Copilot tools are handy, because it can kind of double-check their code.” Well, what I’m talking about is what if an organization had been mindful about tagging and consolidating all of the relevant context for the problem that the developer has been asked to solve. And programmatically, it’s just automatically injected into the context of that Copilot, so that it’s aware of the voice of the client. It’s aware of the complaints online. I mean, I know this is far down the road, but this is probably going to be built for us pretty soon. I think it’s a new paradigm of work, a new way of working. The way we work now is legacy.

The new paradigm will be that way upstream of our work, way upstream of our work, special care has been put into collecting and curating and injecting the appropriate information into something like a language model or some more sophisticated, maybe multimodal version of that, so that it can augment our workflow. And I only say that because it’s how I work now. So I don’t let anything go to waste. I mean, use it all. Every scrap of communication. I like to transcribe if I can, and make sure it exists in some sort of secure context, so that – it’s just double-checking my work, and helping me ideate. And I find that - it’s muscle memory for me now, but the time requirement to set that up pays off exponentially down the road, when I need to reopen a case, and I’ve got this context that is fully locked in, completely aligned to the who, the what, the when, the where, the why of what we’re being asked to do… And I just, I’m really excited for a future where that’s just handled for people, you know, who aren’t weird like me.

[00:26:19.25] And yeah, I guess that gets a little bit – I know one of your passions is thinking about how your work generally, but also the technology and other things impacts people at a human level, and in particular, even vulnerable people, or marginalized groups. And it’s interesting for me to hear you talk about that, where a lot of times what people hear is, “Hey, how can we make this role more efficient, so that maybe we have this focus workforce-wise on efficiency? And how do we make this role more efficient? We’re going to need less of these people”, which is certainly a relevant thing that we need to talk about. But here, what you’re talking about is more of a new paradigm, where those people that are working are enabled to do actually net new things. And I also imagine - maybe you could speak to this if you have an idea about it, but opportunities and things that are unlocked for maybe people that are in vulnerable or marginalized situations, that maybe would be opportunities or things that they would have access to, that they didn’t before. And so this is maybe a different paradigm of looking at the workforce impact of AI, I guess.

I love that you’ve asked me about this. I’ve had a few different big light bulb moments when it comes to just thinking through imagining future products that might exist as a result of this brand new technology. It’s a brand new thing to people. And I imagined once that there might be a vulnerable single mother, sitting at home, who maybe doesn’t realize she’s being exploited. I mean, I’m imagining a fictitious character. I don’t know this person, but maybe there’s a vulnerable single mother sitting, who’s being exploited and doesn’t even know it. Can you imagine – the costs of running these are trending towards zero. We see that, right? So it’s not hard to imagine a wearable that ingests contexts, and sort of helps someone navigate life. Maybe might make them aware of their rights. Maybe might be able to act agentically on their behalf, by maybe filing a complaint somewhere, or putting a situation where some marginalized person is underrepresented in a way.

It excites me when I realize that a primary difference between the forgotten, the neglected, the marginalized, the primary difference between them and others is that the others generally just have someone willing and able to act on their behalf because it makes financial sense.

When the cost of cognitive work and some of these automated or these agentic type actions are free, or could even be ad-supported, because they’re like “One tenth of a penny for that type of help”, that’s exciting to me. And I do look forward to a future where people have access to advocacy, even though there’s no real advocate.

Yeah, that’s so encouraging. Certainly, like I say, there’s an element of workforce impact of AI that needs to be considered on the negative side, I guess, if you want to consider it that way… But also, there’s a positive side and opportunities that are unlocked, for sure.

I have to say, that tool will never exist if no one builds it, Daniel. And so this is just a challenge. Like, I’m doing this personally, and we can unpack how.But if you’ve been given gifts that overlap with like an expertise here, it’s incumbent on each of us to think through, “How can I make the world just a smidge better?” because I didn’t just ignore all the need that overlaps with my expertise. And it’s not just about writing a check at Christmastime, you know… Yeah, hunt that out.

[00:30:18.19] It’s about - yeah, creating that outpost of goodness and beauty in what we create, and not just like in your case, in your background with art, and actual sort of fine art, but also in the technologies and the products and the tools that we’re building. I love that. I’d love to maybe talk, maybe just highlight a couple of these solution archetypes, because I think this is also kind of helpful maybe for those in our audience, maybe they’re on the business side, and they would connect with them, but also maybe they’re on the technical side, and they’re just struggling to connect certain ways in which they can communicate the potential of this technology to stakeholders… So I love the one that you have here of generating new ideas with AI. How do you think about that as a solution archetype?

Well, the obvious way to do this, that anyone would just think of on their own, is you could go to something like ChatGPT and say – you know, go a step beyond what a normal sort of ho-hum user would do, which is to use it like Google… But to ask it for help with inspiration. And so anyone could just lean forward in their chair and open ChatGPT and do that right now.

But the use case that we highlight in our solution archetypes documentation - there are two or three under that specific archetype. But the one that I’m most proud of, and will talk about today, is the first app I ever built. And this app was a persona named Andy. So Andy is a roundtable discussion focus group style facilitator. So there’s essentially – it’s really just a system command, plus some tuned hyperparameters that help Andy do his job well. But Andy’s supported in Python, with the ability to synthesize or spawn new personas.

And so what Andy will do is listen to the transcript of a meeting - and I would love if there was like a low enough latency audio… You know, like the files that OpenAI is kind of working on right now. I’d love to be able to get Andy and the whole group participating in real time. But for right now, the way it works is you can kind of give him a chunk of transcript, and what Andy will do is kind of look at the problem you’re trying to solve, and spin up a focus group.

So Andy creates - I think it was six or eight, I think maybe eight, additional personas. So all that means is in the Python code there’s eight new things that Andy just created system messages and some hyperparameter tuning, so that now in the code the tool has the functionality to summon any of these additional personas to weigh in on the conversation. And so Andy as a facilitator can pick which one has a turn to talk.

And so then Andy opens the group - and you can use the world knowledge for this bit, and I did. And he just understood how to run a focus group, which actually is cool, because I didn’t know how to do that when I had the idea of doing it. And so Andy will say “Okay, today we’re going to talk about new product ideas for X industry.” We happen to do this for the publishing company where I worked, so I was still working for them when I made this tool. And Andy could – I think it was like a fireman, a nurse, a web developer, a doctor, around the table… And the problem this company had was their new product development team was struggling to come up with new products. And pretty much at the end of each meeting, it would just be the same product. And this was a problem.

[00:33:50.10] And so Andy understood “My job is to think of new ideas, but then kick it to the group for discussion.” And it was so fascinating to watch them talk about it. In this instance, the very first time we ever used it, someone in the group came up with an idea for something that was very close to the products that they had been making, but a little different… And then the nurse piped up and said “That’s great, but that form factor would not work at all for my job. And if I wanted to bring that to work and benefit from it, it would need to be smaller, so that it could fit in my pocket or purse.”

And so then someone else built on it and said “You’re describing a deck of cards, and this could work, because…” And then the web developer kicked in behind and said “That’s a great idea. If there was a QR code on the cards, you could then link it to an account that they have. Here, let me write that right now.” Of course, the code didn’t fully work, but it was cool. You could see it try to write the code… And this - we were just letting it run. They loved the idea. It was the freshest, best idea they’d had in the seven-year existence of the new product development team. It was a brand new product idea. And it was the first app I’d written, and it was the first time we ran it, and that for me was a major moment. And that was on GPT-3.

Yeah. And there’s an element to this solutioning that you’ve been doing that makes me think about how quickly you’re able to get through to a proof of concept, or demonstrating the value that you had in mind with a certain AI use case… And I’m wondering, as you’re doing this solution, I’m guessing that there’s a balance or maybe potential objections here. In this scenario I could see how a company might object to putting in so much product information or IP information into one of these systems for privacy reasons, or maybe they’re thinking “Oh, what you’re describing is going to take so many, so many LLM calls. It’s going to be slow, or maybe costly”, which maybe are relevant concerns, but also, you kind of need to know what is possible before you get to some of that optimization, and also try to get to an end-to-end solution. So how do you think about, as you’re solutioning, balancing this desire to move quickly and get to a solution, an end-to-end thing, versus privacy, security types of concerns, and balancing those things that maybe are optimizations or scale issues or environment issues with the ability to get to an end-to-end solution?

Yeah. I’m sitting across the table from maybe the best person in the world at this, the best I’ve ever met, when it comes to thoughtfully considering security and the risks associated with it. I will say, this is not something to ignore.

Ever. And we’ve learned to start with security when it comes to forming relationships and assurances with a new client. And we need to be good at that, right? But. [laughter] You’ve got to start somewhere. And sometimes what we do is we start with synthetic data. So it’s kind of like “Hey, pharma company, we’re going to invent a synthetic drug, and we’re going to make all the synthetic assets that a company would have, and we’re going to–”, and what’s neat about that is these tools do that for you. So, I mean, if you have a persona – remember I said the new paradigm is having a persona fully aligned? Well, that persona that I’ve taken the time to get up to speed on the project can just spit out all this stuff I need, to prove to them that what they want to do is possible.

There are so many trust-me’s floating around in this space. Like, “It can do it. We’ll get you there in 36 months. You only need to pay us X million dollars to build out your…” And it’s BS. Maybe that works for a big co who has a metabolism that is the same as a 20-ton whale. But for mid co, mid/small, that has a normal human metabolism and would like to see something before Christmas, there are ways to get to POC, proof of concept in sometimes hours.

I have had persuasive proof of concept scripts written before the end of meetings, where the client is talking about a solution they want. And how do you do that? Because I probably went into the meeting with an aligned persona on the goals, I hear a few little things in the meeting, and I have a bot that I’ve created called PersonaCraft, that I can just throw in a few things and bang, bang, bang, okay, throw that in Python… “Is this what you’re talking about?” Like, holy smokes. So yeah…

Break: [00:38:42.03]

Yeah, certainly there’s an element of this where you are acting as an architect and a solution developer, getting to that proof of concept, showing value… But I know one of the other things that you’re particularly passionate about and have been exploring is actually finding individuals in organizations that are champions, or could be champions of AI… And maybe they’re not from a technical programming background, maybe they are, but teaching them how to utilize these tools effectively in a sort of crash course kind of way… So how do you think about that, and what are some of the things that you’re taking people through to help them understand how to use these tools in an effective way?

I just want to preface this bit by saying I am trying to become a master at what I’m about to describe, and I’m not there yet. This has been the hardest thing for me to navigate so far. Here’s the reality. Companies need more than code. Whether or not they realize it, when they tell me “We want to become AI-enabled”, I think some part of it is they want their people to know and understand this stuff.

And I think if you just kind of trust it to luck that people who have a really demanding job are going to develop an accurate sense for what’s going on under the hood, you’re going to be disappointed. And so we have developed – the company that bought my small business happened to be a change management, an instructional design company. They have helped BigCo for decades navigate massive change. And so it’s just kind of lucky… I mean, it’s actually part of why I liked the offer. But I have worked with their experts to develop a pretty well-defined approach, with mixed results. Where I want to get better is I want to get better at identifying the people who are really going to become the super-user rockstars.

So we’ve developed this tool called a FitChecker, that essentially is a quiz. It’s like some sliders they slide, and it helps us kind of score… I don’t even really know if that’s going to work great or not, but what we are doing is letting them get a peek under the hood at what’s going on with these tools. Because we want to demystify it, right? But they think it’s magic. Or BS. Or both. Can I just kind of run you through the beginning of what it would be like in one of our early sessions with them, so you can see – okay. And feel free to try this in your organization, if you’re kind of trying to align with someone on like “What’s actually happening with these tools?”

And so what we’ll do in the very beginning is I’ll stand in front of the room or one of our coaches will stand in front of the room and say “Hey, everybody, I’m going to say something, and I’m going to say a few words, and what I want you to do–” and you can maybe even do this in your car, or if you’re listening to this at home, just blurt out what pops in your mind. And so I’m going to say some words, and then you’re going to blurt out what pops in your mind. Are you ready? Here we go. “Peanut butter and?”

Jelly.

Jelly. It’s jelly every time. It’s always jelly. And the whole room says jelly. No one says anything except jelly. And then what we do is we explain to them that they essentially provided a completion to our prompt. And jelly happens to be a token, and peanut butter – peanut, if it’s a capital P, is three tokens. Butter is a token. And that’s the GPT-4 tokenizer. And so then we can open up a tokenizer and we can show them those individual tokens. And we can click on the token IDs and we can show them that those are just numbers. And that there’s this language model that is trained to spot weird patterns to what tends to come next as a number when you give it a string of numbers. Those were converted to bits or entire words, and then it just spit back jelly.

And so when you understand that – and then we tell them “Look, if it spits a sentence back, it’s just kind of a token, token”, getting longer, getting longer, and it’s starting from the beginning… And they get a sense for the “Oh, it’s just mimicking language.” It’s mimicking what a next word that a human might say when they heard a thing. When you understand that it’s working that way, it helps you use it better.

[00:46:11.06] So then, while I’m talking about all that, I might start talking about blues, and rock and roll… I might mention Memphis, Cadillacs… Maybe I’ll say something about Graceland… And I’m talking about all this, and then I’ll say “Let’s try it again. Peanut butter and”, just like jelly, jelly… Someone yells banana. And behind me on the board, I take a thing off and we’ve written banana on the board. And then we just have this weird moment, because some people in the room are young, and they hardly know who Elvis is… And I don’t even know if you would have gotten that, Daniel.

Yeah, I definitely would have gotten the Elvis, Graceland et cetera… But – I would have been somewhere there.

Yeah. And so I think it’s that Elvis, like, peanut butter and banana sandwiches, or something. And so now we can just unpack the concept of context. Because by inserting some new ideas, we sort of shifted the probability of that index of potential tokens and the order that they would be. And we’ve shifted it enough so that a banana could potentially show up as worth picking. And someone did. And that just sort of demonstrates, “Well, most of you still said jelly, but someone said banana.” I’m just telling you, it has never not worked. Like, it always works.

And so then we start talking about, okay, so these are tokens. They’re parts of words, they’re numbers, they go together, it can predict the next one to mimic human communication or thought… And then there’s only a certain amount of these that can fit in its brain, or its memory, its context window. And we’ll show them on a board. We just put little hashes for like “This is a token, and this is a token…” Well, and what happens when we get to the end? Because Daniel, when I started working with these tools, we only had 2,000 to work with. And now it feels infinite, but not really. But because you learn early, because you learn early, you learn to work within the limitations of the tool and you learn elegant workflow then… Because we appreciate – even though it feels big, it’s still scarce, because they’ve got to pay for what we submit.

And so we show them, these are how the tokens stack up, and then we show them, like “When you start getting toward the end, if we fill up this 2,000, what’s it going to do?” It’s chopping off the ones in the beginning. And then we ask them, “Have you ever had an experience with ChatGPT where it feels like “I’ve already told you that”? Oh yeah, you see some – oh…! Oh, because it didn’t forget it. It’s selectively deleted it, you know?

And so we just run them through… That’s like the first 10 minutes, and we just build on that, and build on that, so that they build good AI usage habits, and they can interact with the tool in a way that leverages the strength of the tool, and with an awareness of what can go wrong.

Yeah. I love this approach. In my own workshops, when I start going through some of this, of how a generation actually happens, how text is generated, I often see that a number of really important and interesting questions naturally pop out of that. Like “Hey, if this is just producing these tokens in this way, how could it produce anything meaningful? And how does it seem so coherent, like there’s an intent behind it? Why would it produce anything actually valuable and connected to the real world?” So yeah, I love this.

So because we get all those same questions, we have identified what we call Layer 1, Layer 2, and Layer 3 knowledge. We only teach Layer 1 knowledge in our bootcamp. We will address Layer 2 and Layer 3 questions individually, but not as a group. Because the Layer 1 knowledge is the strict information set that is required to use the tool well. And so we would never talk about prompt injection attack in the group setting, because what we don’t want to do is scare off the non-technical person… And in fact, we target the non-technical people. This is their moment. You finally program computers with plain language, you know?

[00:50:22.00] Yeah, really good points. And on the data side, I think one of the interesting things is – like you say, not going into some of those details in that context, but definitely getting to a point where people can realize how data starts to become integrated with these AI systems in a meaningful way. An example I often give is, you know, I go into a chat interface, I say “Summarize this email for me.” And of course, you haven’t pasted in an email, the model doesn’t know how to summarize an email that it’s not given, but everyone knows what to do… They could paste in the email that they want summarized, and ignoring the kind of potentially policy-related things around that… It immediately makes sense that people can bring the right data to the table, inject it into these existing models, without having to do complicated fine-tuning, and actually produce value.

Can I tell you our five-step process to effective reasoning? So we have a slide on this, and we teach it. I hope I can remember all five, because I just told you there’s five. What we teach when it comes to reasoning over data, or just processing information - it could really be almost any project. We teach to do it in this order. So we say, what we like to do is teach them to start with end goal alignment. And so because we know that – I think it’s known, that these models tend to pay a little more attention to what’s going on early and late in your context, we want to get the real end goal articulated very early. So a neat trick I have for end goal – so this actually can be fun. So this is a way you can work from your hot tub. Maybe you should open the show with that clip. [laughter] So this is a way you can work from your hot tub. I like to put in my AirPods and hit Record on my phone, and just talk about what I need to get done on this project. And just talk, talk, talk, talk, talk. Get it out. This way I don’t have to sit in front of my computer at my desk; or I might go on a walk, or on a treadmill, and just talk through “This is the who, the what, the when, the where, the why”, and get it all out there. Okay.

Now, what we do is we transcribe that. I use Mac Whisper. So I’m doing that locally. So it doesn’t matter – if this is something for a top secret project, I can transcribe that locally. And then I can inject that into essentially my first prompt. And it might be big. I’ve had those go on for three hours, or as quick as 10 seconds. And what we tend to do is try to rush to work, but there’s a bunch of steps before we start working.

So step one is establishing the end goal. Step two would be to validate the end goal. So now we can go back to that same model and say, “Tell me what you think we’re going to do. Just lay it all out. Give me bullet points, you know?” And so it will go through and say “I’m expecting you to do this, and you to do this, and then I’m going to do this, and we’re going to do this, and what the client wants is this, and so we’re probably going to do that…” And you can look at it and go “Actually, this or that is wrong.”

Now, what I like to do, instead of playing AI whack-a-mole at this point… So any AI nerd knows exactly what I’m talking about. I use these next few bits of conversation to identify where it misunderstands the end goal, if it does at all. And then what I do is I go back to my source material and I modify my source material to clarify. And so what we do is we get rid of all those subsequent prompts for validation. Validation, for me, is not anything that needs to linger in the context. So I delete all that.

[00:54:09.14] So step two was validation of end goal alignment. Step three is we curate and ingest our assets; like you said, the email.

Remember I said curate, and that is big. You don’t always have to do it, but it’s helpful. Remember I said I learned on 2,000 tokens? This is why I make a habit of curating my content before I put it in there. What’s the least – now, there’s a balancing act here. There’s a trade-off. Cost-benefit. You don’t want to overdo this step, but it is helpful to trim the fat out of transcripts. If you’ve got a transcript saved as VTTs, and you look at it and you open that up in Notepad, 80% of the tokens are time code. So maybe we get a docx that just has speaker names… So we’re just mindful of like what are we throwing in next that isn’t going to distract the model that’s relevant?

Okay, let’s pretend like we’ve done all that work. What do we do now? Do we start working? No. We validate again. So we go back and we say “What do you understand that I just gave you? And how do you understand that it relates to our end goal?” And it will talk about it. Usually, this part it gets right, if we’ve done a good job of end goal alignment and curation. But sometimes you’ll spot “Oh, it can’t see the infographics.” I assumed, by attaching these PDFs that – okay, what color is the biggest slice on page 14 of the pie chart? That’s what a validation thing might look like over your data, right? “Okay, you can see that pie chart. Okay. You see the orange. Okay, you know what that is. How’s that relate to our project? Okay, you understand. Okay.” You validate all that. Now, get rid of that, because that doesn’t need to persist, but I need to be comfortable this model understands the goal. So it’s aligned.

Now it’s finally time to start work. We’ve done all this work, and boy, can you burn through a project when you’ve done those steps… And it’s just, this is the fun part, really, where we just start – you can almost just say “Okay, go” at this point, and it starts to do the work. And then I think the fifth step - I haven’t been numbering them on purpose. The fifth step is just letting it synthesize assets. And this can not only be like – if you’re using ChatGPT, you can have it create spreadsheets or whatever, but you can also, if it understands, “Hey, I want to use UDO to make a song about that”, or “I want to use ideogram 2 to like create slides for that”, or “I want to use…” And you can just have it start to create prompts for other things, and go off and do it, and it’s really fun now. So we’ve done all that, and we’ve got that context persistent. The client sends us an email and complains about whatever - what do we do? Copy-paste. Boom, done. It’s just so little work.

Yeah, that’s super-encouraging. I love this example because oftentimes I might be in a meeting, and you have to respond very quickly, but a lot of times it’s those slow times where you’re able to think about something; for me, on a walk around the block, or on a hike, or something like that, where I’m really present, and I’m maybe more calm and thinking through things in a slower fashion… But I also want to capture that, and I want to summarize it, I want to bring it into conversations in the midst of my work day. So I love that example. And I think it also illustrates how you’ve integrated this technology… You’re practicing what you preach, to some degree, kind of integrating this technology into your daily rhythms. And that brings me to maybe a last question here, as we’re starting to wrap up. As you look to the future and you’re thinking about the new solutions that you’re creating for people, the things that are on your mind, the things that you’re architecting, what’s most exciting for you now? What are you excited to explore? What are you excited to kind of prototype with the partners and the customers that you’re working with? What excites you looking forward?

[00:58:05.17] Well, I will stand by what I said initially, that if we never got another innovation, these tools are good enough as they are to justify going down on the timeline of humanity as a massive technological revolution. And the tools that I wish for, that are already sort of – they’re almost done baking. The tools that I wish for are really low-latency, in and out, audio, video, image agents. I’ve built so many tools that interact with people. One class is called like Mind Mappers, where we’re essentially trying to convert tacit knowledge to documented process. So you’ve got this single point of failure person in your organization… How do you map out their subject matter expertise?

So what would be awesome is if there was a low-latency conversant model. So like what OpenAI kind of has, but it’s walled off right now. An API where I could give a person an option between an email chain, text messages, a phone call… And right now I can’t do the phone call. And that bothers me.

And then I think too, just as people start to see the value of the tools, that maybe they’re a little less weirded out and fearful of interacting with them. And so, of course, it makes sense that people feel nervous that they’re going to lose their job. The reality is they are going to lose their job as it exists today if there’s knowledge work. They should, because that’s like continuing to want to farm land with an ox, and whatever the thing behind – a plow, I guess. That’s wishing for keeping that, when the tractor dealership just showed up and you’re like “No–” We’re not going to want to go back to the days of like trudging through tedious knowledge work that could be automated.

But it is going to be a process that requires empathy on the part of us, the AI engineers, the architects, the developers, the leaders, and to appreciate, like, this is a scary thing. Another thing I wish for is just to get past this adolescent phase, to where the tools feel less scary. And then I’ve got to go back to - I am very excited to see what it means for learners, who maybe their teachers don’t have time to go on every curiosity journey with them, or explain every little lesson… That’s really big. I could give you such a long list… So I’ll just stop there. I am very excited for how this technology develops, but I will say, the good guys have to have a hand on the wheel here, Daniel. And if you leave it up to BigCo, we won’t have the stuff that really makes life better for a wide range of people.

Yeah. Well, I think that’s a great note to end on. Thank you, Mike, so much for joining. And I would definitely encourage you to heed Mike’s call. Get your hands dirty, think about those solutions that are going to create goodness and create new paradigms of the way that we work. So yeah, thank you so much, Mike, for joining.

Yeah, thank you.