Daniel Whitenack: I’ve found some of the comments on specific ones of these interesting from particular people that are especially well-positioned to comment on some of these items. So for example, the number five one, the focus on small language models, but also with the perspective of becoming more economical, and cost and compute-efficient… Clem from Hugging Face, the CEO, made a video which is really nice; I recommend everyone watch that on Twitter, or other places that it’s posted… But he made some comments about his prediction that one of the hyped AI companies - certainly a lot of them now - would go bankrupt in 2024, or get acquired for a low price. And he tied that in with the comments along the lines of cost efficiency, and focusing on cost of running these models… Because yeah, you likely have a lot of startups that have raised big money; their compute costs are probably astronomical, because they’re running these large models at scale, and hoping that their margins get better over time… But once you make the shift to open models and cost-efficient models, that may not work out in their favor, and so the ability for people to run models in their own infrastructure, run more cost-efficient models - that’s not going to play out well for certain people. But it will play out well generally for the costs of running these sorts of systems in enterprises, whether that be – it still could be a software system that runs LLMs and is self-hosted within an enterprise, but it’s going to be much more cost-efficient to do that, especially for those that are wanting to pull some of that in, not rely on external systems, be more privacy-conserving, not have data leave their infrastructure… That’s going to become more and more possible.

So yeah, I thought it was interesting how Clem tied together some of his predictions around yes, being more cost and compute efficient, which is a benefit to the climate, for those that are thinking about those things, but also cost-efficient in terms of enterprise and operational costs… And how the focus on that will not work out that great for certain of these kind of hyped AI plays.

Jerod Santo: I hope he survives, yes, just so that we can call Chris wrong last year… But that’s an epic call. Alright, Nick, can you top that? 2024 prediction, better than what Chris just did… Because one’s amazing.

Abubakar Abid: Yeah. And we had thousands of people use the model, or I’d say maybe tens of thousands, and tried on their own profile pictures, pictures of their pets, everyday objects… And they were interacting with actually state-of-the-art machine learning, which is something that’s never been done before. And that is really cool, because now machine learning developers are thinking, “Hey, the audience who is using my model is a lot bigger. Let me make sure that my models are robust, they can handle diverse images, diverse inputs.” And that leads to a level of, I think, concern for the end user that wasn’t really there before. I think that’s really important, because then issues of bias and accessibility are addressed as well.

I want to give one other example as well, because I think this is interesting as well, because demos have a big purpose for education as well. And I remember one of the early days of Gradio, actually, we had built like this demo for an MNIST model, and I actually shared this… Anyway, this MNIST model, you can draw handwritten digits, and you can see what the prediction of the model is… And I shared it with my sister, my younger sister, who has like no background in machine learning whatsoever. She tried using the model and she drew a six, and it predicted six, and all that work. And then she just drew a little dot in the center, and I think it predicted like a seven. And she was like, “Why did it predict a seven? That makes no sense. I just drew a dot.” And then I told her, “Well, it kind of has to predict something, you know, and maybe the sevens were just the most thing in the dataset.” And she was like, “Hmm, that doesn’t seem right”, and I was like, “Well, what should it do?” and she said, “Well, it should just avoid making a prediction. It should kind of abstain for making a prediction.”

[20:21] And so she had stumbled upon this idea of abstention, which is now a really important topic, and lot of people think about that. But if you’ve never really interacted with a machine learning model in this way, you might not even realize the importance of it. And this is someone who has no background in machine learning. So I think demos can go a really far away in both accessibility, and then also education as well.

Zac Smith: Even 18 months ago it wasn’t even on the radar. Now - right at the top. So… Okay, that’s great, because now we see business drivers… I think people are gonna pay for this too, which is really important… Because you don’t just get sustainability for free. We don’t get to just do “Oh, we did green power for you. It was a good marketing thing.” No, no, no. We invested tons of money to make meaningful impact to change our world; that is going to cost money. We are going to invest together. So I think that’s an important – that’s number one.

I think actually number two is that at some point, if we can solve this distribution of technology - right thing, at the right place, at the right time, so that way you could pull up on your iPhone and see the tracking of your cool computer to the right market, and then just turned on… And if we could snap our fingers and – let’s say you figured out just the right technology that you needed to use for your platform, and then you clicked a button or hit an API call, and somebody like Equinix got it into 50 to 60 markets around the world in like a couple weeks… That would be rad. [laughs] And I think we would see disruption in content delivery, and CDNs, and edge computing, and all kinds of things that we would do, and networks, and all things it could run on - I’m gonna call it hardware and software moving at their own pace.

So pending we solve this distribution thing, I think the big – and this is, again, probably… Now, you’d have to ask me what’s the 2025 predictions. That’s way more my style. But 2022, I’m not sure.

2020-something - the other thing I think is gonna be security. Right now, people just try and get the hardware or the thing in the right place, at right time, and they’re lucky to have it. That is not going to be our long-term challenge. We’ll solve that. Then we need to solve a way different approach to security, and that has to start at the hardware level. So I think our enablement of hardware-level security has barely begun. Most people don’t think about it in the software enablement side. They think about “Oh, I’m gonna encrypt my stuff, I’m gonna get my TLS going, I’m going to do all those things…” But really, even things like basic time protocols, basic boot processes… Is this machine the thing I think it is? Who touched it in the supply chain?

Daniel Whitenack: Yeah. I don’t know if you’ve seen the trend… I think this was a prediction for 2024, which I think was a well-informed prediction for 2024 from many different people… And I think we talked about this in our own discussions of 2024, around multi-modality of AI in 2024. Whereas in 2023, you kind of saw this explosion of, in many cases, text-to-text AI, meaning I put in a text prompt and I get some text back. Now we’re seeing an explosion of multiple modalities of data input and/or output to these models… And that’s mostly what I’m seeing. Is that consistent with your view as well, Chris?

Jerod Santo: I’m ready to make my first prediction for 2021, now that I’m sitting here and listening to you two… And I’m gonna go ahead and predict that TypeScript will have reached the top of the hype cycle, begin its way down…

Divya Sasidharan: Prediction for 2020.

Jerod Santo: It’s now time for Sponsored News!

Six predictions for AI in 2025

“The rise of specialty models is coming”, according to Augment Code CEO, Scott Dietzen.

He says:

With DeepSeek and open source AIs closing the gap, concerns raised about LLM commoditization, advances in LLM reasoning, and questions about the future of the scaling laws… 2025 is shaping up to be a tumultuous year in AI. And it’s only February .

Scott has some interesting takes from the front lines of AI tooling. His third prediction in the list of six is one you’re likely to appreciate: Coding AIs increase demand for software engineers

Check out the entire list of predictions and what the Augment Code team is doing about it at augmentcode.com

Feross Aboukhadijeh: [58:00] So I have two predictions for 2023. So the first is - and this is really just kind of the continuation of a trend… I guess both of mine are really like continuations of trends that are already happening, I think. The first is that all the biggest open source projects seem to be backed by companies these days. I have mixed feelings about that, because I love independent open source, published by just like random individuals, just doing it out of the love… But it’s kind of unavoidable. If you look around at all the projects that we use, they’re usually all backed by companies, and that’s actually kind of a good thing in some ways, because they get a lot of support, and they get a lot of money and stuff put into documentation, and all this other stuff… So yeah, I just think about the stuff I use… I mean, Next.js, React, VS Code - all that stuff has big companies behind it, and so I just predict that trend will continue.

The other prediction is - if you look at 2022, there was several headline-grabbing supply chain attacks involving npm packages… The two big ones that come to my mind are Colors.js, where the maintainer sabotaged the code, decided he wanted to kind of put like infinite loops, and weird print statements and stuff into his package, and that affected a whole bunch of folks. And then the other one was the peacenotwar package, where the maintainer of that wanted to protest the war, and did that by deleting people’s hard drives if their IP address appeared to be coming from Russia. And so those are the ones that stuck in my mind.

So I just predict there’s going to be more of this stuff. I know that npm has done some good stuff on the security side with mandating 2FA for popular packages, and stuff like that, but I just feel like there’s still so many ways that the maintainer themselves can go rogue, sabotage their code, or can add a new maintainer who turns out to be an attacker in disguise, or just biding their time, and then they turn evil at some point… So I just feel like despite the progress we’ve made, 2023 is going to have several more of these headline-grabbing attacks in packages with millions of downloads. Yeah, just stay on the alert, and do what you can to avoid it.

Jerod Santo: Let me slightly change the subject, but keep it on point. I think this will be interesting, because I would love to have your take on this. Your prediction of a major AWS outage in 2024 reminds me a lot of the predictions that were made late 2023, when Elon Musk bought Twitter, and laid off something like 70% of the company. And the predictions, which I was pretty much like “Makes sense to me”, was like “There will be a major Twitter failure, operational, technical, operating at scale. Twitter will be down and gone soon. Sometime this year.” And - I mean, this hasn’t been hiccup-free, by any means. There have been outages, and stuff. But on the whole, it seems it’s going okay over there in terms of twitter.com and the APIs. Or x.com now. I don’t know. Mine’s still twitter.com, but it says X…

Daniel Whitenack: Not to sell-fulfill my own prophecy from our predictions from last year… I think in our 2024 predictions episode one of my predictions was that we would see a lot more combination of, I think what is generally being called neurosymbolic methods, but maybe more generally just like hybrid methods between what we’ve been doing in data science forever, and a kind of frontend that is a natural language interface driven by a generative AI model.

So in this case, what we have is good old-fashioned data analytics, just like the way we’ve always done it by running SQL queries. It’s just we gained flexibility in doing those data analytics by generating the SQL query out of a natural language prompt, using a large language model. And I think we’ll see other things like this, like tools, and Langchain is a great example of this, where you generate good old-fashioned structured input to an API, and that API is called and it gives you a result… But this could be applied in all sorts of ways, right? So let’s say time-series forecasting. I don’t think right now language models - and I’ve actually even tried some of this with fraud detection forecasting and other things with large language models, and they’re not very good at doing these tasks. But they can generate the input to what you would need in the kind of traditional data science task. So if you say - again, imagining bringing in this SQL query stuff, if you have a user, and you want to enable that user to do forecasts on their own data, well you could have them like fill out a form in like a web app, and click a button and do a bunch of work, or you could just have them say “Hey, I want to forecast my sales of this product for the next six months”, or something.

[34:19] From that request, a large language model will be very good at extracting the parameters that are needed, and possibly generating a SQL query to pull the right data that’s needed to be input to a forecast. But that forecast is going to be best to – best that you just use like Meta’s profit framework or something, that’s just a traditional Arima statistical forecasting methodology… And you just like forecast it out with that input, and then you get the result. So this is a very – it’s the merging of what we’ve been doing in data science forever, with this very flexible frontend interface. I think we’ll see a lot more

of that.

Adam Stacoviak: At any age. Forget that. Let’s zoom out for one second, because there’s one thing that I took a note of from something you recently wrote… And you mentioned this, so I’m gonna go back to it; it was your “What to expect from data engineering in 2024”, some predictions, essentially. And you mentioned Coilot being a gateway to AI, and a lot of the incoming stuff. I kind of want to – not so much end, but I think we’re coming close to a version of an end, so I want to get your take on this aspect of AI-enabled workflows, and the long-standing idea of engineers actually engineering. How’s that going to play out? Like “Well, we always prefer writing code”, will AI overflow this thing and turn us into just generative people? I don’t know. What’s the play out there?

Chris Benson: Absolutely. And it also reminded me of something else. I know that a stat that I throw out commonly at talks is that Gartner a while back was predicting that by 2025 half of all of our primary care in terms of medical attention would come through AI assistance. I know when they first put that out I was thinking, “Well, some of it… I don’t know…”, but I think they might be right, and maybe even most of our medical care. Because if you have this ability to have that visual and audio experience that is almost indistinguishable from talking to a human, all of the AI-enabled medical capability that might be behind that - then yeah, I think the way that we are living our lives in terms of seeking primary care and lots of other use cases might really change over the next few years, and we may see a lot of that this coming year.

Ali Spittel: I think it’s really interesting that you talked about serverless, because ui.dev has this really fun, hilarious frontend newsletter, and this week they did predictions for the frontend for 2022, and one of those predictions was that serverless was going to allow more frontend developers to be full stack developers… And I wanted to get your thoughts on that, because we’re talking about this frontend ecosystem, and serverless… Maybe that serverless may be an easier bridge into building backends for frontend developers.

Ro Gupta: Anyway, but to answer your question - yeah, it’s a big, big deal, and that’s why we’re very involved… Like, the trust is the biggest deal, actually. And we try to get really involved in that… For example in shows like this, and really just educating and being super-honest about where we are today, and where we’re going… We’re on the board of the main educational body of the AV industry. It’s called Partners for Automated Vehicle Education (PAVE) I think. So yeah, we’re big believes in that…

And look, I’m not actually a Tesla hater at all. In fact, I actually give them huge credit for whipping much of this industry into gear, including on the EV side… But we are very worried about lasting impacts to trust with stuff like FSD (the so-called Full Self-Driving). You’re seeing a lot of the incidents that are happening, and what’s it gonna take to set us back because of losing trust for training Tesla’s neural net.

So I think the best way to trust is all the things we’re doing, which is transparency and education, but ultimately, it’s making the technology boring. Again, going back to aviation… Aviation is incredibly, amazingly safe, and it’s boring, and no one things about those things really anymore… And it took time. It didn’t happen overnight. I don’t think autonomous driving needs to take as long to really industrialize at scale as aviation did from the Wright brothers. I don’t think it will take that long. But it’s definitely taking longer than 2019-2020 predictions everyone was saying back in CES 2015, of when we’re gonna have robot taxis everywhere.

First off, everyone knows by now it’s way harder than people thought. It’s the classic 90/90 problem; you’re 90% of the way there, but then you realize the last 10% is actually 90% more…

But I think the boring part is really key, because for example when Waymo won, they really have been the first to truly take the driver out in Phoenix… And you know, not just them, but other AV companies we’ve worked with or know really well, they always say that same thing. And by the way, my smoothest AV ride has been when I forgot that autonomous mode was engaged. This particular one was in downtown Detroit, and I got out and I told them “I am so impressed, because in the other AV rides I’d done, I had been thinking about it, I was remembering it, and I remember I kind of zoned out for a second.” So that, whether you’re thinking about level four, the mobility-as-a-service model, or you’re talking about kind of the more Super Cruise, autopilot on a highway - once the user really feels like they’re at the comfort level where either the whole ride is such that it’s kind of mundane and they don’t even have to think about the technology, or in the case of human assistants, where it’s more highway assists (like I said, like Cadillac Super Cruise), where they’re still confident that the system is really good at bringing them back in the loop, that is what it’s gonna take. And the good thing is those are parallel tracks that are happening. They just need to play out in the right way.

The problem with boring, using kind of a Tesla approach that people fear is it’s really hard to square boring with “Oh, but you still need to be in the loop.” It’s really, really hard to do that. It doesn’t work very well.

Gerhard Lazu: Okay. And which version did we install of the provider? Version 0.0.0-12. So there’s no tag for this. This is like a dev-only version. We trust dev in production. The dream is real. Production became dev. Great.

Okay, so we installed it, we configured it, it’s all there. So how can we check that the provider is there? If we maybe get all the pods in the namespace, Crossplane system? Because that’s where everything gets installed. We see that we have Crossplane installed, we have the Crossplane RBAC manager - these are two pods, and the third one is the Jet Linode pod. Cool. So what can we do next? I’m using K9s as the CLI allows me to do things really, really quick. So if we go to look at all the aliases, which is Ctrl+A for me, and I search for cluster, we see a new CRD. And the CRD is in the Linode Jet Crossplane IO v1 Alpha 1 group. That’s how we can provision new clusters. So let’s try that.

If we go to Cluster, to list out clusters, we have no clusters. Great. Let’s do “Make Crossplane LKE.” All this does - I still have to run a command, okay… I know what I mentioned earlier, there will be no commands, but this is a different type of command. I’m not telling the Linode API “Hey, Linode, create me an LKE instance.” I’m telling Crossplane to create on my behalf an LKE instance. And there’s something really cool about this, because Crossplane will continuously reconcile what I ask of it. How cool is that? I think that’s my favorite Crossplane feature, which happens to be a Kubernetes feature as well. You know, declarative, you tell it what you want, and it will make it so. I love that story. Great.

Okay, so this succeeded. What are we seeing now? We are seeing that 42 seconds ago a new LKE 2021, 12, 17 - by the way, it’s the 17th of December when we are recording this… It just uses the current date when this new cluster has been created, or it’s asked to be created.

So if we go to Linode, and if we go to our Kubernetes lists, we see a new cluster which is Kubernetes version 1.22. Nice. I’m wondering, could that be our new production cluster for 2022? If you could see me, I’m winking; yes, it will be. 1.22 Kubernetes will be the first version of our production 2022 Kubernetes cluster. This is it. Because it’s ready, it’s synced, we have the external name, which is the ID, the instance has booted… Great.

Okay… So did it work? Well, let’s try make a Crossplane, LKE kubeconfig… All these, by the way, are in our repo, you can check them out. Actually, do you wanna tell us what happened behind the scenes, like how were we able to do this?

Gerhard Lazu: Yeah. I mean, I do – it’s almost like a generation. So for example, our Fly app, the one that is currently running in production, is 2022, 03, 13. And guess how I remember it? I just remember; it just sticks with you. And the next one, the one that we’re currently experimenting with, is 2023, 12, 17. So 17th of December, this is the new generation of the Changelog app.

Jessica Kerr: Right. I don’t like working on greenfield projects for that reason. I don’t wanna work on anything that’s not in production… Which is why back in May of 2020, when Avdi and I were like “Let’s make a Rails app”, step one is get it online. Get it up at a domain.

Gerhard Lazu: Yeah, and seeing what happens. Anyways, 399 had to happen first. We wanted to provision a Kubernetes cluster with Crossplane, and we did that. That is actually our next year’s - actually, this year’s, 2022 - cluster, which is what we tested to see how this would work before taking it into production. So we are still running on the 2021 LKE cluster. We’re in the process of migrating to 2022, and we want to test this new integration out with S3 in something which looks like production, and is as close to production as possible, but it’s not our actual production. So if there’s any mistakes, we can go back. It’s a nice, safe, low-risk, low-stress way of trying things out and seeing if they work. And in our case, it did.

Gerhard Lazu: We are in Chicago at KubeCon 2023, talking with two people that I bet my production Kubernetes on, Steve and Spencer. Welcome.

Daniel Whitenack: Welcome to another episode of Practical AI. This is Daniel Whitenack. I am the founder and CEO at Prediction Guard, and I am joined today to talk about a very interesting report that we’ve talked about on the show today, the AI Index report from 2024, from the Stanford University Human Centered AI Center. I’m joined by Nestor Maslej, who is the research manager at the Stanford Institute for Human Centered AI. Welcome, Nestor.

Chris Benson: It has. And for what it’s worth, I’m gonna brag on you for a second, since I know that you would not do that to yourself… With Daniel being the founder of Prediction Guard, this is a topic that he is a global expert in; really, really knows what he’s doing… And as we were talking about – I’ve had so many people asking me these questions that Daniel was just talking about lately, and I was like “Well, one of my best friends is a real pro at this…” So thank you – if you can kind of start walking us through… And this is a moving topic, as you just pointed out; it has changed in the last few months, and will continue to evolve over time… But yeah, if you can start walking us through what that looks like today. We’re in the beginning of the fall of 2023; something that might help the rest of us for at least the next few months.

Kelsey Hightower: That’s why I said the prediction for System Initiative is they’re going to have to go beyond, because I just don’t think you can do that in 2023. People have their favorite languages…

Chris Benson: [00:05:59.23] So I don’t think you’ll be surprised when I say this, and we’ve alluded to this on some previous episodes… But it is a difficult business concern to navigate. I know all of us who straddle into the AI technical realm are incredibly excited, we’re trying to figure out how to do the models, and put them out there, and everything like that… But if you are not in our shoes, if you’re walking on a slightly different path, and let’s say you work for a legal department, or a compliance department, or other business concerns, and suddenly these technologies are coming at you hard and fast, week by week in 2023, and you’re trying to navigate that, and look at things like licensing on how the data that goes into models is used, and you’re looking at compliance concerns, and you’re looking at protecting your intellectual property… There’s a whole host of challenging business problems with essentially no guidance. This is a brave new world that has to be pioneered through, and so I have talked to a lot of businesspeople in various roles, including attorneys, and this stuff is scary stuff. It is problematic stuff, it is challenging to navigate. And I definitely want to take you down the path today of talking about the space, and Prediction Guard, relative to how you actually get these models out there in a productive way, in a business environment, so that people can take advantage of the technology, and understand what the pitfalls are, and such.

So that’s the big thing that I’ve been hearing… I’ve been getting an earful of it lately, like “Chris, settle down. Stop taking us down this AI thing. We’ve got to figure some things out first”, so I’m coming to you for answers, man.

Beth Rudden: So I was working at IBM, and I was a pleaser, so I went after the information architect certification. But I had a couple of friends that were mathematicians and statisticians, and really data engineers, and DevOps people, and software engineers… And they didn’t really want to be an information architect, they didn’t want to be an IT architect, so we rinsed and repeated experiential certification to be able to say, “Well, how can we make sure that we’re making data scientists that actually know how to use data in the scientific method to solve business problems?” And so we started that, and it took us about six, seven years, it’s accredited through Open Group, anybody can get it… And it’s hard, because you actually have to submit - depending on your level, one, two or three, you have to submit several projects that say “Here’s how I took my problem and put it into a hypothesis that could be tested, and then here’s how I negotiated with my business stakeholder, and then here’s how I showed my results”, and then as you get further and further along, “here’s how I integrated my model into production.” And I think that’s when things get a little crazy, and people are “Wait, what? How do I do that? Here’s my Jupyter Notebook. Isn’t that great?”

But I’ve been writing, or looking at how to deliver AI, but I’ve been doing a lot more on the linguistics or the semantic side, and for probably about 15-20 years. And if you look at how the NLP work is really – you know, a lot of people are like “Oh, hey, I pulled down this thing from spaCy. I can write NLP, right? I just call this a cosine similarity kind of model. I’m good to go.” And I was an archaeologist in the late ‘90s; that’s what I actually got my degree in. I did Greek and Latin, and spent some time in Italy… And when you’re learning languages, you’re learning declensions, and you’re learning etymology, and you’re learning stemming and lemmatization and tokenization and all of the text pre-processing, so I was always the squishy human data scientist. I was the one that was studying languages and doing semantics. So I think it was 2015 Andrew Yang was like “Oh hey, if we use GPUs or graphical processing units, we can process all this structured data really, really fast against these statistical models.” And so a lot of people, I think, forgot about entity extraction, and ontologies, and the Semantic Web… And I use the OWL, which are formal knowledge graphs, and I hopefully am not speaking Greek to a lot of people, but really looking at the language side, as opposed to the machine learning side.

And that understanding of semantics has really put me in a great position now, because all of the statistical models… NLP has three – I kind of chunk them into three things. NLU, natural language understanding, NLC, which is classification, which is your prediction, and then NLG, which is your generation. And prior to having access to GPT, we were generating language old-school. And it’s super-hard. I mean, it’s really, really hard. If you think of like pronouns, and trying to say “John was the guy who we’ll refer to him, and then he took the notch or the wrench from Joe” etc, etc. I mean, it’s just so complicated.

[06:08] But now that we have a really good natural language generator, I’m kicking butt, because I have the semantic side, so I have really, really high NLU, or natural language understanding… Because how an ontology works - and I try to remind people of this; ontology is the study of the nature of your reality based on the language that you use. So when you use language, I’m understanding your reality. And so I map that into a knowledge graph, and then take that language to understand whatever you’re going to say against it. So I have a map. So when the natural language generator generates all the variation that I need in order to put into my machine learning models, and then I have a map to make sure that I understand what kind of things you actually want to talk about, I can create conversational AI that has lineage and provenance, that has the source of where you’re starting from, and then I use all of the GPT generators to effectively generate the fluff or the syntax around whatever entities that I extract about whatever you want to talk about. If you guys could help me break that down into better English… That’s what I’ve been trying to do for the last couple of months.