Practical AI

Practical AI Artwork

Making artificial intelligence practical, productive & accessible to everyone

Practical AI Practical AI #238

Fine-tuning vs RAG

Play
2023-09-06T12:30:00Z #ai +1 🎧 39,132

In this episode we welcome back our good friend Demetrios from the MLOps Community to discuss fine-tuning vs. retrieval augmented generation. Along the way, we also chat about OpenAI Enterprise, results from the MLOps Community LLM survey, and the orchestration and evaluation of generative AI workloads.

Practical AI Practical AI #234

Vector databases (beyond the hype)

Play
2023-08-01T18:30:00Z #ai +1 🎧 38,299

There’s so much talk (and hype) these days about vector databases. We thought it would be timely and practical to have someone on the show that has been hands on with the various options and actually tried to build applications leveraging vector search. Prashanth Rao is a real practitioner that has spent and huge amount of time exploring the expanding set of vector database offerings. After introducing vector database and giving us a mental model of how they fit in with other datastores, Prashanth digs into the trade offs as related to indices, hosting options, embedding vs. query optimization, and more.

Practical AI Practical AI #228

From ML to AI to Generative AI

Play
2023-06-21T13:30:00Z #ai +2 🎧 36,886

Chris and Daniel take a step back to look at how generative AI fits into the wider landscape of ML/AI and data science. They talk through the differences in how one approaches “traditional” supervised learning and how practitioners are approaching generative AI based solutions (such as those using Midjourney or GPT family models). Finally, they talk through the risk and compliance implications of generative AI, which was in the news this week in the EU.

Practical AI Practical AI #219

Capabilities of LLMs 🤯

Play
2023-04-19T21:00:00Z #ai +1 🎧 35,865

Large Language Model (LLM) capabilities have reached new heights and are nothing short of mind-blowing! However, with so many advancements happening at once, it can be overwhelming to keep up with all the latest developments. To help us navigate through this complex terrain, we’ve invited Raj - one of the most adept at explaining State-of-the-Art (SOTA) AI in practical terms - to join us on the podcast.

Raj discusses several intriguing topics such as in-context learning, reasoning, LLM options, and related tooling. But that’s not all! We also hear from Raj about the rapidly growing data science and AI community on TikTok.

Practical AI Practical AI #240

Generative models: exploration to deployment

Play
2023-10-03T19:00:00Z #ai +2 🎧 35,853

What is the model lifecycle like for experimenting with and then deploying generative AI models? Although there are some similarities, this lifecycle differs somewhat from previous data science practices in that models are typically not trained from scratch (or even fine-tuned). Chris and Daniel give a high level overview in this effort and discuss model optimization and serving.

Practical AI Practical AI #236

The new AI app stack

Play
2023-08-23T12:00:00Z #ai +2 🎧 35,415

Recently a16z released a diagram showing the “Emerging Architectures for LLM Applications.” In this episode, we expand on things covered in that diagram to a more general mental model for the new AI app stack. We cover a variety of things from model “middleware” for caching and control to app orchestration.

Practical AI Practical AI #272

Rise of the AI PC & local LLMs

Play
2024-06-04T18:45:00Z #ai +2 🎧 34,109

We’ve seen a rise in interest recently and a number of major announcements related to local LLMs and AI PCs. NVIDIA, Apple, and Intel are getting into this along with models like the Phi family from Microsoft. In this episode, we dig into local AI tooling, frameworks, and optimizations to help you navigate this AI niche, and we talk about how this might impact AI adoption in the longer term.

Practical AI Practical AI #237

Automating code optimization with LLMs

Play
2023-08-29T21:30:00Z #ai +1 🎧 33,602

You might have heard a lot about code generation tools using AI, but could LLMs and generative AI make our existing code better? In this episode, we sit down with Mike from TurinTech to hear about practical code optimizations using AI “translation” of slow to fast code. We learn about their process for accomplishing this task along with impressive results when automated code optimization is run on existing open source projects.

Practical AI Practical AI #150

From notebooks to Netflix scale with Metaflow

Play
2021-09-21T14:45:00Z #ai +2 🎧 33,602

As you start developing an AI/ML based solution, you quickly figure out that you need to run workflows. Not only that, you might need to run those workflows across various kinds of infrastructure (including GPUs) at scale. Ville Tuulos developed Metaflow while working at Netflix to help data scientists scale their work. In this episode, Ville tells us a bit more about Metaflow, his new book on data science infrastructure, and his approach to helping scale ML/AI work.

Practical AI Practical AI #235

Blueprint for an AI Bill of Rights

Play
2023-08-09T16:20:00Z #ai +3 🎧 32,948

In this Fully Connected episode, Daniel and Chris kick it off by noting that Stability AI released their SDXL 1.0 LLM! They discuss its virtues, and then dive into a discussion regarding how the United States, European Union, and other entities are approaching governance of AI through new laws and legal frameworks. In particular, they review the White House’s approach, noting the potential for unexpected consequences.

Practical AI Practical AI #251

AI predictions for 2024

Play
2024-01-10T19:30:00Z #ai +1 🎧 32,782

We scoured the internet to find all the AI related predictions for 2024 (at least from people that might know what they are talking about), and, in this episode, we talk about some of the common themes. We also take a moment to look back at 2023 commenting with some distance on a crazy AI year.

Practical AI Practical AI #285

AI is more than GenAI

Play
2024-09-05T14:00:00Z #ai +2 🎧 32,296

GenAI is often what people think of when someone mentions AI. However, AI is much more. In this episode, Daniel breaks down a history of developments in data science, machine learning, AI, and GenAI in this episode to give listeners a better mental model. Don’t miss this one if you are wanting to understand the AI ecosystem holistically and how models, embeddings, data, prompts, etc. all fit together.

Practical AI Practical AI #232

Legal consequences of generated content

Play
2023-07-18T18:30:00Z #ai +2 🎧 31,705

As a technologist, coder, and lawyer, few people are better equipped to discuss the legal and practical consequences of generative AI than Damien Riehl. He demonstrated this a couple years ago by generating, writing to disk, and then releasing every possible musical melody. Damien joins us to answer our many questions about generated content, copyright, dataset licensing/usage, and the future of knowledge work.

Practical AI Practical AI #233

There's a new Llama in town

Play
2023-07-25T19:00:00Z #ai 🎧 31,606

It was an amazing week in AI news. Among other things, there is a new NeRF and a new Llama in town!!! Zip-NeRF can create some amazing 3D scenes based on 2D images, and Llama 2 from Meta promises to change the LLM landscape. Chris and Daniel dive into these and they compare some of the recently released OpenAI functionality to Anthropic’s Claude 2.

Practical AI Practical AI #280

Broccoli AI at its best 🥦

Play
2024-07-31T21:40:00Z #ai +1 🎧 31,263

We discussed “🥦 Broccoli AI” a couple weeks ago, which is the kind of AI that is actually good/healthy for a real world business. Bengsoon Chuah, a data scientist working in the energy sector, joins us to discuss developing and deploying NLP pipelines in that environment. We talk about good/healthy ways of introducing AI in a company that uses on-prem infrastructure, has few data science professionals, and operates in high risk environments.

Practical AI Practical AI #261

Prompting the future

Play
2024-03-20T13:45:00Z #ai +2 🎧 31,062

Daniel & Chris explore the state of the art in prompt engineering with Jared Zoneraich, the founder of PromptLayer. PromptLayer is the first platform built specifically for prompt engineering. It can visually manage prompts, evaluate models, log LLM requests, search usage history, and help your organization collaborate as a team. Jared provides expert guidance in how to be implement prompt engineering, but also illustrates how we got here, and where we’re likely to go next.

Practical AI Practical AI #221

Large models on CPUs

Play
2023-05-02T16:00:00Z #ai +1 🎧 30,780

Model sizes are crazy these days with billions and billions of parameters. As Mark Kurtz explains in this episode, this makes inference slow and expensive despite the fact that up to 90%+ of the parameters don’t influence the outputs at all.

Mark helps us understand all of the practicalities and progress that is being made in model optimization and CPU inference, including the increasing opportunities to run LLMs and other Generative AI models on commodity hardware.

Practical AI Practical AI #256

Gemini vs OpenAI

Play
2024-02-14T20:00:00Z #ai +2 🎧 30,723

Google has been releasing a ton of new GenAI functionality under the name “Gemini”, and they’ve officially rebranded Bard as Gemini. We take some time to talk through Gemini compared with offerings from OpenAI, Anthropic, Cohere, etc.

We also discuss the recent FCC decision to ban the use of AI voices in robocalls and what the decision might mean for government involvement in AI in 2024.

Practical AI Practical AI #282

Only as good as the data

Play
2024-08-14T21:15:00Z #ai +1 🎧 30,490

You might have heard that “AI is only as good as the data.” What does that mean and what data are we talking about? Chris and Daniel dig into that topic in the episode exploring the categories of data that you might encounter working in AI (for training, testing, fine-tuning, benchmarks, etc.). They also discuss the latest developments in AI regulation with the EU’s AI Act coming into force.

Practical AI Practical AI #249

The state of open source AI

Play
2023-12-12T19:45:00Z #oss +1
🎧 30,471

The new open source AI book from PremAI starts with “As a data scientist/ML engineer/developer with a 9 to 5 job, it’s difficult to keep track of all the innovations.” We couldn’t agree more, and we are so happy that this week’s guest Casper (among other contributors) have created this resource for practitioners.

During the episode, we cover the key categories to think about as you try to navigate the open source AI ecosystem, and Casper gives his thoughts on fine-tuning, vector DBs & more.

Practical AI Practical AI #288

GraphRAG (beyond the hype)

Play
2024-09-25T18:30:00Z #ai +1 🎧 30,396

Seems like we are hearing a lot about GraphRAG these days, but there are lots of questions: what is it, is it hype, what is practical? One of our all time favorite podcast friends, Prashanth Rao, joins us to dig into this topic beyond the hype. Prashanth gives us a bit of background and practical use cases for GraphRAG and graph data.

Practical AI Practical AI #230

Cambrian explosion of generative models

Play
2023-07-06T17:30:00Z #ai +3 🎧 30,255

In this Fully Connected episode, Daniel and Chris explore recent highlights from the current model proliferation wave sweeping the world - including Stable Diffusion XL, OpenChat, Zeroscope XL, and Salesforce XGen. They note the rapid rise of open models, and speculate that just as in open source software, open models will dominate the future. Such rapid advancement creates its own problems though, so they finish by itemizing concerns such as cybersecurity, workflow productivity, and impact on human culture.

Practical AI Practical AI #222

The last mile of AI app development

Play
2023-05-11T13:00:00Z #ai +3 🎧 30,183

There are a ton of problems around building LLM apps in production and the last mile of that problem. Travis Fischer, builder of open AI projects like @ChatGPTBot, joins us to talk through these problems (and how to overcome them). He helps us understand the hierarchy of complexity from simple prompting to augmentation, agents, and fine-tuning. Along the way we discuss the frontend developer community that is rapidly adopting AI technology via Typescript (not Python).

Practical AI Practical AI #284

Metrics Driven Development

Play
2024-08-29T20:45:00Z #ai +1 🎧 30,154

How do you systematically measure, optimize, and improve the performance of LLM applications (like those powered by RAG or tool use)? Ragas is an open source effort that has been trying to answer this question comprehensively, and they are promoting a “Metrics Driven Development” approach. Shahul from Ragas joins us to discuss Ragas in this episode, and we dig into specific metrics, the difference between benchmarking models and evaluating LLM apps, generating synthetic test data and more.

Practical AI Practical AI #242

Deep learning in Rust with Burn 🔥

Play
2023-10-24T20:40:00Z #ai +2 🎧 29,963

It seems like everyone is interested in Rust these days. Even the most popular Python linter, Ruff, isn’t written in Python! It’s written in Rust. But what is the state of training or inferencing deep learning models in Rust? In this episode, we are joined by Nathaniel Simard, the creator burn. We discuss Rust in general, the need to have support for AI in multiple languages, and the current state of doing “AI things” in Rust.

Practical AI Practical AI #243

Self-hosting & scaling models

Play
2023-10-31T18:00:00Z #ai +2 🎧 29,949

We’re excited to have Tuhin join us on the show once again to talk about self-hosting open access models. Tuhin’s company Baseten specializes in model deployment and monitoring at any scale, and it was a privilege to talk with him about the trends he is seeing in both tooling and usage of open access models. We were able to touch on the common use cases for integrating self-hosted models and how the boom in generative AI has influenced that ecosystem.

Player art
  0:00 / 0:00