The perplexities of information retrieval

Welcome to another episode of Practical AI. This is Daniel Whitenack. I am founder and CEO at Prediction Guard, where we’re safeguarding private AI models, and I’m joined as always by my co-host, Chris Benson, who is a principal AI research engineer at Lockheed Martin. How are you doing, Chris?

Doing great today, Daniel. How’s it going?

It’s going great. I am sometimes perplexed throughout my work day, but generally the week has gone well, and I’m really excited, because hopefully our guest today will provide a lot of ways for us to navigate through the Perplexity.

I hope he finds that punny.

Yes, yes, of course. We have with us today, Dennis Yarats, who is the CTO and co-founder at Perplexity. Welcome!

Thanks for inviting me, and great to be on the show.

Yeah, yeah. Well, we’ve been wanting to make this one happen for a good, long while… Of course, we’ve been following the things that Perplexity has been doing, really impressive and inspiring work… And yeah, just really excited to hear a little bit about that story, and also this space of answering, giving people knowledge with generative AI, with language models, search, more generally, maybe… Could you set us up by maybe talking through – of course, people have followed, everyone’s followed this surge of generative AI over the last couple of years… But there’s been a segment of that that has focused a lot on answering questions, discovering knowledge, curiosity, exploring topics, and intersection with search, more generally, I guess. So could you help us understand maybe that journey, how that’s developed, and how also you and the co-founders of Perplexity came upon your approach to that?

Yeah, that’s definitely been very fascinating almost two years, I guess. I think in general web search – so what do you want to ultimately get as an answer to your question, right? So the current iteration that we have with Google and all the other classical search engines is an approximation of to get to this point, right? So you your question, you ask it as a formal query, and then you get a bunch of documents that are very relevant, but you have to still do additional work to get to the bottom of it. So you have to scan through the documents, you have to for yourself understand what “Is this a true answer? A not true answer?”, you have to trust the information. And as you can imagine, it’s a lot of work, especially if you’re trying to search for something very complicated, maybe things that are not so obvious.

And wouldn’t it be nice to avoid that step, where you ask the question and you get an answer right away. So that’s the ultimate destination where we’re trying to get. Obviously, it’s been tough to get there over the last decade or so, there has been a lot of work, but it never quite worked. There is this much higher level of hallucinations, much higher level of maybe not perfect synthesis of the information… You basically get a Frankenstein… So instead of a coherent and nice, easily parsable and readable answer, you get some just basically extracted pieces of the information and just concatenated together, so not very pleasant. And it’s funny that when we started – so one of our angel investor was Jeff Dean; he requires no introduction. And he was saying Google actually wanted to always build something this, but because they had such high expectations for accuracy, because millions and billions of users are using Google… And if you hallucinate 1% of the time, you’re gonna get a lot of unhappy people. And so they were never able to – because the models were not as strong as they are right now, they were never able to get to just 99.9% of accuracy. And that’s why like this work never panned out.

[00:06:20.00] But something great happened in 2022. When we started our company, both myself and Aravind, my co-founder, we come from academics, so we’ve been doing a lot of research in language modeling, reinforcement learning and stuff like that… And he was actually at Open AI at that time. We’ve been very literally following improvements of GPT models, GPT 2 and then GPT 3; that’s where it actually got very interesting. And it became obvious there was going to be something there. And this was primarily the motivation for us to start a company.

We wanted to build an answer engine from the get go, but it was very ambitious. I remember we would go to the investors and say “Oh, we’re going to build a search engine”, and they’re looking at you like you’re crazy, which makes a lot of sense. They’re just like “Oh, there’s Google ready.” And they had a fair point. But we still weren’t very discouraged by that. We knew there is something there, and we started prototyping.

So the first version of Perplexity we actually created as a slug board, or Discord bot, where it was a very primitive combination of a search engine plus - at that point it was DaVinci 2 models; it was still pre-ChatGPT. And it worked much better than I expected. It was very quick; we put up this demo in a couple of days. And you could already see that in certain cases it is very helpful.

Because this was a very early company, we were trying to hire one of our first engineers, and we didn’t know how to organize the insurance for him… And so we actually used this bot to ask those questions about insurance. Because if you go to Google and start asking the questions about insurance, you’re gonna get a lot of ads, and you’re just gonna get very quickly disappointed.

I’ve appreciated that same thing in founding a company and using these models in that way… It’s very useful.

Yeah, exactly. And then it was useful, so we’ve been trying it out and playing with it… But then what really started happening is - I think it was a couple of weeks before ChatGPT - Open AI released this DaVinci 3 model. And I literally remember very clearly - I changed literally the model name and started asking the same questions that I would ask it before, and I could see right away it was so much better. It was just so much better at understanding your intent, so much better at understanding what should it say, what it should – not say; so much better at synthesizing the answer. And I was just really blown away. I was like “Okay, so there is definitely something there.”

And then obviously, ChatGPT happens a few weeks after, and we’re like “Okay, so we have two–” Our initial product was actually – because as I mentioned, we came from an academic background… Our citations were the core component. Because it was very clear to us from the beginning - like, if you want to get an answer, you want to make sure it’s accurate. You want to make sure you can verify it.

And so the citation was this sort of first class citizens in our product. And then when ChatGPT came out, one of the biggest points of feedback for them was “Okay, so I don’t know if this is accurate information or if it’s a hallucination. If it’s not, how would I verify it?” And that’s why we were like “Okay, so this seems like a good opportunity to release our product.” We literally in a matter of two days put up a website, connected to our backend that we had, and just – we obviously did not expect that people were going to use it and the usage was going to grow as much as possible…

[00:10:08.03] But coming back to your original question, I think what happened - literally in a matter of days or a month… I mean, obviously, it follows many years of research, but it was a very clear step function in the quality of the generated answers. And you can literally – if you sort of spend some time playing with it, you can clearly see that it now becomes very good. We also realized at that time “Okay, so models are only gonna get better. Things are only going to get faster and cheaper, so there’s something. There’s a lot of stuff to build here.”

For those who are still learning about your organization and what you’re offering, could you step back for a second and if you were talking to Jeff Dean or another investor, kind of giving them the elevator pitch about what you’re doing specifically, and how it’s differentiated from the GPTs and stuff out there… How do you define that? What how do you describe yourself in terms of the specific opportunity that you’re pursuing?

The fastest way to get the most accurate answers to your questions. Essentially an answer engine. We’re one of the first people who coined this term.

And can you differentiate that from a search engine a little bit there?

Yeah. So basically, in a search engine you get all the way; as I mentioned before, in the search engine you have to search first, you get the documents, let’s say you have top 10 results, and you have to scan through all of them and identify the information to get your answer. Here, we do this step for you. So we take the first step of the retrieval of the relevant documents, and then we synthesize them into a human-readable, nicely-formatted answer that you can then, if needed, if you want to get more information, then you can click on the citations that are nicely attributed for each sentence, and then you can learn more information.

We wanted to do things very simple. Early on we identified “There are two things we care about: accuracy and speed.” Because you want to get information fast. Google trained us all to get instant results, search results… I think that was very important, and early on it was challenging, because those models were very slow, our infrastructure was not very advanced to do that… I remember the very first version you would have to wait for three seconds or five seconds to get an answer. It was very slow. But because it was such a better experience than just looking at the search results of Google, so people still would use it… But our ultimate goal was “Okay, so can we be as fast as possible?”

So the main differentiator is just we care a lot about quality, so we minimize the chance of things being inaccurate, or hallucinate, and we want to do it as fast as possible. And so that distinguishes us from Google, because Google doesn’t, for example, generate the answers… Even though more recently they started doing this, which has kind of just validated our idea… And ChatGPT probably primarily focuses on different things. But I guess also more recently they’ve started doing web search as well.

Break: [00:13:18.23]

So you mentioned a few things there… You mentioned web search, you mentioned retrieval, you mentioned the large language model… So at least in how I think about it - and maybe others categorize it differently - there’s one element of information that you can get from an LLM, which is “I’m going to put in a prompt, and it’s going to generate text, and that may contain some facts, or made up facts, or some text, but it may be informational.” So there’s some sort of knowledge that can be gained there. And then in a second case, there’s a way to retrieve on the fly external data. So that could be from your company’s documents, it could be from the web, whatever, and then inject that into prompts into the model, which kind of grounds it, and like you say, would give you a citation.

There’s also more agentic approaches to this, where maybe there’s multiple ways that you could get knowledge, maybe doing a web search, or searching a certain database that you have access to, or any handful of sources that you’ve curated as tools, and you call them in a more automated way… So I’m wondering, from your perspective, obviously, you are part of a team that have been exploring this very deeply, from very early on, like you say, when these models made that jump… From your perspective now, both in terms of what you’re building with Perplexity and also how you generally see the ability to get accurate information from these models, how do you view those categories, in terms of their utility and what you’re relying on, for the accuracy elements specifically?

The way I see it’s going to unfold, I think the tools and the agentic behaviors - I think that’s where it’s going. I think it’s going to be the main bottleneck for this right now; it’s just models are not smart enough yet to take into account and sort of reason all of the information that is out there… But I think it’s going to be a main component. So there’s going to be models – they’re very powerful already right now. They’re trained on a lot of data; basically, internet. They have a lot of internal information, internal knowledge. And they can do already a very good job of synthesizing information. There’s certain things that they don’t do well, and perhaps they’re never going to do well those things. For example like computation, when you need to maybe run some code, or do some sophisticated math computations… The LLM architecture as transformers is going to struggle at that.

[00:18:26.00] Also, because those models are so big, sometimes it’s going to be very expensive to then update very frequently. So you need a way to ingest something, some new information that just happened and it’s still not part of the LLM weights. And this is where we’re also specialized. Also some private documents, as you mentioned. Sometimes if it’s like enterprise, you have some of the documents that obviously the model was not trained on, and you maybe want to reason about those documents. And there’s all kinds of other tools that you can – yeah, eventually there is going to be agents that are going to do actions, like maybe book a ticket, buy a ticket or something like that, and stuff like that.

So I think definitely where it’s going, it’s going to be a synergy; everything’s gonna come together. We just need a top-level powerful model that’s gonna reason behind multiple things, and we’ll have to have long context windows, maybe some memory as well… And then just utilize those tools as much as possible.

How we at Perplexity are thinking about it, we’re like – and there’s multiple ways, and multiple sort of applications, multiple use cases of this general approach. We’re primarily focusing on the information retrieval part of it. So initially, web is the main component, because there’s lots of things to extract from wed, but also we’re thinking about how to integrate different data sources; maybe some of the different databases, maybe there’s some more complicated or more specialized documents… Maybe there’s PDFs, or something like that, or some financial data, things like that.

The other aspect that I’m very excited about them and we’re working on is kind of do – right now we can do a great job of answering complicated questions that you can get answers on Google, but still not something where you can ask experts, and then can get an answer. What if I have a question that just requires multi-step of reasoning, so it requires searching the web multiple times, analyzing information during each retrieval, and then maybe refining the search… So those things are maybe going to take some time. But if you do it yourself, you would spend let’s say like an hour. So here maybe the system would spend 30 seconds, and it’s gonna save you a bunch of time. So it’s kind of those use cases that can answer very complicated questions. And we believe that there is a world for those types of questions; technology like this is going to be useful.

As people are using an answer engine like yours more and more often going forward, and you’ve alluded a moment ago to the fact that LLMs are not the be-all; there are things they don’t do well, like mathematics and such, and a variety of other things I’m sure that we can all throw out there… But they’re really powerful at what they do. But clearly, there is a place and a need for both the LLMs, these largest models that get all the – they kind of soak up all the air in the news cycles, as well as many smaller models that are specialized, a mathematics model that you plug in… As we’re looking at trying to use answer engines to retrieve information and that information is increasingly multimodal in nature in terms of what you’re asking, how does the architectures of those come together? This is a space – it’s not the first time we’ve asked it here, but it’s evolving so rapidly; we’re no longer hosting a model, you’re now hosting a whole collection, and they may be mixed with models that your API-ing out to, and such… How does that look to you as you’re building this company at this point?

[00:22:11.12] It’s going to always be a trade-off between – if you have one powerful model, I mean, yes, ít can do lots of general things super-well, but it’s going to be slower, it’s going to be more expensive… One of our key principles is just we want to do things very fast, so we can get answers as fast as possible. That means you have to design your orchestration system in a way where certain things will have to rely on customized models. And something that is much smaller, much faster, but it’s a specialist model. So it’s not a model that knows how to do everything, but it knows how to do one task. And basically the challenge here is how do you balance between these general models and these specialist models? And I think we’ve been doing this from the very beginning. So when you send a request to Perplexity, it’s not just one model; there’s at least ten different models trying to do lots of things with your request. It’s all kinds of ranking models, a bunch of embeddings, all different classifiers, and stuff like that.

And the other trade-off here is with a general model - let’s see, one of the big, and I think it was actually a very critical component of why a company like Perplexity in the first place became possible, it’s the speed of iteration. You literally can change the prompt, and you can just get a new product in a matter of hours. Imagine a couple of years ago, if you wanted to build something like Perplexity or another gen AI product, you’d have to collect data first, you’d have to train the model, launch the product, see if this product makes sense, does it have market fit or not? If it has market fit, then you’d start collecting data, and then you’d just keep improving. So what’s been possible with GPT models and APIs - this just kind of flipped over. So you very quickly can build a product, see if there is any signs of life in this product, and then you start collecting data, which is I think honestly the most important thing. And once you collect data, you can distill it, you can build many other smaller models, and optimize the experience so you can make the models faster, you can specialize them. I think this is the key, I think this is the – honestly, it was one of the most fundamental changes in the development. And we took advantage of this theme early on, and then still using…

But it’s still tricky… Imagine every time you have these specialized models, and if you have tons of them, you have to then treat each model, care about each model separately. So if you want to retrain this model, you have to spend some time in it, you have to evaluate it… So it becomes more difficult to manage. But on the other hand, you have some great benefits.

So the key is just you don’t want to go too overboard with those models, like everything is on customized models, but you also clearly don’t want to have just one model that’s going to do everything.

Yeah, I have a question maybe related to that, which I think is a pain point a lot of people are feeling, and I’m guessing your teams have felt, which you even mentioned this… You can make a small change in your prompt or create a new prompt, and all the sudden it’s almost like you have a new product… Which is sort of amazing in one sense, and really frustrating in another sense… Because as you were just alluding to, it’s like, oh, maybe I have these 17 different things chained together, and they all have prompts that I’ve worked really hard on… And then tomorrow LLaMA 17 comes out or something, and now it has a different character of behavior than the previous model that I was using… I’d love to use it, but now I have all of this – it’s almost like AI model debt that I’ve got in my system… Do you have any perspective on that, or anything that sort of has happened in your experience in this regard?

[00:26:21.23] Yeah, this is clearly been a theme, and it’s been happening quite often… And if I would guess, that’s going to continue happening. One thing we realized early on is, okay, so this is going to be the case; there is not going to be one model that rules them all. I mean, even though for some time it was GPT 4, but now we can see there’s particularly like Anthropic, Gemini, LLaMA… There is going to be a future where there’s several frontier models. Because of that, we decided, okay, so let’s design our infrastructure and our system in such a way that it’s going to be model-agnostic. That means there’s ways where you can evaluate each component independently, there is a way where you can quickly change things up to adapt for a new model, and stuff like that. It took some time to get there, but I feel like it was a very correct decision for us. So for example, one of the advantages we have over let’s say things like ChatGPT, or Claude, or one-model providers companies is just we can seamlessly integrate many different models. Our users can decide they want to use this model, or they want to do that model. Later on, as we progress, I think we can even decide based on the complexity of the query, or the type of the query, we can route to a particular model that does the better job for those type of queries, and minimize – maybe some of the queries that are super-simple, you don’t need to run a very large model for that type of answer. So then you can optimize speed, and things like that. So I feel like you have to just make a system in a way where it’s agnostic to the model.

Break: [00:28:11.03]

So to follow up on what we were talking about before the break there, I know that you were talking about really building around model agnosticism, to be able to handle that… I couldn’t help but wonder, occasionally, as we get a new model out, it breaks new ground on modality, being added in a whole new approach, that kind of thing… And so how do you as a business builder who is having to try to accommodate all these different models, when you have one that jumps out and has a completely new thing added in, that was unexpected prior to the announcement, in the organization how do you guys pivot to accommodate that, and keep the agnosticism, and yet provide that extra functionality that’s now available? How do you all tackle that problem?

The most important thing is to not be caught off guard, and try to anticipate what’s going to happen. I think that’s very important. And for the most part, I wouldn’t say it was too hard to predict what’s going to happen… But after that, I think it’s primarily product decision. So does this new feature benefit your product or not? Do we want to build something in product or not? For example, one great example was image uploader, and multimodality. So it was last year that we knew for sure that this was going to happen at some point. There was already some smaller models that were supporting that; we knew that a much better model is going to come out.

And then because of that, we in advance started – first of all, we understood “Okay, so this is going to be important in our project, so we can support this, and this, and that, those use cases.” And we decided to build infrastructure in advance, and anticipate it. Obviously, we didn’t fully predict how exactly it’s gonna operate, but it was very close. It required literally a couple of days to adjust. And then we were one of the – we very quickly can release it as a product.

So having the system that [unintelligible 00:32:03.29] update can support those new modalities, it’s very important. Get some great deal of anticipation. But also sometimes certain features that maybe come out, maybe they’re not useful for your product. You also don’t want to put everything into “Okay, so this is a cool feature. Let me just add it to your product.” That’s always not that great an idea in general. So only things that make sense. And if those things make sense for your product, you likely already thought about how would you implement them in advance, so it just makes it a little bit easier.

While you were talking, I was thinking there’s sort of one axis that you have to navigate here around model releases, and functionality, and modalities, all that stuff… There’s sort of another maybe around UI and user experience. There were multiple people making the comment “Oh, well, the chat interface came out with ChatGPT”, so everyone’s sort of focused around the chat interface. Is that the best way to utilize this sort of technology in the long run? There’s probably a lot of exploration that’s still open around UI, and user experience with this type of technology. And certainly chat is relevant, and we’re using it a lot already… I’m wondering, from your perspective, especially as we see this functionality maybe more embedded in the physical world around that, whether that be in our glasses, with Meta glasses, or in kiosks, in airports or whatever those things are, what is your perspective on how important it is to explore new types of UI or user experience with this technology?

[00:33:55.28] I believe and – I mean we were very confident early on chat interfaces is a temporary thing. It’s just too limiting. It has a lot of constraints. And that’s why we didn’t follow the usual road of all of the chatbots. They literally copied ChatGPT and put like a chat interface. We thought a little bit more about this, and we decided – I feel ultimately, right now we’re still in this early stage where people care about the model itself, so the model is the theme… But as this thing gets more advanced, as more people start using gen AI products, I feel like the main thing is going to be the product itself. What kind of things can the product do? Do you do it better than this? Do you have the best UI? Do you have the best UX? And that’s why we early on we were thinking about those things, and designed our product in a way that is the most suitable for the things that we wanted to do. If it’s search, we knew that chat doesn’t make sense for search. It’s just like, that’s not how people search for information. That was a very big factor, I think, in our success.

The other thing I guess we also – even last year, we started prototyping and experimenting with this concept of generative UI… So something where the LLM can guide what UI elements you can generate… Sometimes one of the things in a chat interface, if you want to ask a follow-up question, sometimes it doesn’t make sense to ask it as a sentence. So maybe you want to show a checkbox, or a button, or whatever. Especially on mobile - everybody uses phones - it’s just not very convenient to type, especially if you’re on the run, so you’d rather press a button. That’s why maybe speech, and I guess voice technology is going to be one of the interesting modalities, for sure, for an interesting interface, because it has a lot of advantages. Obviously, it has lots of a disadvantages too, but it’s definitely going to be interesting. And I think going forward, as we go towards agentic behaviors and more things are going to become possible, I think it’s definitely not going to be chat interfaces. It has to be something else.

I’m really fascinated by this topic, and it’s I think something that both Daniel and I have some passion for just in daily use, and stuff… I’m wondering, with you thinking about that kind of product customization, do you think that’s something that Perplexity engages in in a direct way, or supports other companies through that? And then there are so many times in the course of a typical day where I’m wishing I had other ways of interfacing with these capabilities that we’re talking about… And I’ll give you just a trite example. I will take my dog for a walk at a nearby park every day, and that’s my thinking time. That’s where I’m really trying to be creative. And I’m walking, and I have to keep walking; I don’t want to stand. Right now I stop, and I pull out my phone, and it’s frustrating, and people are going by me, and I’m trying to hold my dog… But I want this experience - it might be while driving, it might be while walking the dog - this seamless way of utilizing these capabilities that we’ve grown accustomed to come about. A, how do you see yourselves being part of that next journey on the interface side? And B, do you have any ideas on how to get there? It’s just, that’s the next thing I want.

Definitely looking into this, I think, and we consider multiple options. There are certain things I think we can do ourselves, for certain other things we probably have to work with some other partners. But I truly share your experience, and I think it’s – yeah, if you sit in front of a computer, I think by far the best interface is the keyboard. I don’t think you can do better than that. But yeah, if you’re occupied with something else, if you’re driving a car, maybe you’re walking, there has to be something else. Even the phone is – it’s okay, maybe even taking notes, maybe you say a command and something like that, and you get the voice back, that’s already something… But it misses visual information. So you’re gonna want to add that. So that means you probably have to have some sort of glasses on. I think it will definitely happen.

[00:38:20.25] And we will try, for sure – we spent a lot of time this year improving our mobile app to do voice-to-voice, and we invested a lot into voice generation. So for example, you can ask various questions; if you need something quickly, like you’re walking and you’re like “I want a quick lookup of information”, so we support that.

There is something for example for if you drive a car, we have this – you can read up the stories, or discover from Perplexity, that’s also AI-generated voice. So it’s like you listen to a podcast, or… So that’s super-important. Yeah, I think that the next step is vision, and so how do you get there.

Maybe one challenge that I’ve been thinking about this entire time while we’ve been talking relates to definitely a danger that I think people have identified as related to this new technology, which is - you’ve already mentioned that you’re doing web retrieval, or retrieval from certain sources as a primary way of grounding answers, of ensuring accuracy of citations… But I know a lot of people are concerned about and thinking about this sort of idea of data poisoning, where we’re putting out actually a lot of generated content on the web, and that proportion of human and generated content is going to change over time… Which means even for retrieval systems, especially if you’re doing web-related searches, there’s a potential that you could retrieve generated content itself, and get in this kind of weird loop… Of course, there’s a separate problem for the models and how they’re training… I mean, this affects a lot of different areas, but I know probably you’ve been doing a lot of thinking about this, because it’s key to how you operate as a system. Any perspective on that, that you feel like people should keep in mind moving forward, or things that you’re thinking about in terms of whether that’s data curation, or validation, or – I know a lot of people are talking about detecting generated content, and that’s maybe hit or miss… So there’s all of this kind of connected stuff, but generally around this idea of data poisoning, or generated content on the web - any thoughts?

To me it’s a technological problem. I feel like it’s very reminiscent of spam classifiers. It’s just a whole other level. Let’s say 20 years ago, when you received emails, you would receive a lot of spam. And then eventually, people developed technologies that can detect it. I feel like something like this will happen. So it’s always gonna be a constant battle between generators and discriminators… So at some point a generator’s maybe going to be better…

It’s like fighting malware.

[00:41:13.27] Yeah, exactly. It’s the same concept. It’s definitely going to be an issue, for sure… But my hope and my belief is that the good guys, the good generators are going to be – from machine learning fundamentals, discrimination is a much easier problems than generation. It’s much easier to tell what is good, what is not, than generated. And usually it seems like we’ve been more successful at detecting this stuff, and I don’t see any reasons why it’s not going to continue.

So this has been really fascinating. As we wind up here, and have you here for one more question, as we – we’ve talked about the future and have been kind of talking about what our expectations might be, and how those might be fulfilled… Are there any other areas that we haven’t addressed, that you’re interested in? And possibly, as part of that, any way of summarizing your own vision, without it being just answering questions that Daniel and I have thrown at you, but your own vision for what the future looks like to you, and what you want it to be, and what Perplexity is trying to realize it as, to kind of paint a picture of what we might see over whatever timeframe you want to address?

Yeah, so I’m very excited, basically, to get to a point where I have any question, any problem I want to have, and then just go to Perplexity and get an answer or a suggestion, or even perform an action for this. I already mentioned this thing, increasing the complexity of the type of questions you can ask, and then making sure that the system will be able to handle those. I think this is definitely going to be the future, and I think we work hard on that. It kind of opens up another dimension; you can just – you can ask lots of simpler questions, or you can just have one hard question. So it’s kind of a complexity versus quality. And I think, ultimately, when you get information, you usually use it for some sort of decision-making. And so if we can then take this information that we retrieved for you and synthesize in a form of answer, can we also then do some decision-making for you? And can we perform actions on your behalf?

So imagine you’re researching something, and maybe you want to research to buy the best riding shoes… It’s a pretty painful procedure right now, because there’s so much stuff on the internet; you don’t know what to trust. So that’s why if you identify usually riding shoes, you stick with them forever, because it’s just, you trust – but things like that.

So imagine if somebody will do this research for you, and really nails it down, weighs all the pros and cons, and then suggest “These are the possible variants? Do you want to buy this stuff?” And then you say yes, and then it goes and automatically buys it for you, and then two days later it’s delivered, and you only need to type the questions once. You don’t have to put your credit card in, and stuff like that. And there’s tons of examples.

So I would say the future is, first, you would have to make sure that you can nail sort of the information retrieval part, and generating the most useful information. The next step would be decision-making. So make a decision based on this information. And then third step, do actions. So that’s where – I think I would be excited to get to that point.

That’s great. Yeah. Thank you for being willing to take time to dig into a number of topics that I know our listeners are exploring themselves, and also interested in, and certainly Perplexity has been leading in a lot of these areas… So keep up the good work, and thank you for the work and the perspective and taking time to talk. Appreciate it.

Yeah, thanks for the great questions, and thanks for having me.