Jacob Hoffman-Andrews: Yeah, you’re definitely right, it’s a lot of work. I’ll go through a bunch of the stuff, and I should clarify a lot of this is not done by me. There’s a whole team that spent a lot of work doing this.

To be a certificate authority doesn’t mean much. You could become a certificate authority today. You can create a CA certificate on your own machine and issue your own certs, but nobody would trust them. If you loaded that on a web page, people would get a warning when they reached it. So you need your root CA certificate to be in trust stores. That’s in operating systems, or browsers, or things like Java; on Linux there’s usually a package called CA Certificates that ships down a bunch of these trust anchors.

[\00:24:10.08\] The problem is even if you were to convince all of those (we call them) root programs - the programs that manage who gets to be a CA and who doesn’t - even if you convince them all simultaneously to let you in today, there’s all the older software that doesn’t yet trust your root CA certificate. The usual way around this is what’s called cross-signing. You get somebody who’s already a CA to sign your root CA certificate or an intermediate, and essentially vouch for you and say “This organization knows how to safely issue certificates and based on our existing presence in these root stores, certificates from them should be trusted.” That’s step one - finding somebody to cross-sign for you.

Let’s Encrypt was very lucky to find IdenTrust, who was willing to cross-sign for us. Even if you are running an operating system that was built years before Let’s Encrypt existed, your browser will still load Let’s Encrypt certificates successfully because of that cross-signature. Now, in order to get that cross signature, you have to meet a certain number of requirements and pass a certain number of audits.

First, obviously, you have to generate your keys and your certs; you wanna generate those keys in an HSM (Hardware Security Module). The whole point of a hardware security module is to store crypto keys in a way such that even if somebody hacked you to bits, they still wouldn’t be able to make off with the keys; they might be able to trick your HSM into signing something incorrectly, but they wouldn’t be able to go make a copy of your private key.

So once you’ve generated those certs safely and written down all the steps you took – or rather, first you write down the steps you’re gonna take, then you follow them all on what’s called a “key ceremony”. You do it all on video, and you have auditors check that you did it all [unintelligible 00:26:20.26]

Seth Vargo: Exactly. So what we do for Vault Enterprise is if you need a UI, if you want a fool-proof backup and restore strategy, if you want integration, you want the 24/7 SLA, you want integration with a hardware security module - all of that is what we consider an “enterprise” feature. Everything else is open source; everything I’ve talked about is open source, and there’s a ton of other features that are available straight up in the open source offering. You don’t even have to give us your email to download it. The code’s on GitHub, you can download the compiled binary straight from the website.

Chris Beams: Yeah, so I’ll talk about “centralized” exchanges in just a second, like you talk about, but just to get first to principles - you’re asking “Why did we create it the way that we did?” and it’s because it’s really modeled after Bitcoin itself. We wanted to build the exchange that the Bitcoin network needed in our minds. What does that mean?

Well, Bitcoin itself is this remarkably censorship-resistant network; it’s really designed as such. It’s global, it’s at this point by all accounts unstoppable, right? It’s definitely powered through different governments and different agencies, considering ways to take it down or attack it, and so on. We’ll probably see lots and lots more of that over the years to come, but for essentially zero downtime on the networks, and January 2009 the Bitcoin blockchain has just been running along more or less smoothly, right?

[08:08] We thought that that’s exactly the kind of exchange that we wanted to build - something that’s just as permissionless, privacy-respecting, and indeed censorship-resistant as Bitcoin itself. What we’ve seen emerge since Bitcoin essentially first got a price, around 2010 – the network was running for many months before anybody ever actually exchanged Bitcoin for value, and pretty much the moment that began, the first exchanges started popping up, and if people know anything about this space, they probably know at least the headlines that they read in 2013 about the Mt. Gox exchange, which was responsible for something on the order of 700 million dollars worth of Bitcoin being stolen. Of course, the reason that that was possible, for Bitcoin to be stolen, is because when you’re dealing with a centralized exchange - meaning, by the way, a website, right? Fundamentally a web app, where three parties are involved - the person who wants to sell, the person who wants to buy, and the exchange itself. When things are designed that way, the exchange has to have custodial control over the user’s Bitcoin for at least some period of time. And typically, just for reasons of convenience, people would keep their Bitcoin on the exchange for much longer than just the moment that was necessary to match a trade and then cash out.

When Mt. Gox was hacked back then in 2013, something like the tune of 700 million dollars worth of Bitcoin were under the control of Mt. Gox, and that fundamentally means the private keys that were capable of spending that Bitcoin were in the control of Mt. Gox.

That’s basically a great, big honey pot. It’s like, there’s 700 million reasons that an exchange like Mt. Gox would get hacked, and it’s utterly predictable that the beatings will continue if this sort of architecture doesn’t change. And of course, that’s exactly what happened. Over the years, there’s been an almost predictable pace of exchanged getting hacked, and various amounts getting stolen and so on. It’s just the nature of the beast. When there’s enough value in a given system or location that can be hacked, it probably will be hacked. So that’s always been our motivation - that’s the security side of Bisq, it’s the security of users’ funds; we wanna make sure that people can actually fulfill the promise of Bitcoin, which is… You know, people sometimes say “Be your own bank”, that you can actually keep custodial control of your own private keys, your own Bitcoin, and you can have as much security as you can manage.

Assuming you have well put-together operational security practices, you’re using things like hardware wallets, or air gapped computers, or whatever it is that you deem appropriate to secure however many Bitcoin it is that you have, you have that full control, you have that full capability. The Bisq application never maintains control, it never has the opportunity to steal your funds, because it never holds your funds; it’s just between you and your counterparty, essentially.

Neil Daswani: So 60 seconds is a long time. And by the way, for those folks that are software developers in the audience here as listeners, there’s actually two different standards, two internet RFCs that are used to design that kind of authentication. There’s what’s called HOTP and TOTP. HOTP uses an HMAP, a hashed message authentication code, to generate the two-factor codes… And then TOTP uses time. So it doesn’t just rely on a seed, but there is a synchronized clock in between the authenticator and the website. So there are two ways to do that. But at the end of the day, it is possible to get phished by having [unintelligible 00:20:12.25] that accepts a username, a password and a two-factor code.

[20:18] So the most secure way to do two-factor authentication is to use what’s called a security key. A security key is a piece of tamperless hardware which you have to either plug into your laptop or your mobile phone… Or many mobile phones have secure enclaves and whatnot on it, that can be used to generate the appropriate two-factor authentication information… But the idea there is that in order to log in, you provide your username and your password, and what a website checks for is the ability for your security key - whether it be something like a Yubico YubiKey, or whether it be your mobile phone - to generate the two-factor code, but not in a form field that you have to manually enter. And that is a non-phishable form of defense against phishing.

And if you look at that particular set of security key technology, when Google deployed that, I believe in 2017 or 2018 - I can’t remember exactly which it was - they deployed it for tens of thousands of employees, and when they looked at it the next year, there were absolutely no phishing attacks. And both Google and Salesforce have used security keys to eliminate phishing as a root cause of any potential breach against themselves. And I really hope that more organizations learn from that experience and leverage security key technology to eliminate phishing, instead of having it continue to be a major root cause to breaches.

Jerod Santo: Three was the adage on computer performance, which states that the software is getting slower more rapidly than hardware is becoming faster. Number five was the security of a cryptographic system depends on the key length and algorithmic complexity rather than the secrecy of the algorithm.

Zac Smith: Even 18 months ago it wasn’t even on the radar. Now - right at the top. So… Okay, that’s great, because now we see business drivers… I think people are gonna pay for this too, which is really important… Because you don’t just get sustainability for free. We don’t get to just do “Oh, we did green power for you. It was a good marketing thing.” No, no, no. We invested tons of money to make meaningful impact to change our world; that is going to cost money. We are going to invest together. So I think that’s an important – that’s number one.

I think actually number two is that at some point, if we can solve this distribution of technology - right thing, at the right place, at the right time, so that way you could pull up on your iPhone and see the tracking of your cool computer to the right market, and then just turned on… And if we could snap our fingers and – let’s say you figured out just the right technology that you needed to use for your platform, and then you clicked a button or hit an API call, and somebody like Equinix got it into 50 to 60 markets around the world in like a couple weeks… That would be rad. [laughs] And I think we would see disruption in content delivery, and CDNs, and edge computing, and all kinds of things that we would do, and networks, and all things it could run on - I’m gonna call it hardware and software moving at their own pace.

So pending we solve this distribution thing, I think the big – and this is, again, probably… Now, you’d have to ask me what’s the 2025 predictions. That’s way more my style. But 2022, I’m not sure.

2020-something - the other thing I think is gonna be security. Right now, people just try and get the hardware or the thing in the right place, at right time, and they’re lucky to have it. That is not going to be our long-term challenge. We’ll solve that. Then we need to solve a way different approach to security, and that has to start at the hardware level. So I think our enablement of hardware-level security has barely begun. Most people don’t think about it in the software enablement side. They think about “Oh, I’m gonna encrypt my stuff, I’m gonna get my TLS going, I’m going to do all those things…” But really, even things like basic time protocols, basic boot processes… Is this machine the thing I think it is? Who touched it in the supply chain?

Mike Riley: Right. I mean, people who wanna go that route, I would suggest looking at some of the lock manufacturers like Schlage, for instance. They make Bluetooth-capable… But again, as far as I know - I haven’t checked Schlage out in a few years, but I remember that when they first came out with their devices, it was tied to their platform. So you had to use their app to be able to communicate. It wasn’t an open platform, I think partly because of some security, and also because it was first to market, so they weren’t really sure how capable they were when it came to “Should we open this thing up and be beholden to the world coming down on us? We’re trying to say it’s secure, but all of a sudden there’s all these security vulnerabilities that are discovered with this piece of hardware, and its firmware, and we can’t update it… Then what do we do? So those are some of the more concerning problems.

There’s certainly – for those people who have an electricians’ bent and they are woodworkers, yeah, they certainly could implement something like that. But it would not be for anybody who’s just looking for a simple solution at this point.

Elecia White: Probably sensors. And for the hobbyist, or - see, it isn’t just the hobbyists. I’ve gone to Supercon from Hackaday; I would happily go to Teardown. There’s Open Hardware Summit… There are a ton of really fun places where people will show you how to get into embedded systems, or hacking devices, or making cool things.

There aren’t as many professional conferences, unless they’re about certain topics, like Zephyr, or security. Hardware.io is one of the international conferences that’s pretty popular, and more security-based. I can’t think of any other ones, but I know I will when we get off, so I’ll send you an email and you can provide links.

Josh Aas: [35:53] Yeah… “You should trust us, and put your business and your reputation on the line” - it’s not an easy ask. So these cross-signing agreements - there’s a lot of liability involved, and for that reason they end up being non-trivial amounts of money. So that’s a big startup cost for anybody. Then going forward from that, that’s probably the biggest startup-specific cost, aside from maybe initial capital; you’re gonna need to buy servers, and HSMs, and things like that.

Ongoing, we have to buy a certain amount of hardware every year. We do use some cloud services, we use some external services, but mostly, aside from people, it’s about the data centers and what’s in them. Publicly-trusted CAs are not allowed to operate in the cloud, so we can’t run our CA systems on AWS or GCP or Azure or something like that. We have our own hardware in special, secure rooms inside data centers, and they’re not even normal data centers; they’re special, walled-off rooms in data centers, with a bunch of extra biometric access, and stuff like that. That stuff is a non-trivial expense, and you’ve got all this hardware that goes inside, you’re gonna have a lot of redundancy… So we pay for that stuff, and that’s where a lot of the rest of our budget goes.

Johan Vos: Yeah. Well, actually, quantum computers are already being used, but not at the scale that one envisions. For example, you can create entangled particles and use those to do secure encryption. There is already hardware that allows to do this, and with systems that can work with one qubit only you can already do a few things.

Now, what people really think about when they hear about quantum computing - the first example that most people talk about is about breaking encryption. And then the question is “How far are we from breaking RSA?” That’s very difficult to answer, and I personally don’t think that this is the most important application in the short-term… But anyway, how far are we from this? Well, it’s hard to say, again. Initially, people thought that the number of qubits is what defines how far we are in quantum computing.

Similar to classical computing - remember, 640k would be enough for all applications; then computers became bigger and bigger, so memory increased… And one would assume that this procedure would also be solved with quantum computers. We start with one qubit; you cannot do much with one qubit, but then two qubits, four qubits, eight qubits… And then the question is “How much qubits do we need before we have something that can really be worked on?” and more importantly, “How many qubits do we have today?”

Well, the question is not so much about how many qubits, because it would be not extremely difficult to get a quantum computer with a large number of qubits… But they have to be stable for a long enough time, and that is another problem. So getting a large sample of qubits is one thing, but having them stable for a long enough time to do some processing is the other thing… Because as we said in the beginning, if you measure that system or if there somehow is interaction between the system and noise, then your calculation is over. So the quality of the qubit is extremely important.

At this moment, we heard about 50 or 100 qubits that are sufficiently stable to do something useful… And I think this is good enough for developers to start working with it and to start experimenting with software on top of real quantum hardware.

It’s gonna take a while before we have quantum computers that are strong enough to break encryption, but it is not going to take a long time before we have quantum computers that can apply encryption. That is something, for example, that the Q-tech department of Delft University of Technology - they hope to achieve that in 2020. So this year they want to show a network of quantum computers, very small quantum computers.

You only need one or a few qubits in order to have them interact with each other and send messages in a completely secure way… And I think that is going to be a first widespread application of quantum computing. It’s maybe not as popular as breaking encryption, but guaranteeing encryption is for financial institutes, healthcare, everything where privacy is important, is going to be an important aspect… And I think that give it a year or so from now, that should be achievable. Not yet for everyone at home, but it should be used in between banks and different industries.

Christina Warren: Right? For me, that was kind of a big one. For me, building a website - that was the click with me. Because I loved writing. I was like “Oh man, I can have my own online zine, basically. That’s amazing.” And for some people it might be like “I want to have a YouTube channel, or a podcast, but I want to figure out how I can make that work more efficiently, more technically. Maybe I want to customize my Tumblr.”

[50:08] This is actually a great story… There’s a whole generation of, especially younger women, I think, that we haven’t catered to enough, who – because of Fandom, because of Tumblr, because of Discord because of other services… But frankly, they’re a little mini DevOps people, right? And they’re sometimes very skilled web developers and JavaScript developers who never anticipated that that’s what they were doing; they were just, “I want to make my site look as pretty as possible. I need to figure out bots to work so that I can get updates when my favorite band member posts to something, or when a new shoe becomes available, or I want to have other engagements…” And these people who then build up these skills. And I talked to some of them, and they’re like “Oh, but I’m not really a developer”, and I’m like “Look at all the stuff that you do. You have these really complex scripts, or you’re doing some really interesting things in JavaScript. You have all the stuff. Of course you are.”

The same thing as you were saying about maintainers. Okay, so you have 50 stars, and only a handful of hardcore users. You’re still updating and releasing, and doing security updates, and making changes based on feedback, and solving issues. You’re a maintainer, right? Don’t hold yourself to some standard, because - going back again, I’ve talked to people who are on those core teams for things like npm, or Deno, or whatever, who many of them still have impostor syndrome. So I don’t think it ever goes away, but it does, I think, make it better when you realize that it happens to everyone. That makes it, I think, a little bit easier to be less hard on yourself.

Feross Aboukhadijeh: Yeah, it should be. I mean, if it’s stored in local storage, then it means it is probably accessible to other apps on the machine, which I guess that is a weaker guarantee than – I think 1Password might do something like storing it in a secure enclave if your device has some kind of hardware place to store stuff. I’m not actually sure… So that might not be accessible through Web APIs. I would have to check. That is a good point; it’s probably a fine place to store it, but I’m not exactly sure what guarantees – how it compares to 1Password, so I’d have to look into it.

Nick Nisi: Agreed. And I think that’s probably their plan; it’s probably a safe assumption that that’s their play, is that running on their hardware fits into the narrative they have about being the secure platform. Because if it’s all local, you’re not sharing that data out anywhere. Whether that’s actually true or not - that’s another story. But it does fit with their perceived motives.

Kris Brandow: I don’t know if I agree with that first premise you had. I think, a) there’s a little bit of a fallacy in there, of “Oh, I’d rather live in this timeline, or this time period.” This is the only time period we live in, so this is what we’re used to. I think that that makes a mistake of assuming that this technology has been unilaterally good, or even net good, and I don’t know if that’s the case. I think that there have been a lot of very, very bad things that have happened to society as a whole as a result of the technology and software that we have, that were much less severe in the past because we didn’t have the software enabling it. But I also think that - sure, we have all of the software, it has built the modern world, but the question is not “Is the modern world good?” or “Are we doing good?”, but “Are we where we should be given the technology and advancements we have?” And I think the answer to that is unequivocally no.

I think the advancements we’ve had in hardware kind of prove out that software has not improved nearly enough over time. And in fact, it has regressed some. Our hardware has gotten tens of billions of times better over the last 60 years.

And I would not say that our software has gotten – maybe it’s gotten a thousand times better over the same time period, but we’re orders of magnitude off there. And I think the reason we’re orders of magnitude off is because we – I think we look too much, we focus too much on the micro issues, at the end of the day. Like the kind of “How did we accumulate all of this technical debt?”, instead of asking “Why is it that we have technical debt at all? Why is it that we assume that you have to have this thing as part of the process? Why is that a base assumption?” Why can’t it be that we know how to build things because we’ve been building them for a while, and for the most part, for most systems, you build it, and it works, and you move on to the next thing. And of course there’s going to be problems. It’s not to say that it’ll be perfect every time, it’s not to say there will be no issues, but that the amount of problems is small enough that we don’t constantly feel the need to have to spend months redoing things and reworking things.

[00:51:55.24] I think it’s a failing of us as an industry that we get ourselves in this position over and over and over and over again, and I think it’s from those same base level things. People pick up technology off the shelf. They don’t actually understand what the core parts of it are. They think that they need to be using it, and they quickly find out that no, we don’t know how to operate this, and now we’ve got to go back and learn, but “Oops, there’s no time to learn.” And so we get stuck in this cycle of just being trapped with this software, and trapped with these things that we can’t really pay down over time as much as we want. But that’s not necessarily because we made those choices. It’s because somebody, somewhere made those choices and said “No, you must do these things. No, we must operate this way.”

We as an industry say “No, of course you should be using Kubernetes. Of course you should be using gRPC and GraphQL and OpenAPI, and all of these things. If you don’t want to use these things, you’re just wasting your time.

You’re just trying to reinvent the wheel. You’re not really doing it.” But at the end of the day, how do you actually learn how any of this stuff works if you never actually build it yourself, if you never actually sit down and learn about it or understand it? So I think that’s what we’re really facing.

Once again, I do not believe that the complaint about technical debt is actually about technical debt. I think that’s where we’re probably differing, Johnny, is that it really feels to me like this technical debt thing is a symptom of a much larger cultural problem that needs to be resolved, which aligns with also the dislike of the hustle culture, the dislike of this capitalistic layoffs, and lack of job security.

So it just feels to me like that’s the root of it, and that’s where we have to start addressing the problem. It’s like, how do we make it so that at the end of the day we don’t actually wind up with much, if any, technical debt? I want to believe that we as an industry can get to a point where we can successfully build a software project, and have it run for more than a couple years, and not want to burn the whole thing down and rebuild it. I feel like we should be able to get there. I feel like that’s something we should be able to do.

Justin Garrison: Maybe they see something in either the stock market continuing to grow, or startups in general growing in the future… We’ll see. My link of the week is a Wired story, which - sorry, it’s paywalled. But for anyone listening to this podcast, you probably know things, archive.org, and whatnot… But it’s the full story of the stunning RSA hack, which is also from 2011, when Google bought Motorola. But I remember this story. I listened to a lot of security podcasts back in the day, Security Now, and various other security shows… And this one was brought up multiple times, because it was always just like “Something happened, and we don’t really know what”, but RSA being the keeper of basically two-factor authentication before it was mainstream… Like, if you had two-factor back in the day, you needed it, for some reason. This wasn’t like everyone had two-factor on their Gmail.

I actually have a semantic two-factor here that was an RSA key back in the day… And I had it for finance investments, or whatever. Some account, they wouldn’t let me have a strong password, so I’m like “Send me a two-factor.” And I’ve had it forever, but RSA got hacked back in 2011, and they stole the seeds for those keys, the physical keys. So you couldn’t reseed these with new keys and say “Oh yeah, just scan a new QR code.” No, these were physical devices that had it embedded in the hardware what the two-factor code was going to be.

Bailey Hayes: Yes, absolutely. So WASI started as the Web Assembly Systems Interface, but really now it’s very much the Web Assembly Standardized Interfaces. So we’re going through and finding that common set, or really it’s most often the case that there’s already some standard API or interface that already exists. I’ll use libusb as an example. So we’ve been working on was-usb,i that is very much a “Let’s come up with a Web Assembly-native way of describing that interface, that makes a lot of sense and allows for capability-driven security interfaces.” However, there are a lot of cases where people need a specific thing that they’re doing, and so in high-performance use cases for SQL contracts - that’s the most common one. People actually need exactly that thing. And in wasmCloud - Taylor and I are both maintainers of wasmCloud, which is a CNCF project; it’s moving into incubating likely soon. There we propose wasmCloud Postgres, which integrates the entire set of types inside Postgres. Everybody uses it, it’s easily the number one database that’s at play in the open source space today… And there we came up with an interface that we think is highly optimized for it.

There was previous work on “Can we come up with WASI SQL-style interface, that can just use kind of a lingua franca of SQL and hope for the best?”, bypassing strings and kind of just basically creating a SQL socket, is all we gave people… And that very much loses out on the promise and all the work that we did in the component model to be able to provide semantic and static high-level type information for people when they’re building… And then you just lose all that and give them…

So the answer is we are focused on WASI Cloud [unintelligible 00:22:17.10] stuff, which is the 80% case, we have some that are very, very optimized for a specific use case, at even a hardware level within WASI… And now there’s this third case that wasmCloud enables; we call it “bring your own components”, but essentially, it allows us to support custom interfaces. So write your interface for your provider, that you want, write your own provider, so you can write your own custom provider… If we don’t support it. But there’s a good chance that we already do. Then that can be very optimized for your use case.

The place where we’re seeing that a lot is for people who have their own maybe proprietary providers, that are a very specific data store that they’re using, and a very specific way of perhaps doing things like pagination… They define a WIT interface for that, which is the Web Assembly Interface Types, the IDL… So they write a semantic API for it, that’s driven for talking really easy between Web Assembly components… And then they can have lots and lots of different Web Assembly components that act kind of like a microservice that is able to communicate with that now. That’s one of the coolest things that I think we provide in wasmCloud, and the way we do that is pretty neat. But all of it at the end of the day is enabled by the standards, by WASI and Web Assembly component model.

Ramin Mohammadi: Six months ago I was going to the AI Summit in Austin. I got to the airport and I was passing the TSA pre, where the TSA agent asked me to check my bag. It turned out that the machine picked up on this preworkout container which I had in my bag. The agent used this device on the box, and the result was positive. It was like “Oh man, I need to call for this special unit to come and check this.” I was like “Sure.”

Then this special unit with something like a Hazmat suit, they came and used a kit with a bunch of different reagent to more specifically test my pre workout. They started sampling from the pre-workout, and added to a bunch of these test tubes… Long story short, the agent was like “Yeah, you’re good.” He was like “This happened to my cousin also. These pre-workouts are causing false positives.” So we are like that special unit, with a bunch of different models trained to find and protect the dark PHI… While current tools in the market are like the first and the second machine, which leads to false positives or unknown false negatives.

At Tausight, we do see this problem as a personal problem. It’s our PHI that’s being targeted, and clearly the current tools in the market cannot protect it. So a key part of security also is that - you must do a complete and true assessment of all your risks and vulnerabilities to ePHI. It is so fundamental to what we need to do. And AI is such a critical piece of taking advantage of the newer technology. Solving that used to be a labor-intensive problem, and that could be much easier if you can define the scope of the problem, and you can have machine learning models which can run effectively and accurately in a calibrated manner. That’s what we do, we take advantage of the AI to find sensitive data.

I think we get to the point where risks and threats and vulnerabilities are going to be detected at the edge, using the AI… As opposed to “Gee, I have all these heuristic rules”, which is how we do lots of stuff today when it comes to recognizing patterns.

[00:24:04.10] So at Tausight we use AI, for example, to recognize when the sensitive data is in unstructured content. It doesn’t require us to say “Hey, there’s a keyword here, there’s another key word here.” That would be how you do heuristic programming. A combination of these three words must mean this; this combination of these four must mean this. Those – you never get to do all the rules; you will never get to the variability you need. So in our model, for example - one of our models, with about 50 million parameters, that can be set to recognize this stuff. You will never in years of programming get that much logic into your regex.

Now, the other main factor for us is the ability to run these models right at the edge, where the data is being created, emailed, printed, copied or faxed. We bring the AI to the data, rather than taking the data to the AI, which most of the current AI solutions do that. By doing so, we can ensure that our data is always protected, agnostic of hardware specs or network connections.

Eric Boyd: So yeah, it’s a constant conversation of “Hey, this is what we’re working towards, and here are the early drops”, and starting to sort of make sure that we can stand up the infrastructure and run it at scale. And when it runs on Azure, we have to make sure that it lives up to all of the Azure promises, the things that people expect from us around the security, the privacy, the way that we’re going to handle data, the really boring features like VPN support and all of that, that VNET support… You can’t run an enterprise service without those things. So there’s all that work that has to go into it. But a lot of the work too is immediately working on optimizing the model, and how can we make it run as efficiently as possible on the hardware.

We’ll look at everything from literally the kernels that are writing effectively the machine-level code to the GPUs, all the way up to what’s the way that we should orchestrate and send requests to this across the data center. And so just every sort of layer across that stack, we have people whose job it is to really go and optimize and think through every part of it and just squeeze out every percent of performance that we can… Because it shows up for customers, and it shows up for us. We’re running at just such massive scale that a 5% improvement is a lot of money. And so it’s really important to see all of that.

Daniel Whitenack: Yeah. I was asked – I was at a conference last week, and I was asked which direction things would be going, either local AI models or hosted in the cloud… And I think the answer is definitely both, in the same way that there is a place for – if you just think about databases, for example, as a technology, there’s a place for embedded local databases, that operate where an application operates. There’s a place for databases that run kind of at the edge, but on a heavier compute node that serves maybe some environment, and there’s a use case for databases in the cloud. And sometimes those even coexisting, for various reasons.

In this case, we’re talking about AI models. So I have a bunch of files on my laptop; I may not want those files to leave my laptop, so it might be privacy reasons that I want to search those files or ask questions of those files with an AI model. So privacy security type of thing, or in a healthcare environment they may have to be airgapped, or offline sort of thing, or public utilities sort of scenario, where you can’t be connected to the public internet… But then it might just be also because of latency or performance, inconsistent networks, or flaky networks, where you have to operate sort of online/offline… There’s a whole variety of reasons to do this. But yeah, there’s also a lot of ways that, as you said, this is rapidly developing, and people are finding all of these various ways of running models at the edge. And we can highlight – if you’re just into this now, and getting into AI models, maybe you’ve used open AI’s endpoint, or you’ve used an LLM API… If you wanted to run a large language model, or an AI model on your laptop, there’s a variety of easy ways to do that. I know a lot of people that are using something like LLM Studio - this is just an application that you can run and test out different models…

There’s a project called Ollama, which I think is really nice and really easy to use. You kind of just spin it up; you can either spin it up as a Python library, or as a kind of server that’s running on your local machine, and interact with Ollama as you would kind of an LLM API. And then there’s things like llama.cpp, and a bunch of other things. These I would kind of categorize as local model applications or systems where there’s either a UI, or a server, or a Python client that’s kind of geared specifically towards running these models locally.

And then there’s a sort of whole set of technologies that are kind of Python libraries or optimization or compilation libraries that might take a model that’s maybe bigger, or not suited to run in a local or lower-power environment, and run that locally.

[00:10:03.11] So if you’re using the Transformers library from Hugging Face, you might use something like Bits and Bytes as a library to quantize models, shrink them down… There’s optimization libraries like Optimum, and MLC, OpenVINO… These all have – some exist for some period of time. Actually, I think in the past, we’ve had the Apache TVM project on the show, and we talked about OctoML… So this is not a new concept, because we’ve been sort of optimizing models for various hardwares for some time. But these optimization or compilation libraries are also usually kind of hardware-specific, so you optimize for a specific hardware. Whereas other of these local model systems are maybe more general-purpose, less optimized for hardware specifically. I don’t know if you’ve got a chance to try out any of these systems, Chris, running some models on your laptop…

Bryan Cantrill: Exactly. And so actually, with this particular firm, I remember telling them that the – also, in particular, on our first slide, it’s like Jeff Bezos, at the time - now it’s Andy Jassy… “This is the company. Jeff Bezos is not going to own and operate every computer on the planet. Not every server is going to be a public cloud machine. There is going to be on premises computing. Folks that run on prem, whether that is for economic reasons, as Samsung had to, whether that is for risk management, or security reasons, as certainly you see in federal spaces, government spaces and so on, whether that is for latency reasons, as you get people kind of collocating near their data, and so on… There are lots of reasons to run on prem.” And they’re like, “Okay, yes.” But then you get to the next slide and their first objection is like “But won’t it all go to the public cloud?” I’m like, “Okay, this is why we shouldn’t be here… Because you only see public cloud computing. And we are very pro-cloud. I love cloud, I think it’s great.” I think for startups, public cloud is perfect, because when you are starting from zero, of course you should not buy your own hardware. Like, go spin that up in the public cloud. That’s awesome. But you get to a certain scale, or you have certain constraints, you will need to own and operate your own computer, in your own data center, collocated data center. So they just don’t see that. They don’t see that world.

[01:24:18.14] So with this particular firm – so every time I’d be like “You know what, this is why we should leave.” So kind of like “We should leave.” “No, no, no, no. Stay, stay, stay. No, we’re interested. You’re making us more interested.” And like “Oh, God, what are we doing? Why are we doing this?”

And the real problem is they would kind of convince us to stay, and we would say. We should have just like walked out. At one point I did tell that particular firm, I’m like “You have to understand, we are going whaling. You should be fishing from the pier. You should not come whaling with – it’s like, stay here. You don’t want to come out on this whaling vessel with us. You do not want to be on this multi-year, dangerous voyage.” But again that did nothing. And of course, they spent a ton of time, and then ultimately, they realized, like, “I think we should fish from the pier.” It’s like, “Yes. You should fish from the pier.”

Justin Garrison: [00:10:05.05] Of those bullet points in the article too, like “Publish vulnerability disclosure policies for reporting security flaws.” They didn’t have to do anything. They didn’t have to have a blog post to say “Hey, by the way, this is vulnerable.” And the second one, “State minimum periods for providing security updates” - again, they don’t have to do anything. They just have to say “We are going to have 12 months of security vulnerabilities”, or something.

And then that last one on there is “Provide a mechanism for securely updating software.” That’s the first one that like “Oh, they have to have a way to update it.” Like “This is the thing that’s wrong, this is our guarantee for this thing.” I wonder how this applies to things that happen… Like I know California is pushing out a really big law for right to repair, and right to be able to own some of the hardware and stuff, and be able to fix it, and be able to “Hey, if this is vulnerable, I should have a way that I can get into – even if you don’t provide security updates, I should be able to push in my own security updates if that is something that is outside of your cycle of maintenance anymore.”

And so I’m wondering how this kind of plays into that as well, where it’s like “Hey, on one hand they need to be responsible for some amount of time”, and then beyond that, a consumer or someone smart enough, or a community should be able to take over and provide the other security updates and things… Because I have plenty of IoT devices that are long past their lifecycle, but I’ve been able to flash them with different firmware, or close them off from the internet, and do things that I have some control over, but not necessarily like route them… Not all of them, at least. I can’t take full control. So…

Kris Moore: So I’ll do my best to try and put people at ease and explain the situation. So Core, again, FreeBSD-based; it’s a lot more stable, it’s mostly in a maintenance mode, and we’re trying to keep it very rock-solid, and we’re gonna keep supporting it. Again, we’ve got enterprise customers running this, so it’s still here for a while to go. And then on the Linux side, that’s kind of the up and comer; that’s the teenager in the house, right? A lot of energy, a lot of new, exciting things are landing there… And we’re seeing all the momentum shift to that variant of the product. So that’s not a real secret. Again, anybody who’s paying attention can see that with their own eyes.

[16:05] But that said, we’re not dropping people on their head. We’re gonna support Core as long as it makes sense to do so. And especially when we have enterprise customers who prefer to run that platform… Just because some people don’t want to do any kind of upgrade or migration; they’re pretty happy just to set up the NAS and not touch it again for five years. So it makes sense, so we’re gonna do our best to support both, make sure security is good and both.

But again, it’s not a secret - Scale is where all the innovation is happening these days. So if you want that shiny, new feature, it’s going to be on the Scale side. And if you’re a homelabber, this is particularly important for anybody who’s a do-it-yourselfer. The hardware supports is just flat out better on the Scale side of the fence. The Linux Kernel - again, no offense to the FreeBSD guys; it’s been a great product, and I’ve been a BSD guy for almost 20 years. I was on the core team a couple times… So that’s where my heart was at. I get it. But at the end of the day, just as far as sheer breadth of hardware support, and kind of features and functionality and all the bells and whistles, that’s going to be on the Linux side.

Jerod Santo: Yeah, there’s no one size fits all. I do think that maybe when it comes to software products, it may be that the successful ones, because of the profit margins on software sales, which you all are enjoying… Like, every marginal sale for you has not nowhere near as commensurate work on your side, right? You have support, but you could just sell them to the hills. And so when you start to actually get over that hump, and the flywheel is rolling - well, it’s hard to stay small at that point, because…

[01:15:53.27] The other one I’m thinking of is – and they actually sell hardware, so it’s slightly different, but it’s similar… It’s Thinkst, with their canaries. We had Haroon Meer from Thinkst on the show last year, talking about Canary… And they’re relatively small, and making really good money selling these security devices into enterprises. They’re like honey pots, basically. And they have a hardware aspect, which means that their margins aren’t as good. But once their sales got rolling, they’re making really good money. And he has to try hard to stay small, because it’s so easy to chase every opportunity when you have capital to spend.

And so maybe that’s why people end up chasing larger growth, is because, once you’ve conquered the mountain, then you’re like – wasn’t there an old saying about a guy who cried because he had no more mountains to conquer? Was it Alexander? I think it was Alexander.

Adam Stacoviak: But I had this hardware that was essentially non-useful to me now. And that’s where I discovered your brand and who you are. Well, not so much you personally, but who your brand is. But I’ve never gone even beyond that until recently, with your involvement, in these bills getting passed, or these acts getting passed, and all the fight that has to happen. So here’s me, saying [unintelligible 00:27:45.14] because they’ve basically built gigantic farm computers… And this whole movement there. Talked to Cory Doctorow several times, again, I follow Louis Rossmann… And just like knee-deep in this. How does someone like you, software engineer, decides that you want to do something with a repair guide? Now you’re part of this change that needs to take place, because I do want to gut my Mac Mini six years from now or eight years when it’s not useful to me anymore and repurpose it if I so choose. I want to be able to run Linux on – if it’s a computer, I feel like you should be able to run Linux on any given computer in the world. Because Linux is open source, and it should be the replacement OS when that OS can no longer work. In this case, thiis macOS version was like on Mavericks, or something like that, for this Mac Mini. So even if I wanted to run macOS, I’m limited to – I can’t go to the newest operating system. I have to go to this older generation that has limited security, no updates - great. But Linux still works. Ubuntu still works on any given thing. So long story short, I’m new to learning about you and your company, but how did you get into this movement? Why are you so passionate about it? Obviously, there’s some steam here, but what makes you be the right person to build the company, to distribute parts, and to be the “advocate” for those who are not advocating, or don’t know how to advocate for their rights?