David Crawshaw: [00:20:18.21] I think that is exactly the right way to frame the question for a business, and I don’t know the answer to a lot of those questions. I can talk to some of the more technical costs involved. What the benefits would be to the company is extremely open-ended to me. I don’t actually – I can’t imagine a way to measure that. Based on talking to customers of Tailscale who deploy it, thinking about the [unintelligible 00:20:43.07] And so to go back to something you said earlier, about how you use it and you don’t pay for it… I think that’s great, because Tailscale has no intention of making money off individual users. That’s not a major source of revenue for the company. The company’s major source of revenue is corporate deployments. And there’s a blog post by my co-founder, Avery, about how the free plan stays free on our website, which sort of explains this… That individual users help bring Tailscale to companies who use it for business purposes, and they fund the company’s existence.

So looking at those business deployments, you do see Tailscale gets rolled out initially at companies for some tiny subset of the things that it could be used for. And it often takes quite a while to roll out for more. And even if the company has a really good roadmap and a really good understanding of all the ways they could use it, it can take a very long time to solve all of their problems with it. And that’s assuming they have a really good understanding of all the things it can do. And the point you’re making, Adam, that people often don’t even realize all the great things you can do with it is true. And I’m sure a tool that helped people explore what they could do would have some effect on revenue.

In terms of the technical side of it and the challenges - there are several challenges. In the very broad sense, the biggest challenge with LLMs is just the enormous amount of what you might call traditional, non-model engineering has to happen at the front of them to make them work well. It’s surprisingly involved. I can talk to some things I’ve been working on over the last year to give you a sense of that… Beyond that, the second sort of big technical challenge is one of Tailscale’s core design principles is all of the networking is end-to-end encrypted. And the main thing an LLM needs to give you insight is a source of data. And the major source of data would be what is happening on your network, what talks to what, how does it all work? And that means that any model telling you how you could change your networking layout, or give you insight into what you could do would need access to data that we as a company don’t have, and don’t want. And so we’re back to - it would have to be a product you run locally, and have complete control over… Which is absolutely – my favorite sorts of products are that. I like open source software that I can see the source code for, compile myself, run locally. That’s how I like all things to be. But trying to get there with LLMs in the state they are today is actually, I think, pretty tricky. I don’t think I’ve seen an actually shipped product that does that really well for people. There’s one, there’s a developer tool that I hear a lot of good talk about, that I don’t – I’m just trying to live-search for it for you. Nope, that’s the wrong one. That’s Magic Shell History, which also sounds really cool. I should use that one.

Jerod Santo: Yes, so that’s one of the core things about Atuin, is its unified shell history. Not just across your tmux sessions, but actually - and here I am with one computer, so I don’t really get to live this life - but across all your machines as well. So that’s one of the core things that Ellie is doing that I don’t really care about as much, because I just have one computer… Is like you could have your shell history and synced across all of your machines via an end-to-end encrypted sync service that she provides. But I don’t really know how I want that to work sometimes, honestly. And there’s moments where I’m like “Oh, I typed this command into another tmux session”, and I would love to just up-arrow and type it here… And it’s not there. And then there’s times where it is there, but I don’t want it to be there… I’m like “Actually, that was contextual to another thing. I don’t want it in this little shell history.” I mean, I am of two minds about how I would actually want the feature to work. I think it’s not straightforward. Have you had that experience?

Jerod Santo: Well, I would definitely agree on the setup process and the documentation was all very hand-holding and nice. And even when I was feeling like “I don’t know if I want you, like –”, right there you’re like “No, it’s end-to-end encrypted.” I’m like “Alright, so this person cares.”

Jerod Santo: sshx lets you share your terminal with anyone by link, on a multiplayer infinite canvas. It has real-time collaboration, with remote cursors and chat. It’s also fast and end-to-end encrypted, with a lightweight server written in Rust. (So you know it’s cool)

All you have to do is install the open source CLI by curling sshx.io/get and piping the output to your shell, or build it from source if that’s more your thing. This gives you the sshx command, which, when executed kicks off a live, encrypted session. Send the link to whomever you want to join in and they open it in a web browser.

Resizing, moving windows, zoom and pan all work, you can see other people’s cursors in real time, and more. Check it out at sshx.io

Michael Quigley: Yeah, yeah. And the nice thing about the design too is you can take whatever layers make the most sense for what you’re trying to do – like, we generally ship the entire stack to customers, and they use the whole thing, but you could use just the mesh if you wanted to, you can… And then going up even higher, now that Ziti is starting to become mature, I’ve sort of pivoted my work and I work on an open source project called zrok now. And zrok was sort of a – the name, you can kind of tell what it was inspired by just the name. Most people are familiar with ngrok; that kind of had the… I was basically like “Go figure out what something like ngrok would look like on top of OpenZiti.” So I built zrok. And one of the cool things about zrok is that you look at tools like ngrok, and it’s awesome that you can share a private resource on a public URL, and let anyone use it, and you don’t have to open any security holes… All those things are great. But one of the cool things about Ziti is because we have this overlay network, and we control the addressing, and it’s all zero trust, we can do peer-to-peer across that network without requiring any of the peers to expose themselves publicly, or anything like that. All the peers can remain “dark”, but we can still have peer-to-peer communication. So that lets us do things like private sharing, where we can say “I want to share a network connection between two completely dark peers”, and the traffic will go across the overlay, across the internet, but none of that’s exposed to any of those things. All that data is end­-to-end encrypted, it’s all zero trust under the covers… So that’s one of the things that’s really cool about it. So zrok ends up becoming like a really good use case for showing people how you can use something like Ziti to build things that you couldn’t otherwise easily build.

Jerod Santo: Our Efficient Linux at the CLI episode revealed my recently-closed ⌃r knowledge gap (I know, right?!), which prompted Matt Lehrer to point me towards Atuin by Ellie Huxtable. Atuin is a command-line tool that enables you to make better use of your shell, by giving ctrl-r superpowers.Every line you write is stored - ready to be queried and run again at any point, from any machine you wish. Never forget again! You can even Sync your history between all of your machines, and search it from anywhere.

How does Atuin do it? By replacing your existing shell history with a SQLite db, recording additional context for your commands, and syncing that db (fully end-to-end encrypted) via a server Ellie hosts or your own setup.

Anna Pobletts: So the big announcement of passkeys is essentially saying “We’re going to take those WebAuthn credentials that have been around for years, and we’re now going to sync them. We’re going to sync them across your platform accounts, across secure end-to-end encrypted channels.” So now you can sync passkey through your iCloud account, or your Google account, or your 1Password account. And so yeah, I have an Android phone, I might lose my phone or get a new phone, but as long as I can log into my Google account, those passkeys will automatically sync. They’ll also be shared across any devices that I have in the same ecosystem, things that to help you kind of have passkeys more accessible in different places, to help with the account recovery problem…

You know, it’s still not perfect by any means, but it really was almost unusable before that. It was so much work to manage your different devices that had passkeys. With passkeys as they are now, where they’re synced between devices, it’s at least a little bit more accessible to people, and we can start to really focus on the more narrow account recovery problem.

Ken Kantzer: Yeah. Funny story about that - I think it was the first or second year at PKC, so this was like 2014-2015. I think it had been out for a while, maybe… But we were actually in the middle of building a product. So we built an end-to-end encrypted alternative to Slack called Balboa. Awesome experience. We decided to build it in Clojure, which is probably not the best choice ever… But it’s a great language, and it’s a lot of fun. So we were in kind of like the middle of building a product, and thinking about VC funding, and so we started watching the show with the original three founders… And halfway through the show, I look over, I’m like, “Wait, Dan’s missing.” One of the founders. We’re like “Where the hell is Dan?”

We go into like this bedroom, and he’s on the floor in like a fetal position… And we’re like, “Are you okay?” He’s like, “I can’t, I can’t take it. It’s too much.”

Feross Aboukhadijeh: Yeah, I think we’re definitely gonna keep working on it. I mean, like I said, it has hundreds of thousands of people using it every month, so there’s no reason to shut it down. It’s not that expensive to run, because most of the data is peer-to-peer. We store the files for only 24 hours… You know, the end-to-end encrypted files, we store them for 24 hours, so the data cost is pretty low. So there’s no reason to not continue running it.

If we were to consider shutting it down or whatever, we’d definitely open source it before we did that. And we might even open source it anyway, just proactively, because it’s a thing – it’s useful for people to be able to run their own instance of it, or whatever… So we’re thinking about it. It’s just a matter of time; someone has to go and do that. And then there’s the burden of pull requests, and issues, and then you’ve gotta run a community…

Feross Aboukhadijeh: Yeah, yeah, so the first thing is that the URL contains two pieces. It contains the room ID (is what we call it). It’s just the ID for the files (the data that contains the files) [51:46] and then it contains that secret key that’s not sent to the server. So you make a request to the server for that room ID, it tells you “Yeah, there’s a room here, but I’m not gonna tell you anything about the room until you prove that you have the key.”

[52:00] Even though everything, like I said, is end-to-end encrypted, you could imagine – we could have designed it so like if you show up and you say “Hey, I know there’s a room with this ID. Give me the files for it, please.” We could have just said “Sure, here you go”, because we know they’re encrypted. We’re not worried about your data getting out there.

We don’t wanna even give the copy, even of the encrypted files, out to random people, so we add this extra layer of security where you as a downloader take the secret key that’s in the URL up there, you take that and you derive a new key from it. So this is a way to sort of take a key [52:28] and make a sub-key, and – how do I describe it…? Basically, all you need is that main key, and then you can always get to this sub-key. You can always deterministically produce the same sub-key. But given the sub-key, you can’t go back to the master key. So what you do is –

Matthew Hodgson: By far the biggest thing that we can do right now is to fix the first-time user experience in Riot… And it’s tantalizingly close. If we can go and get the UX, so that people will try using it rather than Slack, and think “Oh, this is actually a bit better”, then… Ignore all of the open source, all the decentralization, all of that stuff… Just from a pure, pragmatic user perspective, you’ve gotta have that. That’s more than anything else what I’d like to have happen.

Secondarily, we’d need to fix the reputation and abuse challenges. We’ve made some good work on that. We have the concept of shared blocklists, so that if there are abusive folks, you can choose to ban them from your room or server or community or user, and then kind of pool that information together… A bit like email blacklists, but hopefully a little bit more nuanced. That is improving things a bit, but it needs to be a lot more nuanced if it’s gonna solve the kind of grayscale things. If you ban somebody for spamming a room, it’s pretty concrete. But if you just happen to want to dial down the NSFW content in a room, that’s getting a bit more gray as to what somebody considers safe for work etc. So we need to have that subtlety to support that.

[01:16:00.20] And finally, end-to-end encryption has been an epic to make work. We are the only decentralized real-time encrypted system that I know with end-to-end encryption, and it turns out people don’t tend to do that. It’s really hard to get the edge cases to work, and people are very unforgiving for random messages that don’t decrypt, because there was a race between sending the keys, or one of the servers went down in the middle, or there was a netsplit and the server who sent you the message didn’t know you were even in the room, so of course it didn’t encrypt it for you. From a user perspective, if you don’t get the message - that is, you know, the one thing you had to get right.

So we’ve spent so much time trying to get that right and turn it on by default. We got a lot of flack in the privacy space for not having it on by default, particularly as WhatsApp, and Wire and Signal are obviously all encrypted by default these days… So we aimed to get it on for the end of January. It slipped. The best quote I heard is ten days from now is the next mission… But we’re still working through that work, so I really hope a year from now that’s old history, and everybody is used to everything being end-to-end encrypted.

Other than that, on the commercial side obviously more governments and big players adopting it to the extent that it becomes the default way to do chat in an industry, rather than email or phone calls, or jumping on Slack and paying huge amounts of money, would be a good outcome.

I’ve already mentioned France and Germany, the U.S. is also using Matrix in production in various public sector environments… Unfortunately we can’t really talk about them, but some of them are really, really interesting; places where the decentralization is super-useful. In fact, what we can talk about is the Texas Department of Emergency Management and Public Safety, where they previously were using WhatsApp for coordination in the field, and they’re very extensively using Matrix now to coordinate in a disaster scenario, working that domain.

It would be great if open source projects didn’t use a proprietary thing like Slack, they’d instead use Matrix… And if governments did it too, then between the geeks that FOSDEM and the public sector folk, or whatever the open source conference equivalent is for the public sector… They actually do have open government conferences, which as pretty interesting, as all the people compare how to build open public infrastructure… Then perhaps the rest of the world would meet in between, and it would just keep growing.

Oh, the other thing I’d wave a magic wand is to migrate to Dendrite and kill off our Python codebase and move to Go, because the scalability nightmare we have on Python… Not Python’s fault, but we just built it as – it doesn’t scale horizontally, whereas the Go thing is very horizontally scalable, and is shamelessly inspired by some of our high-volume competition in the proprietary space. That would be great because we waste far too much time debugging Python, when I would just like it to be efficient.

James Long: I can start with why I – so I sort of naturally fell into this around 2017, which was around the time – I think the essay came out like a little bit later, after I had started going down this path, and it sort of crystallized a lot for me… But yeah, when I was building Actual, which is a personal finance app that I built and tried to sell for a couple of years, it actually met most of the principles of the original one, where it’s actually completely local-first. Literally, all of your data is local, and the server is a super-dumb server. And in that architecture, you can even like end-to-end encrypt, which felt appealing. I think it was more appealing idealistically… Or like ideologically appealing, because it was just like “Hey, it’s your personal finance data. It sounds cool to encrypt it.”

I think a lot of my evolution from this is like nobody really cared that much about that, and so I’m sort of reeling back from some of the deep research I did back then. But anyway, the main reason I did it was for performance, and just like simplicity. Like, it just felt really cool to just like write SQL queries, and have this SQLite database totally available to your entire local machine, rather than like an HTTP request that came back with like most of the fields, but some of them were missing, because they needed to join some table that was on a totally different sharded database in a separate place, and so then you have to do a follow-up HTTP requests… And GraphQL sort of follows some of this, but GraphQL just feels like super-overly complex. And when you need to do mutations with GraphQL, it just like really locks you into this specific format. And then you’re always waiting on loading spinners, and things like that. And it’s just like this super-compelling simplicity of “Hey, SQL query, get my data back within like five milliseconds, render it, and then subscribe to that data.” And it’s all completely local, and you don’t need a web socket to a remote server that could die anytime now. So that was my main motivation. And I think that might hold for your reasons, Johannes. Is that right?

Daniel Whitenack: I’m glad you bring that up, because it’s probably – even though I may not get access to the DGX system, I am thinking about various applications at the edge. In fact, I had a conversation earlier today with another guy who’s working on totally different stuff in manufacturing; they’re not a large company, but they do stuff at the edge, in the manufacturing setting, with low-power devices already. Think like a Raspberry Pi, and that sort of thing… But if you could bring the power of this sort of GPU right to the edge, to a machine, where you’re doing computer vision to detect anomalies in your manufacturing process or something like that, that’s a pretty major advantage, and that brings that sort of capability to those sorts of people that are working on smaller teams, and have that specific use case for running AI at the edge.

[40:19] Along with the A100 they have the EGX A100 which they’re releasing, which seems to benefit from some of these things that we talked about with the A100, but they also talk a lot about security. Security, and then end-to-end encryption of AI models; they encrypt all the things. I have some ideas about why that may be important at the edge, but… Do you have any thoughts on that?

Andrew Beyer: The keys to unlock your account, or essentially what we call the master unlock key (easily named) is unique to you. It is unique to you and your account. So your account password and your secret key derives the master unlock key. And from there, it’s just a whole host of other keys. So you’ll have a key per vault, and those kind of things. They aren’t unique to a device; so when you have two different devices, you’re using the same secret key and your same account password, and that’s essentially how we ensure the end-to-end encryption there.

We do offer some authentication on top of that, so every device has a unique identifier, and we use that just for the server to know the devices out there that it can download the encrypted blobs of your 1Password items to… And we can do things like multi-factor authentication that way, where you’re doing – the base authentication is all done through encryption, but if you wanna add on another layer, you could add on Ubikey, or a time-based one-time password, and that’s where that device UUID, on the server, it can perform that MFA.

[12:08] And really, the only protection that adds is if someone were to obtain your master unlock key, or your account password and secret key, they can’t use that on a new device to download your encrypted information. So it does add kind of a different layer of making sure that nobody can add to, remove from your account, download your account on new devices.

Andrew Beyer: The keys to unlock your account, or essentially what we call the master unlock key (easily named) is unique to you. It is unique to you and your accounts. So your account password and your secret key derives the master unlock key. And from there, it’s just a whole host of other keys. So you’ll have a key per vault, and those kind of things. They aren’t unique to device; so when you have two different devices, you’re using the same secret key and your same account password, and that’s essentially how we ensure the end-to-end encryption there.

[12:25] We do offer some authentication on top of that, so every device has a unique identifier, and we use that just for the server to know the devices out there that it can download the encrypted blobs of your 1Password items to… And we can do things like multi-factor authentication that way, where you’re doing – the base kind of authentication is all done through encryption, but if you wanna add on another layer, you can add on Yubikey or a time-based one-time password, and that’s where that device UUID will – on the server, it can perform that MFA. And really, the only protection that adds is if someone were to obtain your master unlock key, or your account password and secret key, they can’t use that on a new device to download your encrypted information. So it does add kind of a different layer of making sure that nobody can add to or remove from your account, or download your account on new devices.

Filippo Valsorda: I think this is called the end-to-end principle. The concept that all the layers move to the end points, because the endpoints have the most context about what you need to do. So the TCP stack has to work for every application, while the browser knows it wants to load a web page. So it can make different choices.

One of my favorite facts about QUIC is that it encrypts the headers. Not for privacy, but because they really don’t want the network engineers to mess with them. So they just went like “You know what? We’re going to encrypt the hell out of the headers, so that you don’t get to have an opinion.” If that’s not the end-to-end principle, I –