Seth Vargo: Yeah, very high entropy UUIDs. So those are the different ways you authenticate to Vault, but it all ends up the same. You can think of it as a big funnel that’s all going into this thing called a token. So whether you put a username and password in or you log in with GitHub or LDAP, the thing that happens is you get back a token, and that’s very similar to a session ID on a website. That token is how you authenticate moving forward in the system. You don’t ever supply your username and password again, or your GitHub login again - it’s all that token moving forward, and that token has permissions assigned to it. Vault manages those permissions based on the authorization (AuthZ) and it manages all of that internally through a really verbose policy system.

Everything in Vault is based on a policy. Just because you can authenticate to Vault doesn’t mean you’re authorized to do anything. It would be like logging into a website and the very first page you get is access denied. You were successful in logging in, but you don’t actually have permission to see anything in the system. What Vault administrators, in collaboration with the security teams do is they generate policies that map authentications to permissions in the system.

Here’s some examples before of like anyone in the engineers team can do certain things in Vault, and that’s really where Vault’s power is. Because as new people join the company, they just get added to the team on GitHub and they automatically inherit permissions for all of these things. As a new person joins – say you’re a large enterprise or a really big company that has a massive Active Director installation, and you have an employee who moves from one team to another team… All you do is change their OU in LDAP from Team1 to Team2, and automatically their permissions are updated throughout the system. They might lose permission to things they had before and gain permission to new things in the system, and that’s all handled out of band, because you’re already managing an Active Directory server that’s already part of your company culture and company technology, so we’re just integrating with those technologies to give you authorization to different resources in the system.

Adam Stacoviak: I was just like “That’s a really cool name for an agent that doles out secrets”, maybe. Or just deals with authentication and authorization kind of thing. If there was an agent, if there was a secret agent…

Andreas Møller: So because most of authentication is a backend issue, we don’t handle most of it, because we don’t have a backend, we just work with whatever your backend is… Whether you coded your own, or whether you use like Supabase directly as a backend, or whatever you’re going with. There’s also some – like, Xano is one of that more low code/no code style backend; it works really well with that as well. So from our point of view, again, we’re frontend, so we’re just doing HTTP requests to a server. So what we’ve done around authentication is say “What matters for us is how we store the token, in essence.” Whatever strategy you have for authentication is possible.

What we sort of nudge people towards is storing your token in an HTTP-only cookie. Because that’s generally the most secure way of handling it. And what we’ve done is we have an API proxy. So whenever you have an API from the frontend, if you run it through our proxy, we can actually authenticate that request. So let’s say you need to send an authorization header with your user token to an API - that’s the most common approach - well, you can actually set up “Well, that’s what I want to send”, and you don’t actually have the token on the client, because that’s stored in an HTTP-only cookie from when you authenticate it, but we actually put that token in before that request goes to the server. And that way, even if you have some script injection attacks, etc. they can’t actually steal your token. That’s not going to fit for everyone. Sometimes if you want to do – especially if you’re doing like real time WebSockets and stuff, that’s not always an option, and you do need to have the token on the client. So whatever your authentication strategy is, is possible. And it sort of speaks to the general approach…

I sometimes say “We didn’t really invent that much.” And by that, I mean almost everything works the way you would expect it if you stopped looking at it as a visual tool, and started looking at it as a JavaScript framework. So when people ask “How do you do authentication in React?”, it’s sort of almost a half abstract question, because it’s like “Well, however you like.” And similar, “How do you store things on the client?” Well, there’s local storage, there’s session storage, there’s IndexedDB, there’s all these options… Because it’s a web browser, right? And we are a framework, and we just build visual tools on top of that. But we didn’t invent whole new abstracts. The web’s pretty good. The HTML, CSS - it’s the best we’ve come up with yet, I think… So we’re not reinventing it.

Obviously, it’s not running JavaScript, but it’s very much stealing everything it can in terms of function names and making sure that everything is very familiar to someone coming from that world. And again, all the browser APIs, they’re the same.

Adam Stacoviak: Your manager. So here you are at Clerk - and I know Clerk, because they’ve sponsored our show before… I met the CEO, great team there… Super-cool, the way they do auth. I mean, very focused on Next, and the frontend kind of things for when it comes to authorization and this whole entire UI kit… So here you are, you’re working at Clerk… I don’t know what you’re doing, honestly. You’ve got a job, you’ve got several jobs… How do you be a designer for Cortical, Clerk…? Like, how are you doing all this?

Ilya Grigorik: So a lot of hand-holding with partners and getting the developers to adopt all of those APIs. But the benefit of all of that work today is - I’m not going to say we’re done, because there’s still more things to build… But we’re in a really good place, because now all of our merchants are running on the sandboxed primitive that I’ve described, and what we can provide is, first of all, upgrade safety. We can safely roll forward our capabilities in checkout, knowing that customizations that we’ve deployed will not break as you move forward, because we control the bridge, we control the API interface… So if we change the underlying API on our side, we can still provide guarantees about that.

We have reliability. We know that – for example, we saw examples where merchants would inject scripts where a partner would just timeout. So they would have some logic, and for some reason their service goes down, and then the checkout’s broken, because - well, it’s just waiting to render. Like “Shopify, you’ve broken the checkout [unintelligible 00:35:16.24] Actually, it’s your script that you injected of a partner that failed to scale to your flash sale. So now we have assurances about that.

And then finally, performance and security. Another benefit of putting work into the sandbox is it moves all the work off the main thread. So you can’t have code that monopolizes the main thread and renders the UI unresponsive… Which gets back to our web vitals conversation. We [unintelligible 00:35:45.01] better performance guarantees about how the page is loaded, how responsive it is, and the rest.

And finally, there’s a security bit, which is we know that you can’t inject arbitrary content in a top-level page and exfiltrate data… And then finally, you have PCI compliance, because now we have a clean partition where we say “As Shopify, as a platform, we will provide all of the inventory authorization and integrity checks for the first-party scripts that are executed on a top-level page. And oh, by the way, you can totally bring third-party content, but we will execute it in this isolated context, that allows us to punt that problem and not have to worry about all of the integrity problems that happen when you just include it in a top-level page.”

Adam Stacoviak: Yeah, it seems almost like a proving ground for open source at large. It’s like, if you don’t know what open source is, or - Jerod, back to the question you asked before, like what are the hard questions or what are the pushbacks you get from management? I feel like if you can prove “This is how inner source works. Now imagine how it works for the world.”

Our safe world, behind our firewall, controllable, all those things… It’s still unclear to me though how you begin. Is it simply like a private repo on GitHub? Is there a way to dashboard this stuff? How do you truly spark this? It seems like it’s bottoms up, where it’s “I’ve got this thing I think this team or these other teams could use.” I still don’t understand how you manifest inner source. Is it just behind authentication and authorization? Is that what it is?

Adam Stacoviak: I always felt like GitHub Auth was – anything I authorized with GitHub, it always felt like whatever was being asked of the authorization process was more than I thought was necessary in the authorization process. In almost every case.

James Long: Yeah. I need to check that out. It definitely would have helped me a lot if I had one already that was there, that solves some of these problems for me… But I think some of this is a fundamental piece of the architecture, where like people just wanted to build a little client around it, like themselves, and access Actual’s API. I was like “Hey, okay, we don’t have an API.” I ended up building one and you know how it worked? It was the entire app which downloaded the entire data first. And so to boot up the API the first time you run - it takes seconds. Like ten seconds. So very, very slow, and not very tenable.

And the other thing that I was gonna say is things like protected data. So if people wanted to have these roles and authorization, I think that’s something that – I think that that part I think is not fundamental. I think that’s solvable, but it is hard. If you wanted to say like “This user can input this transaction, and I wanna hide it from my spouse, because it’s like for Christmas.” Those kinds of things just got really tricky.

And just these weird things where like, okay, you’re logged out, and you stopped paying for the product, but you still have it all locally, and so people could like keep using it… It’s just like these weird things where like you log out and you log back in as a different user, but the other local data is still there… To me, I was like “I wanna be a startup that focuses on the problem.” Like, solving the actual – and I kept having to be dragged into this weird mind state of “How do I deal with this crazy, weird, mind-bending situation, that local-first?”

So I think that is the benefit of some of the why the community has moved towards this newer model of local-first, where it’s like a really heavy kind of cash where it’s still a little bit more dependent on the server and it’s not completely local-first, but it adheres to the principles where you query your data locally, but it’s not as –

Jerod Santo: It’s now time for Sponsored News!

Top 5 launches of Supabase Launch Week 12

Number 5: They released Log Drains so you can export logs generated by Supabase products to external destinations, like Datadog or custom HTTP endpoints.

Number 4: Authorization for Realtime’s Broadcast and Presence is now public beta. You can now convert a Realtime channel into an authorized Channel using RLS Policies in two steps.

Number 3: Bring-your-own Auth0, Cognito, or Firebase. This was actually a few different announcements: support for third-party Auth providers; Phone-based Multi-factor Authentication (that’s SMS and Whatsapp); and new auth hooks for SMS and email.

Number 2: Build Postgres Wrappers with Wasm. They released support for Wasm (WebAssembly) Foreign Data Wrapper. With this feature, anyone can create a FDW and share it with the Supabase community. You can build Postgres interfaces to anything on the internet.

Number 1: postgres.new – an in-browser Postgres sandbox with AI assistance. With postgres.new, you can instantly spin up an unlimited number of Postgres databases that run directly in your browser (and soon, deploy them to S3).

That’s the top five, according to us. Head supabase.com/launchweek for details on everything else!

And for new users to Supabase head to supabase.com/changelogpod for one month of Supabase Pro for FREE.

Monday: They launched postgres.new - an in-browser Postgres with an AI interface.
Tuesday: Authorization for Realtime’s Broadcast and Presence went public beta. You can now convert a Realtime Channel into an authorized Channel using RLS policies in two steps.
Wednesday: They shared 3 new announcements for Supabase Auth: Support for third-party Auth providers, Phone-based Multi-factor Authentication (SMS and Whatsapp) & new Auth Hooks for SMS and email.
Thursday: They released Log Drains so you can export logs generated by Supabase to external destinations, like Datadog or custom HTTP endpoints.
Friday: They released support for Wasm (WebAssembly) Foreign Data Wrapper. Now anyone can create a Foreign Data Wrappers (FDW) to allow Postgres to interact with externally hosted data and share it with the Supabase community.

That’s a lot! Learn more about all of Supabase’s Launch Week announcements by following the link in the chapter data and newsletter.

Paul Copplestone: Yeah, so it’s easier to think in terms of the product suite. We have a Postgres (obviously) server, and that comes bundled with a bunch of extensions that we need for the platform. These are the things that developers usually find useful; PostGIS, pg_cron, a bunch of the useful extensions.

The second and most popular second product is our auth service, and that, as I pointed out, stores all your users in an auth schema in your database. So it’s very useful, in that you can join your OLTP data to your users, and you don’t have to reach out to a third-party service.

The second one is the file storage. You can store images, videos… And it all stores to S3. It can be Minio or any S3-compatible service, and then it maps the directory into your Postgres server. And the reason why all of this mapping it into your Postgres server is useful is because for the auth service the way that we do authorization [unintelligible 00:19:46.23] is we lean really heavily into low-level security. And so if you want to restrict access to, say, some files, maybe a certain user that you’ve got in your database can only access their Instagram images. You just write a SQL rule, RLS policy, and that will control that access. So now you can see all of these services kind of start fitting together, even though they are individual services.

[00:20:16.21] The next one is an existing open source tool called Postgres. This has been around since the start. It autogenerates an API on top of your Postgres database. It exists as a separate community; we employ the maintainer of this to work just on Postgres. And so what you do is, as you create tables, it’ll auto-generate or introspect your database and generate an API for you, a RESTful API. We’ve also built a GraphQL extension inside Postgres, so you can use it for GraphQL, if that’s your preference.

And then the last two edge functions - these are Deno, if you know it… Deno is by the creator of Node, and it’s a TypeScript-native, almost like Node.js, but it runs in, say, a serverless environment… And I like to think of these actually edge functions as a bit of a strange term, but for us, the way I like to use it, and what we see most commonly being used is a background worker. So you can offload long-running, heavy compute tasks to these edge functions.

As a good example, our last product is a vector offering. And this is just PGVector and a bunch of nice tooling on top. So for example, if you’re trying to store embeddings, or you want to use embeddings, you want to do RAG, anything like this in terms of AI, you can use these edge functions to create the embeddings. And some Postgres services have this, in that you create the embeddings inside the Postgres database. We think that’s probably not always the best idea. Let’s say someone inserts a million rows into your database; you can trigger off a million asynchronous functions to create these embeddings, and then store them back into your database without, say, having really heavy compute.

Kris Brandow: I think part of it also comes from the fact that your backend code is within your security domain, and your frontend code is not, and you kind of have to treat your frontend code as if it’s hostile, in all cases. So I think that kind of lowers the necessity in some cases of needing to be as security-conscious about what’s going on in your frontend. And most of the time, a lot of the frontend vulnerabilities, like cross-site scripting, can be blocked or prevented on the backend side of things as well. But I do think the kind of lack of security awareness for frontend people, especially lack of cryptographic awareness, does hinder in some ways. I’ve noticed that there’s not a huge amount of these, of the cryptography APIs in the browser, where that could be a much better way to do authentication and authorization. Because the browsers had APIs for years now where you can just create a public-private key pair that is locked into the browser, where another script, even on your subdomain, can’t extract that key. And that would allow you to kind of uniquely identify that browser in a cryptographically secure way, and you’re not passing around tokens that can be lifted, kind of no matter what, and someone can run some JavaScript on your website.

[00:46:07.26] So it’s like, yeah, there’s a little bit you can do through cross-site scripting. You can still make API requests, but you can’t lift that key and send it to a server and start making a ton of requests, or anything that. Whereas you can do that with a [unintelligible 00:46:18.05] token. And I think some of the reason that’s not done is because there isn’t as much of a familiarity with these tools and these ideas in the frontend community, so there’s not a lot of support for this type of stuff. I mean, we’re starting to get it with things like passkeys, but that is super-late compared to what we could have had if we tried to start doing this much earlier.

Kelvin Omereshone: For sure. So the way the Boring Stack works is that I provide templates. So for now we have the mellow templates. Because unlike most frameworks, I don’t feel that when someone scaffolds a project in JavaScript land we see a counter. I don’t think that’s a very good use of our time. Because in other ecosystems you get like a full-blown authenticated app, you have models in there, and everything… So we have templates in the Boring Stack. So when you want [unintelligible 00:29:21.05] the project name –react or –vue, you will get the full templates. So if you choose Vue - for today, the Vue template is more up to date. React, I still have to get to like, you know – well, you get like a React-based Boring Stack app, but you won’t get the auth yet, until I work on it. But if – let’s assume you’re using Vue; you get authentication, authorization, and also OAuth with Google.

So if you want to look at the app right now, you just look at the mellow view template, which is on the Boring Stack repo. So that would be github.com/ [unintelligible 00:29:57.08] /boringstack, then Templates, and you’ll see all the templates. They’re all real cool; you could read, and you will understand how the stack works.

Jerod Santo: Kind of a good, quintessential example of like the decision-making process, I think, is something like authentication. And let’s not talk authorization, but authentication of a web application; like, “Who are you?” kind of stuff. And there’s businesses built around people buying this, right? Like Auth0… That’s just one that comes to mind. I’m sure there’s hundreds of them. That one was very successful; I think it got acquired by Okta, and it’s still out there, and people probably use it. And that’s an example where people will say - I think even Ahmad said “You should never roll your own authentication, because it’s a solved problem. You’re solving solved problems, and it’s a cheap one. And so just spend and move on.” And I also take issue with that for certain reasons, but I’m thinking that probably you roll your own auth in some of these projects.

Ron Perris: Yeah, outside of organizations where they run Node exclusively. People do – on the backend they might write their microservices in Go or Python. And I’ve worked with those teams, and I guess what I’m saying is it’s just a different set of problems. Like, you might be worried about things like “Oh, I have, I don’t know, command injection vulnerabilities” or like “I have data store injection vulnerabilities in SQL”, or whatever the datastore layer is; a MongoDB injection. You might have a lot of the flaws in those microservice fleets are related to too much trust between microservices. So kind of all the microservices end up believing whatever they’re told by the other micro services and their request context without validating it. So they might say “Oh, yeah, I guess you must have permission to do this thing you’re asking me to do, other microservice, because we’re all back here on the backend, and you’ve asked me to do it.” And they might not check access control, and they might not do a good job of like authenticating who’s connecting to them… So a lot of the vulnerabilities on the backend are around like authentication, access control, injection in data stores…

And then you have another whole problem space of business logic flaws, which is like “Should this microservice or whatever even do the thing it’s being asked to do?” And sometimes it doesn’t have all the context to know if it’s supposed to be doing what it’s being asked to do. So you have a bunch of logical checks inside of there, and that’s when you get into conversations about “Oh, you should have some kind of generalized access control authentication and authorization framework.” Every single microservice shouldn’t be responsible for like rolling their own version of access control, and being like “If you’re like this, and if you’re like that, then you could do this.” It should be done in some centralized service, where they could say, “Hey, according to policy, I was just asked to do this, and I’m this type of thing. Should I be doing this?” And then they can find out from some centralized place where there’s a policy whether or not that particular thing is allowed by the policy.

Austin Gil: I mean, I would say basically yeah. If you’re looking at object storage solution providers, I would encourage looking for S3-compatible object storage. I think most object storage providers are S3-compatible, because it’s become such an industry standard. And the benefit there, again, is you have portability. You can write your code in one way, and as soon as a different provider, a different S3 provider, or S3-compatible provider offers something that is a deal-breaker for you, or maybe the current provider that you’re using changes pricing, or planning, or how you’re allowed to access your data, and you want to transfer it - cool; you don’t have to change your business logic of your application. That’s really important to me.

Any more on object storage? It’s highly cost-effective, regardless who you use, and also very simple. And the key is - I won’t get into the weeds on the implementation details, but the interesting thing when we bring it back to how you move your files from writing to your server, your application server, to object storage, matters, because there’s a couple of different ways. And the question is how do you get those files into the object storage provider? And I think almost every piece of content that I’ve found out there suggested using what’s called signed URLs, where essentially your client makes a request to your backend, the backend serves as a proxy for the object storage hosting, and the backend will make a request to the object storage provider for a signed URL. A signed URL is essentially the location where the file is going to be uploaded to, but it can only be uploaded to by the person with that signed URL. It’s basically a URL that includes some sort of authentication mechanism, or authorization mechanism, as well as some expiration date.

[54:41] So your backend can provide that to the frontend, the frontend can take that, then upload the file directly to the object storage provider. It doesn’t have to go through your backend at that point. And they can get the file there. That’s what most people say is the best way, and it is a decent way. I personally prefer building my applications with progressive enhancement built in. You cannot do that, I don’t believe, in a progressive enhanced way, because I believe the only way to use a signed URL is sending a put request, which HTML cannot support. So that breaks. But there is a different way, and there’s a couple – I’ll explain the other way first, and then maybe we can look at the pros and cons…

The other way is you can send the file directly to your proxy service. And again, instead of streaming to writing the file to disk, you can pass that stream along and upload it as a stream to the S3 storage location. The big downside there is that - well, there’s a couple of downsides. One, you are basically paying potentially double for that file, because you are going to pay For the file being stored in your object storage, but you may also be paying as your server, which serves as a proxy, is receiving that file - that’s ingress cost; and as it streams that file out to the S3 provider, that is egress cost. So because it’s passing through this proxy, you may be paying more, because of the traffic that’s being run through. Your mileage may vary, check with your stack… Again, you can take this to anyone.

The other sort of advantage or disadvantage is because that file is passing through your proxy servers, now you have more control over what to do with that file. For example, if you wanted to do file optimization, if it’s images, you can’t do that if people are uploading directly to your object storage provider, unless you also pull it down, in which case you’re going through your server anyway. The other sort of downside is, once again, because that file is passing through your proxy server, what happens if someone wants to upload malware? Now, if they upload malware through a signed URL, it’s going to go to the object storage. It’s kind of in quarantine there as long as you do some sort of malware scanning on it… But if you’re passing a file through a proxy service, that malware could potentially be into your system, and then pass through. So then the question becomes, “How do I deal with malware on my uploads?” And once again, I think that this is the advantage – as much as it might be a disadvantage, it’s also an advantage of passing files through your service, because it gives you an opportunity to actually scan those files for malware. Any questions here so far before I continue into malware?

Gerhard Lazu: It just shows that your head is where your heart is, right? …which is authentication, authorization… I mean, if you build a company around it, you really have to believe it that thing… So you’re committed through and through. And that continues to be top of your mind, which is important.

Kris Brandow: And of course, they’re using all the shiny things with Kubernetes, so that’s using Istio. No one actually knows how any of this works. It’s just like, “Oh, this is what we’re supposed to be using.” So we have this big ol’ cluster, and it’s running, and our DevOps people are pulling their hair, because they hate all of this… And I start like reading through the codebase and looking at things, and I’m like, “Okay, these auth policies look a little funky.” And then I go talk to people, and they’re like, “We don’t really have any auth policies. Everything’s just kind of open right now. Like, everything in the backend can just talk to each other. There’s no auth policies.” And I’m like, “Are you sure? Because I see these auth policies in the codebase.” And they’re like, “Yeah, but we don’t think they’re being used for anything.” I’m like, “Okay…” So I just kind of like let it go, and go about my business… And then I have like a few more of these conversations, and I’m like, “This feels weird. But you know, all these people know more than I do.”

And then months and months later, someone stumbles across this one auth policy that has no labels, and no access rules, which in Istio language means that it applies to literally everything, and allows all traffic in. So this one policy had just opened our entire API, including the public API, to the entire internet, for anybody to do anything without needing any authorization. You just needed like a JSON web token that you could easily get from anywhere. And I was just like – so it caused all these problems, everybody’s freaking out, and then they just rip that policy out and they’re like, “Okay, well, without this policy, we’ll be fine.” But then all of those auth policies that had been sitting there, that I was like, “These look funky”, all those took over, and all of those were broken. So then it broke all the APIs. So then they had to put the other policy back, and then go through, and like, go find every single auth policy within Istio, and then fix all of those auth policies, and then they could finally remove that one policy that was opening everything to the world.

I think the total amount of time that the door was just open was about nine months, and to the knowledge that people had, nothing bad happened… But yeah, it was quite horrifying.

Matias Pan: Actually, that’s an interesting one. VPNs. A bank integrates with so many providers, and everybody wants a site to site VPN. And that’s a very specific – so the networking architecture that we propose would solve that problem as well. That also has to be considered, and that’s what we consider also static.

So yeah, Route 53 records for external access, something like the initial part of a Lambda… So Lambda is, in a way, serverless, but you still need like subnets, and a bunch of other things, so that’s also static… I actually have here the document open, so like caches… For example, we use Redis for OTB and authentication authorization… So yeah, S3 buckets, something that you know you need for a long time - all of those requirements we consider static. We want to unify how those things are created. But - and here’s the emphasis - we want to make sure that they can be created. So we don’t want to create like a fully fledged platform that whenever you have a new use case, we have to run around and develop a bunch of new things. No. Whatever we think have to solve the fact that we already identified a list of static infrastructure, but that we can also grow to other static infra that we haven’t necessarily detected.