Jacob DePriest: Well, you know, it turns out that threat actors also have figured that out. And when they hear things like free compute, they go right after it for – pick your abuse vector. We see campaigns that will try and escalate the number of stars that a particular repo has to increase its popularity. We see people trying to mine crypto using free compute. I mean, we see a lot. Hosting files on github.com that we’ll just say don’t have anything to do with software development. And so it’s a challenge. And it’s something that we have a fantastic team working on, we have been employing machine learning and AI for a while in this space, we’ll continue to do that… But it’s a really complex, challenging problem. And the balance we have to strike here is because we serve so much of the world’s software development ecosystem, we can’t turn the dials too strict, because then we start locking out hundreds, thousands of users that have legitimate use cases [unintelligible 01:08:03.10] And certainly, those things happen. You can’t fine-tune every filter to be perfect. But we really try and strike that balance of how we do that.

[01:08:12.15] So this is where we’re also working with our product counterparts to understand what are things we can make changes to in the actual product, or maybe a signup flow or things like that, that will decrease the likelihood of abuse, or impose costs a bit higher to actors who want to do that… And so that’s certainly something – there’s new campaigns every day that come out in this space that our teams are firefighting, and they’re doing a great job at it, and it’s something that’s just top of mind, because it’s something we don’t see getting less of, we’re definitely seeing an increase.

Adam Stacoviak: Obviously, we left out Spiral from the deeper conversation. We touched on the open source nature of it, the Bitcoin wallet, the hardware to mine, all that stuff… But it began at Square Crypto. And obviously, I did some research in terms of where it began, for this, and you had said “What’s the biggest thing we could do for the Bitcoin community?” and one of your co-partners, Mike, had said “Hire five open source developers and just let them do fun stuff.” Can you talk about the inception of Square Crypto and how that’s evolved in Spiral and what you’re doing there?

Adam Stacoviak: Obviously, we left out Spiral from the deeper conversation. We touched on the open source nature of it, the Bitcoin wallet, the hardware to mine, all that stuff… But it began at Square Crypto. And obviously, I did some research in terms of where it began, for this, and you had said "What's the biggest thing we could do for the Bitcoin community?" and one of your co-partners, Mike, had said "Hire five open source developers and just let them do fun stuff." Can you talk about the inception of Square Crypto and how that's evolved in Spiral and what you're doing there?

Toby Padilla: Yes. And all of our frameworks, all of our libraries and all of our apps are Go. And we actually had a lot of questions; I just got a question about it yesterday, “Do you have a Python SDK?” And the answer is no, we don’t. I mean, we’re a small team, so we have limited resources… But we think Go is a good language to build command line tools in, for a number of reasons. One is you get a compilable binary that you can ship, a single file, and you don’t have to force all of your users to install all of the dependencies that you’ve used to develop, a la Node.js or Python or something like that. So it’s sort of a natural fit for this type of tooling. It’s just a nice language; it’s got a really good standard library… So a lot of stuff that you might want to do to make a really cool command line app is just straight up available in the Go standard library. It’s got an HTTP server, it’s got an HTTP client. It’s got a lot of stuff inside of it. So that was why we focused on it.

We like lots of languages, and we come from backgrounds where we – I was a Clojure developer for six years. I was doing Rust actually for two years before I moved to Go… And Muesli - I follow him on GitHub, and I got jealous of all the cool libraries that he was starting all the time. I’m like, “I want that.” Because he’s a Golang developer, and he’s like “Super-cool crypto thing”, or whatever. And I’m like, “Hey, that looks neat. I’m in Rust, writing my own bencoding library, because such a thing doesn’t exist.” At some point I just thought “You know, I love Rust, it’s really cool, it’s a super-exciting language”, but I jumped ship. I’m like, I wanna be more productive, I wanna use all the fun toys, and so I started doing Go.

[52:19.15] And so some of us enthusiastically, some of us begrudgingly, have moved to Go over the years, just because it is kind of the answer to building these type of tools. So when people ask us, “Hey, can I do a Node.js version of your stuff?”, our answer is “No. Use Go, but come into our Slack and we’ll help you.” So we’re very enthusiastic about onboarding people into Go, teaching people about Go. We’ll answer any question, no matter how basic, about Go, in our Slack. So Charm.sh/slack. Come on in and ask us about Go, because we love to answer it, and we’d love to see more people adopting it, especially for command line stuff.

Toby Padilla: Yes. And all of our frameworks, all of our libraries and all of our apps are Go. And we actually had a lot of questions; I just got a question about it yesterday, “Do you have a Python SDK?” And the answer is no, we don’t. I mean, we’re a small team, so we have limited resources… But we think Go is a good language to build command line tools in, for a number of reasons. One is you get a compilable binary that you can ship, a single file, and you don’t have to force all of your users to install all of the dependencies that you’ve used to develop, a la Node.js or Python or something like that. So it’s sort of a natural fit for this type of tooling. It’s just a nice language; it’s got a really good standard library… So a lot of stuff that you might want to do to make a really cool command line app is just straight up available in the Go standard library. It’s got an HTTP server, it’s got an HTTP client. It’s got a lot of stuff inside of it. So that was why we focused on it.

We like lots of languages, and we come from backgrounds where we – I was a Clojure developer for six years. I was doing Rust actually for two years before I moved to Go… And Muesli - I follow him on GitHub, and I got jealous of all the cool libraries that he was starting all the time. I’m like, “I want that.” Because he’s a Golang developer, and he’s like “Super-cool crypto thing”, or whatever. And I’m like, “Hey, that looks neat. I’m in Rust, writing my own bencoding library, because such a thing doesn’t exist.” At some point I just thought “You know, I love Rust, it’s really cool, it’s a super-exciting language”, but I jumped ship. I’m like, I wanna be more productive, I wanna use all the fun toys, and so I started doing Go.

[51:57] And so some of us enthusiastically, some of us begrudgingly, have moved to Go over the years, just because it is kind of the answer to building these type of tools. So when people ask us, “Hey, can I do a Node.js version of your stuff?”, our answer is “No. Use Go, but come into our Slack and we’ll help you.” So we’re very enthusiastic about onboarding people into Go, teaching people about Go. We’ll answer any question, no matter how basic, about Go, in our Slack. So Charm.sh/slack. Come on in and ask us about Go, because we love to answer it, and we’d love to see more people adopting it, especially for command line stuff.

Rod Vagg: That’s just right, and that brings up an important point, which I haven’t touched on at all which is the importance of leadership. It’s nice to have this idea that you’re gonna have people show up, you’re gonna have this emergent property of leadership, it’s just gonna emerge out of the mass of people doing things. That’s a nice idea and sometimes it works, but very often you still need leadership, and that leadership may be in the form of people putting in more effort than others.

[01:02:01.17] That’s what happens early on in a project - you’ve either got one person or a few people who are passionate enough about making this thing work, that they are willing to put in that much time to get it there; rarely do you have a lot of people doing that. Usually, there is only a small group or only one person, so that’s what leadership looks like initially.

As the project evolves you need somebody or some people to serve in that role of facilitating the community, and that comes down to the fact that not everyone is a leader and we ought to be fine with that, that not everyone wants to be a leader and not everyone has those natural skills to be a leader, and when you accept that then you see that there is a place for individuals to step up and just start being the grease in the wheels. And that can play out in really interesting ways; it can play out just by the fact that somebody is stepping up. Very often, other people will be looking for that person, and they’ll be waiting for it, and they step up and they might start to assert things, and other people involved in the project might say “Phew, finally somebody is here to take that burden.” And we can disagree with them, “Great, at least we’ve got someone to disagree with now. We don’t just have this mess of people that are too timid to really assert thing”, so having somebody stand up and start asserting can be a really valuable thing and people look forward to it.

The other way that it can manifest is simply in the respect that people get over time, and that’s how leadership can evolve in these projects. It evolves in open source in general; you build up this capital of respect by your contributions and by what you know and what people know that you know, and you see that in complex projects like Node. Node has a lot of different areas that are all very distinct and there are different levels of expertise in different areas that are required to understand them and nobody – there’s zero people in the Node Core project, amongst that group of people, that understand it all and that are experts in it all. Everybody is an expert in one area or a few areas, and nobody is an expert in it all.

And the way this kind of leadership evolves in that setting is that people build up the respect and the technical capital from other people that they know this area and that they are somebody to be listened to. One of the most obvious ways this plays out at Node is when you think about Crypto; it’s a really hard area for people to get across, very few people really understand it enough. We have a few people that are really well respected in that area, and we will generally defer to them, and often for even the most trivial things; we’ll say we need sign off from these people, or they need to be involved in this discussion in order for it to move forward, and then when those people in that small group - or it might be just one person even - they start proposing changes in their little area of expertise, they almost get a free pass because everyone looks at them as the leader in that area.

So that plays out really interestingly when you’ve got a complex enough project, but it also plays out in smaller projects when you’ve got simply somebody who has got obvious depth of technical knowledge and understanding about a project; they are generally recognized and they get to play that leadership role, as well.

Erik St. Martin: So Go projects. I have one that I saw come through - I think I saw it on Twitter a few days ago, but it goes along with this whole security mindset that we’re talking about this episode, which is Hewlett-Packard released a library called gas - a command line tool, anyway - to statically analyze your code for common security or vulnerabilities, and some of which were actually validating the TLS ciphers and protocols within your project, and then there was some SQL injection vectors it looked for, and I think using some crypto primitives and stuff that were weaker. There is a whole slew of things, and it will actually be interesting now that this is here to see how many more security checks people add to the code.

Bill Kennedy: Now, when we talk about Go - one of Go’s language design philosophies was being able to do more with less code. It’s coming from these – I believe. I can’t speak for Robert, and all… But I believe it’s coming from these ideas that if there’s less code you need to write, there’s going to be less bugs. Now, why did they say this? Because Stroustrup says “If you’re writing more code than you need, it results in ugly, large, and slow code”, where ugly means you’re leaving places for bugs to hide, large means you’re ensuring incomplete test coverage, and slow means you now start to make shortcuts and dirty tricks away from your frameworks and your patterns, because you’re moving fast and the code gets out of control… And these things absolutely happen.

So I think we’re talking about all of this… It all works together, and I think Go is tied into that. And we complain about Go’s error handling. We love to complain about it. But do you know that there was a study done where they looked at 48 critical failures that brought down systems. Hundreds of bugs in Cassandra, HBase, MapReduce, Redis… How many systems run on Redis today? And they’ve found in this study that out of those 48 critical failures, 92% of them could have been avoided if error handling was done better. Failures from bad error handling.

So again, I think that Go designers knew this. They knew this, because they were developers themselves. They were not necessarily academics. They had to build software. They knew what the average developer needed, they knew where they were falling down. And I think Go comes in and solves these things.

[36:15] Personally, I think when somebody complains about error handling in Go, they’re complaining about – they want it easier to do, not easier to understand. [laughter] We come back again, right? So sometimes when you make things easier to understand, things have to be a little more tedious.

But here’s another design philosophy, Johnny? Two of them. One, you shouldn’t be writing code for yourself. You should be writing code for the next person that has to come along. Because if you don’t, if you’re not thinking about the next person and/or the average developer on your team, when you leave, that codebase leaves with you. It gets replaced. And the 3, 4, 5 years you spent on that ends up resulting in meaning nothing. I’ve got code that’s 20-something years old, 10-something years old in production right now. The 20-year-old code should go, because that’s way too long… But I think it’s there because I always wrote code with the understanding that somebody else has to be able to maintain this. It wasn’t about me, it was about the next person, and that allows that code to now not just have to be replaced, right? You need to have that design philosophy in your head; you need to be thinking about that, “Who’s the next person that’s gonna come along here?” And then you’re always writing code for the average developer on your team.

If you’re the average developer on your team, that means I can wake you up at 3 in the morning (God forbid) if I have to, and you can handle the bug. That’s the average developer. If I can’t wake you up at three in the morning, then you’re below average. So another question is “Why are you below average? Is it because I’m failing you, or are you just not coming up to speed?” And then for me, the next thing is the above-average developer. That’s scary, because those are the developers that tend to get bored, and instead of being able to write for the average developer, or bring the team up - that’s where the clever code comes in. That’s where we trip up.

And I tell people all the time, “When you’re hiring, evaluate who this person is for your team. Are they below, are they average, or are they above?” And consciously understand what you’re gonna need to do as an individual and a team to get this person in the right place. If they’re below average – which is great; let’s hire developers who are below average for our teams, so we can bring them up and we can create a stronger team. Those are the best developers in the world, because you can really teach and train them. And now you’ve got somebody who will stay a long time and really work hard and thank you for the opportunity.

But if you put me on a team that’s doing business APIs, I’m above average. If you put me on a team doing crypto, I’m below average. And if I wanted to learn crypto and you gave me that chance, I would be ecstatic, and I’d work hard, and we’d get there. But if you’re hiring somebody who’s above - and I’ve done it before - they can either be amazing mentors and coaches, which is why you’re hiring them, I hope, or they can create utter chaos and destruction, because everything they’re doing is not comprehensible to anybody else on the team, and you’ve gotta maintain it.

[39:40] So those are design philosophies around building teams, around the ideas of all of this stuff. And you wanna apply it back to micro-level decisions, like constructions, functions versus methods, to macro-level decisions around app layer, business layer, foundation layers of code. Policies for these. Import policies. Error handling policies. Who can shut down an application? Who can’t? Who can log? Who can’t? Who can wrap errors? Who can’t? Who can set certain import dependencies? Who can’t?

And you don’t have to have all of it day one. You have to develop it as “Suddenly, there’s a hole in the engineering decisions. Hm. We don’t’ know what to do here. Okay, that means we may not have a design philosophy here.” I get excited when that happens. I’m like, “Oh my God, we’re gonna have a design philosophy for this. Oh my God, we get to do something new! WOOOH!”

Now, you’ve always got some of your base, foundational, right? But those are exciting days. And it’s also exciting sometimes when somebody finds a hole in a design philosophy or policy, where we thought this was the right thing to do, and suddenly we’ve found an exception. And there’s exceptions to everything. There are some exceptions you just can’t take. I don’t really take exceptions between project layers. I’ll never let the foundational layer log. There’s no exception to that. If you have to log, you’re in the business. That’s it.

But then there are other exceptions… Here’s a good one, Johnny. Here’s one where you might take an exception. So baseline design philosophy - a type system is not to be shared. A type system exists to allow package, which is a unit of code in Go, a clearly compile-time unit of code. A type system is design to allow data to flow in and out of the package API, where a package has a purpose. So if the type system’s job is to allow data - if. There’s my philosophy. If you agree with this. You don’t have to agree with anything I’m saying today, by the way… It is totally fair. But if you believe that a type system’s job is to allow data to flow in and out of a package, then that type system is highly localized to that package and that package only. So now you have to make a decision about every API. When it comes to data flowing into an API, you have two choices. You could say “I want the API to accept data based on what it is.” This is what I would call concrete functions, accepting a concrete type. It can accept a user, and only a user. That’s what it is. But thanks to interfaces, we can write polymorphic functions let’s say “No, this API will accept concrete data based on what it can do.” And that’s a next level of refactoring, hopefully; I don’t wanna start there, but suddenly you realize “Not only can I work with a user, I can also work with a customer. Based on this common behavior, we make it polymorphic.” Okay. We all agree with that. You have both choices, and those are the only two choices you have. And those types should exist as types within the scope of that package.

Now, here’s where the fun begins… I have a strong rule that functions should only return concrete values. The function’s job is not to pre-decouple or wrap concrete data already in an interface; that is not the API’s responsibility. It is the caller’s responsibility to decide whether or not they need the decoupling or not. Not mine.

So minus the error interface, which is a whole another set of interesting design philosophies and things I have, I don’t wanna see a function that uses http.handler as the return type. I don’t care if you know or think they’re gonna put it into a handler already. I don’t care, it’s not my job. My job is to give them the concrete value that they can then do with what they want.

“Well, Bill, then we’re leaking a –” No, you’re not leaking a type. They already imported your package. There’s no leaking there, what are you talking about? Stop trying to abstract for the caller. Let them do it. Now, there are two exceptions to this. One is the error interface; we’re handling errors in a decoupled state. There’s lots of reasons why we wanna do that.

[44:04] And until 1.18 comes out, there are times where you might need the empty interface. It should be a little bit of a smell, but let’s be real, I’ve had to write a function or two over the last six years where I was trying to be, for whatever good reason, generic. Maybe I was just doing some data flow… And we were using the empty interface, which - now in 1.18 we’ll be able to replace with a concrete type. [laughter] I mean, what is generics at the end of the day anyway? Generics is concrete, polymorphic functions, where the polymorphism isn’t happening at runtime, the polymorphism is happening at compile time. We’re choosing the concrete type, the data – because the only data that flows is concrete data anyway. We’re just choosing that at compile time. For me, it’s concrete polymorphism, as opposed to runtime polymorphism.

But there’s a philosophy - we shouldn’t be using the interface as a return type, minus those two exceptions, when they happen. And people disagree with me there, but… There it is. So if I see a function that’s returning an interface, it’s immediately code review style, so “What are we doing? Why are we doing this? Prove to me that we need to take an exception”, but it’s gonna be hard, because if I return the concrete type, that doesn’t prevent the caller from doing whatever it is they’re doing.

Joseph Jacks: Yeah. Some disease. I’m just going to say tweet. I came up with one phrase – because people ask me this question all the time, “What is the thesis?” It’s so strange. And the phrase - I want to say probably late 2019, maybe even early 2020 - is “Open source is eating software faster than software is eating the world.” So that’s been the phrase. And that’s kind of been the slogan, and the whole mantra or whatever. And I think it does really well tightly kind of articulate what the fund is about, which is like, it’s easy to explain things to people when you can kind of build on an existing, well-understood dynamic or concept or whatever… It’s sort of like reasoning from analogy, which I think often is not that helpful, because you’re not really deeply understanding something if you reason by analogy. But Mark Andreessen is very famous for saying software is eating the world, and this is kind of the Andreessen Horowitz thesis, right? And they’re very known for that, so people have a strong understanding… So I was just kind of like trying to build on that to say “Well, actually, we think instead of software eating the world, which is kind of true, we think something more important and more kind of disruptive is actually happening, which is that open source”, which really means something very specific… It’s like a really like kind of specific phenomenon, and important… “Open source is eating software itself at a rate that is faster than software is eating the world.”

[20:23] So I’m beating the drum on this for - probably another year or two years go by… And then as I was raising our second fund, which was sort of like 2022, 2023, I started to visualize this. I started to kind of like write down… I think I was like having breakfast one day or something, and I was just trying to write down two waves. There’s a really big wave, which is like the software eating the world wave, and then there’s a smaller wave, which is the open source eating software wave. They overlap. They’re sort of super-imposed on each other.

So the big wave is like this huge wave. And then if you look at the horizontal axis, it’s time. So a number of years, a number of decades, or whatever… And then the vertical axis, kind of up and down is the size of the market, and how much money, how much industry size kind of has been to sort of like eaten; how much of software has eaten the world.

So it’s like tens of trillions of dollars or whatever on the vertical axis. And then on the horizontal axis, it starts around 1970, maybe 1960 or something… And then sort of the wave kind of goes – Oracle, Microsoft were founded, and then what other software companies and software industry were sort of created… So it’s like this line that develops, and you sort of put logos on it… And then the bottom wave is open source eating software. So how do you visualize that? But again, it’s in the context of companies.

And so obviously, I put Red Hat in there, and I put JBoss, and whatever. A bunch of other companies. And then as time goes on, you’ve got like MongoDB, and Talend, and MuleSoft, and GitLab, and HashiCorp… All these other companies. Databricks… And there’s more and more. Odoo, Grafana… There’s a lot of these companies now.

And my thesis in visualizing this extremely oversimplified image of the slogan, the visualization of this slogan, was that the rate of growth of the small wave, that small curve - the rate of growth is faster and expanding at a faster rate than the rate of growth of the large wave. Because the rate of growth of the large wave has sort of hit this kind of like asymptote or sort of like scale in which you can continue to digitize more industries, and like software is eating more and more things… But it’s almost pervasive at this point. Yes, there’s software for so many parts of the world… And now if you really look at the fundamentals of software and sort of like what it really means, there’s two ways of building and sort of releasing software to the world. You can build something that’s proprietary, where the person who created the technology or the software is solely in control of the ability to make the software better, and improve it, and dictate can you trust it or not… The other way is open source.

And if you look at open source in the context of Linux, there’s so many reasons why Linux succeeded in basically becoming the dominant server-side operating system. By the same token, there’s so many reasons the Android smartphone platform succeeded on smartphones just in terms of the number of smartphones. There’s many reasons why languages and databases are increasingly becoming dominant as open source open platforms.

Now we see things like LLaMA, which is actually not technically open source, but I think there’s some forgiveness that we have to kind of offer there, because neural networks are not source code. They’re a different artifact. And we’ve blogged about this and stuff last year, actually.

[24:04] So the idea, the principles of open source have remained unchanged. There’s a lot of resistance to that in the world of capitalism, and the world of business, particularly by financier types and VCs. I don’t consider myself a VC. I consider myself something a little different, but technically my job is as a VC. But we have a philosophy, we have a view that the world across all technology categories, starting with software - but it’ll eventually sort of like permeate across other things, too - is moving more towards transparency and systems that you can actually trust and verify that you can trust them, and systems that are user-empowered, user-owned, user-controlled, and permissionless. And those things are totally orthogonal to any particular type of technology. You don’t need to be a blockchain, you don’t need to be crypto to have those properties. You do need to adhere to a certain set of principles though.

And so that’s kind of structurally what the fund is about. How we’ve been sort of prosecuting these views has kind of been relegated to software company startups, and open source companies, largely, with a handful of exceptions… But I plan on doing this for the rest of my life, and it’s extremely fulfilling and comforting to see more and more evidence of the thesis actually playing out to be more and more correct. I mean, I can certainly imagine a world where I wake up every day for years on end, and the world is just more and more proprietary, and more locked down, and more centralized, and there’s less hope of any of these ideas having any long-lasting potential. That would be really depressing. [laughter] But so far, I see a lot of evidence on the other side, so it’s kind of cool.

Break: [25:55]

Nathan Sobo: But just from a personal perspective, it’s not like I surveyed the landscape and was like “Let me find a low-hanging fruit to pick.” It was much more the perspective of “I’ve been trying to do this for 10 freakin’ years, still haven’t managed to do it, and nobody else has either, and so I just have to keep going.”

Now, you’ll have to talk to our investors for their particular motivations, but I can tell you what I told them, and really believe, in terms of what the market opportunity is today, in terms of - again, I think if I were in this for the money, I probably would have started like a crypto token, or something. That would have been a faster route to riches. But I do think it makes sense to build a business model around this.

And for me, the opportunity is really about how we all communicate around code. I mean, I worked at GitHub for nine years. I love GitHub, but I also don’t feel like there’s been substantial innovation, since pull requests came out all those many years ago. And I think it made sense at the time to kind of hang this social layer on top of the version control artifacts. I just think we can take it much further. I mean, that’s what I pitched Chris on, to kind of get hired at GitHub, and we didn’t manage to pull it off… But now I think, based on everything we’ve learned, we’re actually positioned to do that.

[44:00] And so what I really want is a world in which having a conversation about any line of code, whether it was written a couple of years ago, or I just wrote it and I haven’t even hit Save, is something that just feels like at my fingertips; I can @ mention a teammate, pull them in, and start a conversation, so that conversation is really growing over the entire codebase. Because again, you want to introduce a new feature - that probably interacts with some other layer of the system that you may not understand. Already, you need to start having a conversation. But do you do that on a pull request? You haven’t even written one line of code yet. You’re just trying to understand what’s going on.

And so there’s just so many things like that, where it would be a great idea to have a tool that really facilitates interaction around code, and it just doesn’t exist. Like, I don’t know what we did; we’d like paste code into Slack, or like cap things inside of backticks. You’re talking about code that isn’t even there, and then it scrolls off the screen, and it’s gone forever when someone else comes to the code and has the same exact question that you had. Or are you hopping on a Zoom call and now one person’s like dictating through the screen, like “Oh, no, no, no. Okay, open this file; okay, now go to this function…” Whereas already in Zed we’ve tackled the real-time piece of it.

When we want to just have a conversation about code, we’re doing that in Zed. Start up voice, that’s not in Zed yet, so we still rely on an external tool to do the voice for us. But then I’ll just open up a submenu, invite one of my teammates in, and we’re following each other around inside the code, instead of trying to dictate through the screen, like “Go here. That – take that”, because the latency is so high, or whatever. I’m just moving around, and they’re following me, and then they’re moving around and following me…

I remember, when Mikhaila, an engineer on our team - she was like integrating the terminal into Zed; she was interacting with GPUI, and there might have been some APIs missing; she wasn’t quite sure. So we had a conversation, and I toured her around inside the GPUI, which I wrote the majority of, and then I followed her, and she toured me around inside of the terminal code, and then I got a sense of what she was trying to do. And then I jumped back into GPUI, and we wrote the code, added the methods together that needed to exist for her to accomplish what she was trying to accomplish. Then we hop back into her code, which she knew much better than me, and used those methods. And within an hour, with a quick conversation, we accomplished what over pull requests would have taken a week, I feel like.

And so that’s the level of fidelity that we’re looking to bring to interaction around code, starting with real time, but not confining ourselves to that. That’s, I think, the innovation opportunity; it just so happens that the primo kind of iPhone, Apple vertically-integrated product that delivers that experience in my mind is the tool in which the code is written itself. That you shouldn’t be Command+Tabbing out to a browser; that that experience should be tightly integrated directly into the authoring environment, just like it is with Figma, or Google Docs, or these other disciplines, that the place to collaborate and talk about code is where you write it. So we have to build a great code editor. But luckily, I want to do that… So that’s our competitive insertion, and that’s the business part that we want to build.

If you want to use it by yourself, I want you to do that and do it without paying us a dime for it into the future. And I want to build a business around teams using this tool to be more productive by having a better mind meld with each other and being more effective in our communication. That funds the people that just want to use it by themselves for free, and drives more innovation into the code editor space than has ever been possible, ever been funded.

Matias Pan: I think it’s interesting, because it’s a context that probably a lot of people are finding themselves in. A lot of companies went through really crazy hyper-growth, and then all of a sudden the bear market hit, and they were left with this application that grew very rapidly, with a team that probably didn’t know much or at all about infrastructure, and a really big team, but not really all that amount of money that they can use to continue growing. And when you get to that point and you want to continue making your product better - because you’re not reaching hyper-growth, you’re reaching for the classical breakeven, or in a way making your product profitable - you will have to make a lot of decisions regarding your architecture and your infrastructure, that they all have to keep in mind that what you want to do is reach that breakeven. You want to actually be able to continue to deliver value to users, and hopefully, that value gives money back to you, and you can keep the company running.

So when I joined Lemon - Lemon Cash is the full name of the company - that was the exact context in which the company was in. There was a really, really big, monolithic codebase, that had so much code, from so many different domains. We’re talking about a bank/crypto, so it has a lot of things related to a bank, but it also breaches you into the crypto world. And not just like tokens, not just like currencies, but also like NF Ts, and the entire gamification that the crypto industry is trying to bring.

[05:53] So this was a big up. A lot of very clear, separated domains in this single place, and the entire infrastructure in which this was laid on was not something that happened on purpose. So it wasn’t far off. It was “Hey, we have to get this to production now. What do we know about it?” “Not much.” “Okay, there’s this thing called ECS, that they’re saying it’s serverless, and they’re saying it’s super-easy, and there’s this blog post that a big company wrote. Let’s just use that.”

And then when I joined, the company grew from 5 developers to 300 people, which brings a lot of really different problems where you’re working on that monolithic codebase… And the idea of going to a service-oriented architecture was kind of natural, in a way. And when I say natural, I don’t mean that it’s the obvious choice. It’s going to depend on what context you’re in. But we’re talking about a bank that has the most critical thing at hand, which is money. And you always want that bank to work perfectly. So for a bank, the idea of services allows you to separate the failure domains in a very clear way, and make sure that the product is the best thing that you can do.

So yeah, the decision itself had already been taken, and when I joined, they were already discussing about going into a service-oriented architecture. But there were no discussions around the infrastructure side of going to that architecture. And for me, at least, that’s the biggest problem you have to at least think about and solve in part, before actually committing to that. It’s perfectly fine to start separating the code, understanding the domains and doing that exercise, but when you want to actually execute that, solving the infrastructure part, the observability part, the monitoring, how you get things to production, how you operate on those services, how do you have on-call policies - everything related to having a service in production also needs to be discussed.

So that’s where this proposal that you mentioned came to be. It tries to tackle how you should be deploying services, at Lemon specifically - that’s why it’s in the title - and it puts that in the context of all these other things that we also have to solve in order for the company to operate services gracefully, I would say, in production.

Alexander Neumann: Initially, I started with the 0.something releases to be able to at some point say like “Okay, at this point we break the repository format and we add something or change something that’s not backwards-compatible.” But this hasn’t happened the last couple of years. The most important thing that I think in terms of the backwards compatibility for the repository would be to add compression. Restic does not support compression yet, because the data would need to be compressed before it is encrypted by Restic. So you need to have something built into Restic.

I can see a way how to add compression to a repository, but this would break compatibility with prior versions of Restic who don’t know about compression. Once we add that, i can think that it would warrant a release of 1.0, or something like that… Like, okay, before we had everything was compatible, and you can even use the newer version of Restic to restore from old repositories… But whenever you initialize a repository with 1.0 and have compression support, then you cannot restore with an older version. We also have a version field in the repository, so Restic can even give you a nice error message that it’s unable to understand the repository format because it’s too new. So this would, I think, warrant a 1.0.

Unfortunately, changing the repository format can open a can of worms, because there are so many things that could be made better… And personally, I started working on this, but I’m not sure where to stop. Like, do I just add compression, or do I also add support for error correction? Whenever you have a file where there’s a one bit flip and you cannot restore this chunk in this file, because there’s a bit flip, and the signature doesn’t match anymore, and Restic says “Okay, the ciphertext verification failed because something is wrong” - it would be nice to have, for whatever correction, as Reed-Solomon called, for example, where you make every file 10% bigger in order to be able to correct one or two bit flips. This is an interesting feature to have, but does it warrant another repository version, or do we do this in one step? And I’m not sure where to go from there.

[01:00:04.18] On the other hand, changing the repository format is not an easy thing to do, because you have to keep so many things in mind… And until somebody else steps up and really does that to my liking, I don’t think we will get that for now… But I hope to find the time in the future to really do that. I’d really like to do that. I’d like to add compression.

In the beginning I didn’t add it because there were concerns by several users with also a crypto background - crypto means cryptography in this case - that adding compression would mean to increase the attack surface that attackers can use. For example, there were several issues with compression in the TLS protocol, which is something different, because it’s an interactive protocol and sometimes they have a man in the middle/person in the middle modifying the packets as they go… This would be a bit different with a repository, but there’s also like – yeah, I can access a repository, the attacker changes something, the next day I access it again… So you have some kind of back and forth.

And the other thing was that at the time I designed the repository format there was no great compression algorithm already baked into the Go standard library, or available as an external library… But this has changed, because there is a person called Klaus Post; he’s working in Copenhagen, and he does all kinds of completely crazy stuff with compression. He’s also a Restic user, by the way, and there’s this issue 21, which is infamous, and I’ve locked it for now, because there’s so much discussion about “Shall we add compression to Restic?” because the answer is obviously yes, but people tend to get distracted by discussing the merits of different compression algorithms. This is the classic bikeshedding problem…

Alex Sexton: Yeah, so PCI determines all of the algorithms for how you must store credit card numbers and things like that. I have a pretty good guess on what it is, but I’m not even credentialed enough to touch or look at any of that code as an early employee at Stripe. That’s another one of the security precautions that PCI mandates. It is mandated by a body, but as far as all of this auth goes, I feel like maybe my security brain is coming out a little bit… The way that your password gets hacked is not hashing algorithm collisions currently.

This one’s bad - I don’t think too many people are using SHA1. Even if you use HMAC or SHA1 it’s fine. There’s even ways to make SHA1 fine, but use bcrypt to do passwords… Actually, my number one recommendation is don’t implement any security stuff yourself. Use libraries that are well known and well tested. The number one rule at Stripe is don’t implement your own crypto; don’t invent your own crypto, because you have not thought it through correctly. That’s my advice.

Kelsey Hightower: So one thing I have to realize is that I think a lot of times when people get money, they believe that they’re smarter than other people that don’t have money. So they think that their ideas should rise at the top. They’re obviously right about something, because their bank account reflects just how right they are. That’s the mistake. To me, I know that I’m lucky, and a lot of things could have went in multiple directions, and it doesn’t end up with this particular number. And so one thing I’ve challenged myself for the last maybe 20 years, “Who am I, and why do I believe what I believe?” That’s it. Sometimes I believe things because I didn’t have time to research them, so I had to just take them at face value, they were taught to me, I didn’t have time to challenge them, so I took them.

So these days, when I look at it, I try to humble myself, and be like “Kelsey, do not fall into the lifestyle inflation. Do not over-index on showing people what you have. It is okay if people don’t think you’re in the top X percentage of anything.” Remind myself that “Yes, you could buy it, but you don’t need to. What’s caused you to want to do that kind of thing?” And that helps me avoid the conspiracy. For example, if you say “I need more money”, you will be really mad that you pay taxes, because people are taking away the very thing you want. I don’t mind paying for the bridges, the highways, the medical stuff…

[01:02:05.01] Do I like the waste? Absolutely not. I’ve seen waste in enterprise, so I’m not surprised that there’s waste in government. So I don’t over-index in that, because I also know that people are hard to be very good 24 hours a day forever. It’s not happening, folks. So I’m a pragmatist when it comes to that stuff.

But the conspiracy theories - whenever one pops into my mind, I say “Kelsey, it’s time to do some research here. Before you start running your mouth, before you go on Twitter just saying random stuff…” For example, when I wanted to understand inflation, I had to go read The Monster on Jekyll Island, how the Federal Reserve was formed… Buy some, Treasuries actually buy Treasuries from Treasury Direct, look at the interest rates, look at the curve… Why do people start feeling a certain way when it goes up, start feeling a certain way when it goes down? Who buys them, who doesn’t? How big is the Treasury market versus the stock market? Treasury market’s dramatically bigger. Why is that?

So these are the types of things that I go into to just try to resolve the conspiracy components with actual facts. And then I just talk to people. “Hey, explain to me how this works, please, because I don’t know.” And then people give me insight, give me something I don’t know, they close the knowledge gap. And then I’ll test. So if I’m really afraid of something, I’ll test it. When the whole crypto movement was going, I was so afraid, because I saw [unintelligible 01:03:25.25] It doesn’t matter if things are logically correct or not. It don’t matter. If the world decides that we should all jump off our roofs, with no helmets, then just people start doing it. It’s like “Oh, what are we doing?”

Jeff Lindsay: Yeah, so a lot of kind of function as a service is usually designed around HTTP, so you’re writing web handlers and you can kind of shell out and do whatever if you have containers. Usually, these things aren’t made to work for – you know, let’s say you have a script. A lot of people’s automation in their companies is based on Bash, or various scripts, and in order for people to run those scripts, they have to download the script, or install it as a package, make sure you have all the dependencies, and of course, you’re not running in containers because you’re talking about people’s development environments… So everybody has to have the right dependencies, and then if there are secrets – like, let’s say your tool is gonna go do stuff with AWS and it needs your AWS credentials and you have to share your credentials with everybody for it to work… So this was a way to basically say, “You can put that script in the cloud… It’ll run in the cloud, it’ll run with secrets that you specify that nobody else has access to, and you can run those commands with SSH.” So you can say “SSH my deploy site command” from your company or your app or whatever, and it’ll go and deploy from anywhere.

They are stateless in that you don’t have – but you can pull from Git, you can kind of do anything you want from the command line. However, one of the things that I’ve been working on for a while is a way to expose your current directory to it. That will open up the possibility of writing tools that actually interact with the file system and particular files in your project.

Again, it’s kind of like the speed to get somebody working on your project, have all the right tools and stuff like that - you can actually put all those in the cloud, and then they kind of run a command… And it’s working on your file system, but the code is executing somewhere else. So it’s run consistently, and any secrets or passwords are protected, and you can share access to commands. So it’s a really powerful – it’s a dumb primitive. I did it because it’s – it’s not like, you know, the most amazing primitive, but there are use cases, especially when it comes to groups, trying to automate things like deployment in kind of an opsy world… And you could easily throw a Slack interface onto it, and now you have ChatOps type of stuff.

[44:25] So function as a service, focusing on command line… That’s really what it is. And yeah, it’s a great tool. So I was gonna rebuild that using this component; it’s using an older version of it, and isn’t as componentized as it could be, but this is a way that people can kind of see “Oh, here’s how you’d build an application or a system that’s fairly complicated (not super complicated) from scratch, using these kinds of components.” And a lot of those components will go into future projects as well.

A common one that we have besides HTTP is SSH. We have a really great SSH library for Go, people don’t know… GliderLabs SSH. It wraps the existing crypto SSH library and gives you an interface that looks a lot like the HTTP interface for servers. So it becomes really easy to build SSH servers and do stuff with SSH… Which is a really cool protocol by the way, because it can do connection tunneling, and all kinds of neat stuff. As a protocol it has a lot of interesting primitives inside it.

Brad Fitzpatrick said he thought that it was dope, so I think that’s good. He said it’s refreshing to see good API design… I was like, “Well, I’m just copying HTTP, so…”

Paul Vixie: I think that that problem would be avoided. Earlier I talked about how in the early days of the web we didn’t have crypto that was fast enough to support – the crypto hardware fast enough to support HTTPS, and so we just sort of didn’t do it… Until you needed it, and then you bought some hardware to assist you with that. And nowadays, I’m thinking about a number of different NIC vendors who make PCIe x16 cards that you can stick into your server, that will do a lot of the offloading for you. Check sum computation, segmentation, reassembly, and so forth.

[00:42:05.27] Now, if you need to operate at 100 gigabits a second, or coming now soon 400 gigabits per second, if your CPU has got other things to do than shoulder every octet through the bus, then you can get a very smart NIC, and driver support for it, and - you know, as we all know, Moore’s Law will continue giving us its annual gift, and the time will come when you don’t need that hardware anymore, and you’re just doing that with your CPU, because you have so many cores, so much cache, and so forth.

So I think any protocol, in order to succeed at all, would have to be the kind of thing – it’d be very difficult in the short term. But in the long run, it’ll just be the way everything works. And so I don’t see that hardware support is going to be called for in any of this. What will be called for is some hard decisions. I mentioned earlier that fragmentation kind of doesn’t work, and it got worse on IPv6. It worked a little bit in IPv4. It doesn’t work at all now. And packet size [unintelligible 00:43:14.10] on our WiFi is Ethernet cells. [unintelligible 00:43:18.20] And that is 1500 octets. And I knew one of the people whose name was on the Ethernet patent. He was my mentor at a mini computer company back in the late ‘80s. His name is David Boggs. And I had an opportunity to listen to him talk about the old bits, talk about being at Xerox and inventing Ethernet, and [unintelligible 00:43:45.20] And so I’m in a position to know secondhand that the intent was the packet size would continue to grow, so that as we got faster at networking, we would also get larger packets. And he was genius in a lot of ways, in this and every day, but his idea about this was every time the clock rate gets you 10x, in other words you go from 10 megabit to 100, to 1000, to a gigabit, or I guess a gigabit to 10 gigs, and so forth - every time you get 10x o’clock, you should probably give about a third of that to packet size, so that the number of packets in a given unit time doesn’t also go 10x. Your packet count and your packet size each go about a square root of 10 [unintelligible 00:44:39.00] And had we been doing that all this time, a lot of things would be simple, that are currently very hard. We certainly wouldn’t hear that fragmentation didn’t work if the packets we could send had gotten larger over time. But they didn’t. And the reason they didn’t is that the Ethernet market relies on backward compatibility. When somebody adds 10 gigabit networking to their office network, they don’t make everybody switch at once. They just say “New ports will be 10 gigs, but the old ones will still be one gig, and we’re going to run a network bridge, a layer two bridge to connect the old one to the new one.” And that won’t work if the packet sizes on the new ones are so big they can’t be bridged backward to the one that made the market exist in the first place.

So Ethernet is effectively trapped at 1500 octets for all time to come. Yes, a lot of us have turned on what we call jumbograms, so 9100 bytes, which turns out to be a very convenient size [unintelligible 00:45:44.00] So if you’re running jumbograms, your NFS is going to be faster, your [unintelligible 00:45:53.15] is going to be faster… Everything you do is gonna be faster. You just can’t use that when talking to people outside your own house, or your own campus. Because there’s no way to discover whether your ISP can carry packets that big, or will at the far end.

[00:46:11.03] So because we don’t have that, because we didn’t do what Boggs inevitably thought was the intelligent obvious thing that everybody should do, give one third of your [unintelligible 00:46:20.29] to packet size, anything we do with DNS in the future is going to have to take that into account. And that in turn means we’ll be making the assumption “Well, I guess we could probably send about 1,400 bytes plus room for all the headers and stuff that was added”, and now we’ve gotta find a way to connect several adjacent packets together, so that we can do essentially application-level fragmentation. Or else we’ve got to deal with the handshake overhead. So I predict, knowing the IETF culture as I do, if we start now, then within no more than four years we will come to an agreement on that sinful issue.

Evan Weaver: And we were kind of surprised… There were a few other companies that set out to solve this same problem around the same time, in particular Cockroach and Yugabyte. But they also found – a different technology than us, but they also found the market wasn’t really ready for this kind of interface model. But their solution was to fall back and build another SQL database. And that’s fine, I guess; there’s 30 cloud PostgreSQL things you can use, and it’s hard to differentiate among them… But if you can carve out a niche, you can make a business there.

We didn’t wanna replicate those old interfaces; we really wanted to build an interface which was for where the world was headed, where the new stack was being built; for people who were building this dynamic edge and mobile and SPA browser applications…

[48:11] In particular, we fit well with blockchain and crypto stuff. There’s no commitment to SQL in that world. People are looking for the newer, better language, the newer interaction model, and these kinds of things. It’s easy to adopt Fauna if you already use GraphQL in your organization in particular, because we offer a native GraphQL interface.

I think one mistake people sometimes make is they’re like, “Well, how do I migrate my existing Postgres cluster to Fauna?” You can if you really want, but it’s not what we’re really designed to be. We’re designed for new workloads and for augmenting the existing systems. So if you have a big Postgres cluster, whether it’s in the cloud or whatever, leave it alone. That’s okay. If it works, great. But maybe you don’t wanna continue investing in it, you don’t wanna run the risk of altering tables, you don’t wanna deal with provisioning more hardware for new use cases and that kind of thing. Use GraphQL in a federated system, like Apollo, to augment that with Fauna, and put your new data, your new applications, your new product features in the new stack, and let the old stuff alone. That’s more the Fauna model.

Aaron Hnatiw: I think the point that you bring up about having the basics covered is really important. It is intimidating to think about all of the things you need to think about to be an expert in security, to know what different kind of vulnerabilities there are out there… And I’m gonna let you know a little secret - no matter how much you catch, there’s always gonna be something there. I don’t think it’s possible to have code that is completely secure, unless it’s maybe one line, developed in isolation, and has no callouts and no inputs. It’s almost impossible. I don’t think you can necessarily lose sleep over that. You have to look at it from the basics perspective, and then from there, I think what a good security person will do is understand – because the way to make something the most secure is to make it not functional at all. You have to have a tradeoff.

A good security person will look at something and understand the business risks and understand really what it’s trying to do, and find the best way to put the most security measures in place. But some of the basics – I mean, I can go over some of the basics you should probably know, as a reference, to cover the basics.

I think three main things stand out that you can easily do and keep in mind when you’re developing something, and then you’re covering a substantial amount of vulnerabilities. The first one is the patch. Just make sure your libraries are up to date as much as possible, especially if there’s a security vulnerability. So just be aware that the latest one is probably going to be the best for that. It’s not always possible to keep things up to date, I understand that. In an environment where you have so many dependencies, sometimes upgrading is not an option. From there, there’s other mitigations you can put in place - put something in front of it, or try to learn what the exact vulnerabilities are and find out the ways of mitigating it. So patching and keeping things up to date is probably one of the most important things and one of the easiest wins you can have. That applies to operating systems as well, and applications and libraries.

The second thing - and probably the most important thing - is input validation. Understanding that the input that you’re getting from a user is what you think it is, and checking that on the server, rather than on the client. Because client-side control can be bypassed very easily.

[28:01] In a browser, if you use a proxy like Burp or ZAP you can intercept the request, change the data after it’s left the browser, and then send it on and it completely by-passes any kind of client-side input validation. So checking that the information that you’re getting is what you think it is, and not some malformed input like a super long string to get a buffer overflow, or negative numbers when you don’t expect negative numbers, or special characters that you don’t expect… The best way to do that is to use a whitelist over a blacklist.

The difference between a whitelist and a blacklist means – a whitelist is looking for a set number of things that are allowed, whereas a blacklist is looking for a set number of things that are not allowed. The number of inputs that can be allowed through is significantly less when you use a whitelist, so you are much more aware and you’re much more in control about the data that comes through, as opposed to with the blacklist… Where there may be a ton of things that you haven’t even thought of that could come through and could get around your validation.

So input validation is probably the most important, because really any vulnerability, any exploit comes from user input. If you can control that, if you can find some way of mitigating that, you could probably reduce at least 50% of the vulnerabilities that you may have in place, which is huge.

The third one is output encoding, and this is more of a preventative measure. When you’re outputting data onto a web application, for example, if you encode it with HTML encoding, for example, by doing that you will then reduce the likelihood of something like cross-site scripting, which is essentially when an attacker can get JavaScript to run in a client browser through input they provided. So if you can encode those characters, then it won’t look like JavaScript and it won’t execute like JavaScript, it will just look like a string of gibberish, almost.

So doing patching, input validation and output encoding are three of the biggest things. There’s a few other things that I can just briefly mention, in case you’re taking notes and you want some more depth… One is hardcoded credentials in API keys. I think probably like four months ago it showed up on Hacker News that you could just do a search for password in Git pushes - you search that and you find thousands and thousands of Git commits that had a password in it, or had someone mentioning a password, or taking a password out of their application. So having those hardcoded credentials, especially in open source software, is trivial to find, and then someone could take that easily; they don’t have to have any skills, and they could just access your machine. So making sure you have a safe way of passing up strings and secure information and secrets to an application is important.

Then two other key things is authentication and authorization, so making sure that people are doing the things that they should be doing, and they’re allowed to do that and they’re gated from doing things they shouldn’t be doing. Then the last key point is encrypting data arrested in transit; making sure that you’re using a TLS over the web, you’re implementing a security certificate. Letsencrypt makes it super easy to do that now. And then encrypting it at REST, so using proven crypto, like AES or Bcrypt for hashing - just some examples.

I know that was a lot to take in, and I’m sure that a lot of people are probably gonna go back and review over that… But if you can just cover those things, just keep those in mind as you’re developing, I guarantee you’ll reduce at least 90% of the vulnerabilities that can be introduced in your application just from those things alone.

Filippo Valsorda: Fair enough. Fair enough. But yeah. Huh. I placed it completely on the wrong side of the world. Oops… Cool. I’m trying to think what other Go 1.23 things are there… And I think there’s the crypto rand changes, and I can’t think of anything else. And crypto rand is the package that you use to get random bytes out of. And it’s mostly fine… Like, it’s not really one of the things that makes me think “Oh no, I wish I could change things in the standard library” so much. But it also has a few things we can make a little better, and I’m thinking of doing just a single pass in this release, and just clean up all the things.

The first one is that – it was actually an idea that came from Russ Cox, which was very entertaining, because I was like “Hey, Russ, can I make a new API in crypto rand that doesn’t return an error and just panics if it gets an error? Because it will be much more usable, and –” I’ll tell you what API was. But usually, he’s the one that argues for being more conservative, right? And so I expected him to be like “Well, I don’t know… How often is that going to panic?” And instead, he was like “Hm. Really? Why does crypt rand ever return an error?” And I’m like “Well, yeah, that’s a good point… We’re using the system calls on modern Linux, and… Well, I guess [unintelligible 00:45:52.14] could be a problem.” And he basically told me “Look, if you can fix it so that it almost never fails, we could just make it throw.” And if you’ve never heard about what a throw is in Go, and you’re about to say “Wait, Go doesn’t have exceptions. There’s no throwing in Go”, no, no, there is a function called throw. It’s an internal runtime function that just crashes your program, and there’s nothing you can do about it. [laughter] Because that’s to make sure that nobody tries to set up a recover around it. Because there’s nothing to recover.

And I went in there, and they changed how we’d get random bytes on macOS, because I texted friends at food companies that shall go unnamed, and I asked “Okay, so what should we use on your platform?” and they’re like “Yeah, that one is much faster, and never returns an error.” Would you look at that. And so now we have random bytes that basically never return an error. If you’re curious, there’s this long proposal that spells it all out. And so we’re just going to make a throw and crash the program if you’re in a very weird corner case in which you misconfigured your system so much that even with all of that, we can’t get random bytes. And that makes me deeply happy. [laughter]

So that’s gonna be fun. Then I want to make it a little more efficient by not causing it to escape the byte slice you pass it into the heap, so that you can avoid allocations… I want to make it not import math big, or at least I want to make an internal version of crypto rand that doesn’t import math bag, so that I can stop some packages in the standard library that need random bytes, and don’t need math big, which - we talked about it in the other episode. I can make sure that they don’t import it at all.

And yeah, a few little changes like that… I feel like there was a listener one… Oh, yes, and there’s the new API, which is going to be something you give it char set, and can give you a random string made of those characters. So you can generate things like passwords, and tokens, and stuff like that. And I want to make it an API that you don’t configure at all; you just give it the char set and it gives you back a string using those characters that’s long enough to be secure. And then if you know what you’re doing, you can slice it down and make it shorter, because it doesn’t return an error. So you can just put the slice [unintelligible 00:48:21.16] right after, and… Yeah, fun stuff with crypto randomness.

Johnny Boursiquot: I’m interested in understanding how we surface the right cut of information to the right people. We’ve been talking about – if we’re able to solve the first problem, which is collect all the things, and allow the customer to come get what they need, when they need it… If we use that as the baseline; if we now turn inward and say “Well, I have some internal customers. I’m an operator of some container orchestration tooling”, whether it’s hosted by a cloud provider, or whether you wanna roll on your own data center, whatever you wanna do.

Now, you know you’re collecting a lot of information, and you make it available in some sort of a data lake, or pool of data, whatever it is; come get it, for internal folks. Is it incumbent upon you or your team or your department to then teach people how to mine insight out of this pool of data? Is it the job of the people collecting it to tell you how to do that, or do you hand it over and say “You do what you will?”

Matthew Sanabria: I have to think of how to how to word mine.

Johnny Boursiquot: Unpopular opinion… How about we circle back to me? Because I’m trying to think of how to articulate mine in a non-IKEA-offending way, like you’ve done… [laughs]

Johann Gyger: [11:45] So the short answer is I was somewhat lucky to become an ambassador… But I can start with the longer version, which goes back to 2015, and it started actually with containers. Let me tell you that story, when a colleague of mine demonstrated to me how Docker works. He just ran a container, some command, and it was so fast I was blown away. After that, I talked – at the time I was working for an insurance company, and there were some ops people, or infra people, and they immediately jumped on this train, like “Yeah, let’s do that. Let’s go with containers as the universal format for production as well.” After that, the discussion came up, “If you have containers, how do you orchestrate those?”

So there was a discussion whether it should be Docker Swarm, Kubernetes, or OpenShift. Finally, we decided to go with vanilla Kubernetes, which was pretty young… It was already there, but it was still a pretty young project… And to decide to go with Kubernetes at this time for an insurance company was a bold move. So that was between 2015 and 2018, and then in 2018 we thought “Well, there’s not only Kubernetes. There’s a whole ecosystem that’s going on.”

With two colleagues we went to New York, and we met some CNCF folks, like Cheryl Hung, and late Dan Kohn… And we thought “Well, we have to do something about that as well”, so we started our own meetup in Bern. It’s the Cloud-Native Bern Meetup. It’s a community now more than a thousand people. And if you know Switzerland, it’s not that large. It’s a small place. And that’s a huge community for such a small city like Bern. And we went on, and made like 13 meetups in between, and somehow I was lucky to become an ambassador as well. The selection process is not transparent, and I think it’s just because of the community work we were doing in Switzerland. I think that was the main reason for becoming a CNCF ambassador.

Jeff Adams: Sure. I’m an alumnus of Brigham Young University, and participating member in the Jesus of Jesus Chris of Latter-day Saints, and in our local congregation here in Massachusetts where I live, we have a large number of Cambodian speakers. Over the years I’ve had a lot of opportunity to work with them, get to know them, I’ve learned a few words and phrases to be able to greet them and ask them how they’re doing, and so forth… I’ve always had this affinity for Cambodian, or Khmer - that’s how you say Cambodian in Cambodian. So when I saw a story in an alumni newsletter that I got from BYU, that there was a group at BYU that was actively collecting recordings of personal histories of Cambodians, especially those who had been affected by the atrocities of the Khmer Rouge, that they were going out and they had collected 4,000 hours of stories, and that people had then manually transcribed like 1,000 of them - it struck a chord with me. I thought “These are people I care about, these are people that I’ve worked with.”

And my first thought was “Oh, let me make sure that my friends here in Massachusetts (who are refugees) might have their own stories to tell. Let me make sure that they can contribute their stories to the collection.” And then I thought “Well, wait a minute, we have here a situation where lesser-resourced languages, where we have now thousands of hours of high-quality recordings with good transcriptions for a big chunk of them”, I thought “Those are the ingredients you need to develop a speech recognizer.” And Cambodian doesn’t have a good quality speech recognition system, so I thought we should do that.

[32:00] So I’ve asked some folks on my team here at Cobalt as a sort of a pro bono project to see what we could do on this. So we reached out to the group at BYU, and they said “Oh, that’s a great idea”, because they could then use the speech recognition to help transcribe the rest of their audio going forward.” So BYU put up 4-5 students to do a lot of the labor of organizing the data, and making sure that things are recorded properly and transcribed properly and ready to be processed in training the models… And we at Cobalt have had people contributing in sort of an advisory role, and mentoring… So it’s good all around. Students are getting an amazing opportunity of learning about how speech technology works, and we’re helping them do that…

Anyway, long story short is we are close now, maybe a couple months away, from having good-quality Cambodian speech recognition, assuming - I’m gonna knock on wood - that nothing goes wrong in the meantime… Because a lot of things have to come together just right to make it happen.

But anyway, Dan, I think that’s the story you wanted me to tell, that we at Cobalt are working in conjunction with folks at Brigham Young University to develop Cambodian speech recognition… And now it’s got me thinking “Well, are there opportunities like that for other languages?” Could we find some other partnerships where someone who wants speech recognition for Swahili, or Aymara, or some other language, that we can go and say “Here’s some audio (I don’t know where we’re gonna get it from), and here’s a partnership where we can work together, where people from the community might work with us to develop the technology.”

Anyway, it’s a passion of mine to figure out how to bring this technology to the whole world, and not just leave it in the hands where it’s gonna make us the most money.

Alex Russell: [01:01:48.18] Yeah, so I’ll try to break them apart a little bit, and say that if you’re an individual who has skills in JavaScript, and you’re looking for how to improve, again, the most important thing you can do is learn to be putting the user that you’re trying to serve at the center of it, and then start to think in terms of hitting frames, every 16 milliseconds; hitting that interaction target that you would like to have hit on your extremely expensive device, on that cheaper device, and for that marginal user. I promise you can do it. Computers are fast. And you’ll find out where your framework might be a stumbling block, or where React will generate way too much GC in a critical section or something, and it’ll hurt you. And that will turn into a challenge and you’ll learn to solve it… Maybe inside the framework, maybe you’ll learn to go around it a little bit… But those are incredibly important learning opportunities for you to understand the trade-offs.

And then I would suggest, again, as an individual, if you’ve been living up in JS land, where you’re kind of doing everything in user land, the most important hours you can spend are not learning another framework; they are learning the system underneath you. Every hour you spend learning how the browser actually processes your content and actually thinks about it, actually thinks about the networking and actually thinks about how we take DOM and CSS and we construct them together, and we create the layout tree, and then we paint out of the layout tree, and then we rasterize that out in the GPU - that flow and how we do that every 60 hertz, every 16 milliseconds, learning the pieces of it and the sensitivities of that system to what you do in CSS, and JavaScript, and through your HTML, will make you a better developer in every framework.

So it’s extremely important that you learn to think about that knowledge not as something that you don’t have to think about anymore, but as something that now that you’re not a beginner anymore is extremely valuable to you… Because you’re not at the end of your career, so it’s going to be important that you be able to adapt on the basis of shared knowledge across all of it.

And then for teams and for organizations, I’d say that that hiring thing - again, that’s trope; it’s not reality. I promise you there’s a fire sale on talent right now. Go look. You could just pay for it, and you’ll pay less than you think you would. But in terms of the choices that you make in terms of stacks, I would encourage folks to be a little bit more data-oriented, right? Again, step back and look at the actual experience. How many of these tools that you think are necessary are there to solve problems that the tools that you thought were necessary created, right?

if you start with simpler output, one of the cool things about it is – okay, let’s just say you move from taking your static marketing website out of Gatsby, which is going to go with some huge GraphQL query to return a bunch of React components, and render that on the server side, and do all this stuff. And then you have to think about how you’re gonna smush all these things out, and like do the CSS minification, and all that stuff… Whereas if you just output the same basic HTML, with the SQL query, and some PHP, or some Python, or some Ruby, how much more headroom would you have in the output in terms of the cost of the output, before you ever had to think about applying a tool to start tamping down on those costs, right? Because once you start thinking in terms of budgeting, latency budgeting, complexity budgeting, you can start to think about the trade-offs as being not about like for like, but it’s about being different in ways that structurally set you up for success if your system is simpler.

Now, once you get to a very complicated product, when you know you have a really complicated product, then you will need sophisticated tools, and they will need a lot of management. But going through that process of evaluating those things, and doing bake-offs, and learning the properties that emerge out of those simpler versus more complex systems will teach you something, it’ll give you a finger feel for whether or not one style of construction is going to be more appropriate here or there, and maybe help you make more informed trade-offs in the future about the style to build in for a particular kind of use case.

[01:05:43.08] And then I’d say at a community level, we need to step back, and I think we need to think about our relationship to society. And I don’t think that that’s going to be a thing that is traumatic, or even hard, right? We’re all here to build great experiences. I believe that. I truly believe that. I think Laurie’s piece in response to mine said something about how I think that developers are stupid, which is hilarious, on a couple of levels. But I just don’t. I work with extremely talented people all day long, and I’m grateful to get to work with them. And if we take seriously that it is evidence and science that are going to get us to a place where when we apply our skills, they turn into good things for the people that we wanted them to, that’s a fundamentally optimistic way of thinking about this technology.

We don’t have to accept what has happened, or accept that it happened for good reasons, or even accept that the people who are so attached to it are correct, or worth having in the conversation, in fact. But once we put the evidence at the center of what’s working and what’s not, I think we can start to reevaluate our relationship to the technologies, and acknowledge on the one hand we’re more than our choice in Java framework… [laughs] I promise you, there’s life after the framework that you love right now. And also, that as a discipline, the more we are reliably delivering good things for users and for businesses, the more likely it is that there’ll be more work for us, more good things that we’re going to be able to create in the future, and more users that we can help. And for me, that’s the goal, is to be able to help users. So when we put that at the center, and we put the marginal user at the center of that question, then I think we get to a good place. So I’m optimistic.

Jerod Santo: Let’s do resolutions, as we are getting near the end. Does anybody – we have a few predictions; we’ll see what happens next year, of course. All in good fun. Resolutions, how about? I tend to just keep mine private, so I can fail to myself… But some of y’all like to put them out there to the public, so that we can help you stay accountable… Does anybody want to be resolved in this new year to maybe change something about their lives, or have a goal for your career, or for your funemployment? …anything, before we get into the big announcement.

Stephanie Morillo: [04:01] Yeah, it’s really interesting, because I have a Strategic Communications background, so for the first four years of my career before I moved into tech I was working in public relations for non-profits, and that’s a very writing-intensive job. And then about four years in I was involved in a project where one of our clients was migrating their entire website onto WordPress, and they had me do some really simple cleaning up of HTML tags on the frontend. I wasn’t doing any of the magical stuff, but I was really interested in that and I wanted to learn more, so a good friend of mine mentored me and taught me how to program. Once a week we’d meet for about 2-3 hours in a Starbucks, or at his place, and we’d just mess around with Ruby for a bit. Initially, I thought “Okay, I’ll become a developer. I really wanna become a developer. This is exactly where I wanna go.”

But over the course of time, I realized that it wasn’t the coding so much that I liked, it was actually messing things up and then writing about what it was that I did… And I wasn’t really sure how to marry the two. I knew, or I at least expected that writing and content was something that people liked, but it wasn’t something that was valued that much, and I knew that just because – you look at the salaries for a software developer, and maybe a technical writer or someone in a content position, and there’s a disparity. Now, they’re not the exact same types of roles; there are a lot of reasons and vectors that go into that… But people thought I was a little strange for wanting to stay on the content track, even though I wanted to continue in tech… But I knew that there was a need to teach people, and the best way to teach people is you have to know how to communicate, or at least you have to make the attempt to communicate your thoughts and your ideas with people. So I stayed the course…

You see, it was easy actually for me, even as a content strategist, to assume that “Oh yeah, a lot of people write, everybody blogs, everybody has a live stream, or something… This is stuff people know”, and I thought that it was self-evident, frankly, so I wasn’t even sure if I should write the book, because I was like “This is too basic. This is too foundational. People are gonna laugh at me. They’re not gonna want this.” And it wasn’t until I told people that I was doing it that people were like “Actually, no. This is exactly what we want. This is exactly what we need”, and I’m glad to have been in the position to have provided that.

Suz Hinton: It depends on “Are you currently using TypeScript right now?” - that’s usually a complicated question for me, because I don’t work on a product team anymore. I’m actually in the process though of rewriting an old JavaScript hardware library; it was actually one of the first libraries I ever wrote, so it was not fantastic code… So it’s been a dream of mine to rewrite it, just to see how much I’ve learned since then. But I thought that it would be perfect for me to also rewrite it in TypeScript, because in hardware things have to be very exactly correct at all times - it is very unforgiving - and I feel that TypeScript would give my code and my functionality in the architecture the structure that it really needs, so that I can maintain this library a lot easier going forward.

So I’m very excited to do that… I’m sort of breaking things out into modules, I’m creating a billion interfaces because of the different signatures and shapes of different payloads that you’re sending over to the device, and things like that. So that’s been really exciting, and probably not the first use of TypeScript that comes to mind for a lot of JavaScript developers, but again - I’m usually the edge case here, so that’s how I’ve been using it recently.

Rico Sta. Cruz: Yeah. One of the things that I also like doing in the same idea is whenever I make a project and make a readme for it, I make sure to link other projects that have a similar goal to what I’m doing, and mention how mine is different. Like you said, there are a couple of people who would be writing things in a way like “Here’s our solution, here’s why it’s so great”, but they kind of fail to go to the side of “What about other projects? What do they do that you’re not doing? What are other solutions that are out there that are different from what you’re doing?”