Paul Smith: It was acquired by MSNBC in, I think, 2011. It was actually part of NBC News, because we had that news angle. Nowadays people take, I think, for granted things like NextDoor, and Facebook Local News about their neighborhood. So we were kind of one of the early pioneers of that. We sold the company, and kept working on it for a little bit… But we did some interesting things on Everyblock. In fact, one of the things I’m most proud about is we built our own map stack.

So at the time, if you remember back in 2006-2007, JavaScript engines and browsers were starting to get faster, Google Maps popped on the scene, and it was suddenly “Oh, you can do these native desktop app things in your browser’ for the first time. And in fact, Everyblock kind of came out of this idea, of sort of like a Google Maps mashup of taking Google Maps and then using its API and slapping data points on it. And when we started the company, we thought it would be great, since that’s going to be such a central part of this - you want to be able to look at a map of your neighborhood, drill into your block, see where news is happening… And when I say news, I’m talking about maybe your block is mentioned in the news, or maybe a building license has been issued, or a restaurant inspection, or things like that. Public records, crime information… And we would aggregate all that and put that onto a map.

So Google Maps was great, but we wanted to have control over the look and feel, and user experience, so we built a map stack from the ground up, using sort of open geospatial tools at the time, OpenLayers, Mapnik, some other tools like that, and then kind of combined that with a Django app server we were using to pull the data out of the database, and then present that on the browser. We worked with a great designer, Wilson Miner, to kind of come up with their own palette and design for the maps themselves, which - I thought they looked really beautiful. So it was a way of visualizing the data, and I think it was a pretty interesting accomplishment. And now, you’ve got things like Mapbox, and there’s just a lot more flexibility when it comes to sort of the in-browser custom map and geospatial experience.

Paul Smith: It was acquired by MSNBC in 2011. It was actually part of NBC News… Because we had that news angle. Nowadays people take for granted things like Nextdoor, and Facebook, local news about their neighborhood. So we were kind of one of the early pioneers of that… And we sold the company, and kept working on it for a little bit, but… We did some interesting things on EveryBlock. In fact, one of the things I’m most proud about is we built our own map stack.

At the time, if you remember back in 2006-2007, JavaScript engines and browsers were starting to get faster, Google Maps popped on the scene, and it was suddenly like “Oh, you can do these native desktop app-like things in your browser for the first time.” And in fact, EveryBlock came out of this idea of sort of like a Google Maps mash-up of taking Google Maps and then using its API and slapping data points on it.

And when we started the company, we thought it would be great, since that’s gonna be such a central part of this – you wanna be able to look at a map of your neighborhood, drill in to your block, see where news is happening… And when I say news, I’m talking about maybe your block is mentioned in the news, or maybe a building license has been issued, or a restaurant inspection, or things like that. Public records. Crime information. We would aggregate all that and put that onto a map.

So Google Maps was great, but we wanted to have control over the look and feel, and users experience… So we built a map stack from the ground-up using open geospatial tools at the time. OpenLayers, Mapnik, some other tools like that, and then kind of combine that with a Django app server we were using to pull the data out of the database, and then present that on the browser.

We worked with a great designer, Wilson Miner, to kind of come up with our own palette and design for the maps themselves, which - I thought they looked really beautiful. So it was a way of visualizing the data. It was a pretty interesting accomplishment.

Now you’ve got things like Mapbox, and there’s just a lot more flexibility when it comes to the sort of in-browser custom map and geospatial experience.

Carl George: Nobody’s told me I can’t say that. But that kind of drove it. Basically, Red Hat was like “We want this project to keep existing”, and so they made job offers to all of the developers. Most of them took it. A few of them turned it down. And then they basically came into Red Hat, partially. They were still kind of kept off to the side. They’re like “Well, you’re still kind of duplicating this product, but we want you to keep going and exist.” And so they kind of sat in that limbo for a while, where they weren’t growing, they weren’t getting people resources, but they had the resources they need to focus their full-time on it, get a paycheck, and keep the project going.

[00:20:01.26] That was a little bit of an infusion, but we still had this problem around this whole bug-for-bug thing, and also being a duplicate of the product. There would never be a business incentive to put the same engineering resources into your products and this project that is trying to match it as close as possible. That would never make sense. No business person would agree to that. But because of all the nuance around how it was being used as a development platform… But we also saw the pain points of it being a development platform that lagged behind the thing it was trying to match, right? CentOS would typically lag about a month behind on the minor versions. Like, RHEL 7.6 would come out, and then CentOS 7.6 would be – it’d be 7.5 for a while, they’d finish the rebuild and publish it, and about a month later you’d get it. So those rebuild gaps were real painful for the developers trying to use it as a platform to build on.

Rod Vagg: I don’t make as much differentiation between casual contributors and whatever the opposite of that is, because I don’t think you can actually know when somebody is a casual contributor and when they are not. Something might be small to start with, but that might be somebody testing the waters. So they might be trying to just change a tiny thing that’s annoying for them, but if you give them the right signals that “Everything is okay, we’re not going to jump down your throat every time you try and make a tiny change, we are actually going to embrace you”, you can actually turn those people very readily into more than casual contributors.

So I don’t think that distinction is actually that helpful, unless you are able to measure it over time. So if you look back in the history of a project, look over the past six months, then you might actually be able to actually see who were the casual contributors. But at the time that it’s happening, I’m not sure you can say who the people are that you could label casual, because very often, those people, when given the right signals, can turn into something a lot more.

I’ve seen people come and show up with documentation fixes, even minor ones, and you give them the right signals that, “We’re open for business, we’re open for contributions” and they level up really quickly. In fact, one of the most interesting things about this kind of contribution policy is that you can actually take people with a low level of technical skill and you can level up their skill through the process.

So I’ve seen people come in, saying things like “I don’t really understand C++ or this fancy stuff you are doing in here, so just contributing to the documentation is good enough for me.” But they may contribute enough documentation to get over that initial hurdle of non-trivial changes, and they may become of the core contributors of it, even though they are asserting they don’t have the technical skill, but the fact that they recognize that is important, and when you put them at the same level as everyone else, it actually encourages them to level up.

I’ve seen people go from saying that they don’t have the technical skill to be a primary maintainer of a project that is very technical, just by giving them the permission and that blessing to say, “You’re one of the team, you’re equals with us and you have the responsibility to help foster this community and this project” and just simply giving that responsibility means people level up.

[\00:35:44.27] It’s really surprising how that works in practice when you give them the responsibility. It’s the responsibility that does it; when you give people responsibility over a project, then it creates in them something that makes them step up to the plate, and they are not just somebody from the outside tinkering and throwing little changes at you; they are actually somebody that takes it on board personally and that does really surprising things.

Tracy Lee: Yeah… So I’ll talk to you guys about Contributor Days. You can go ContributorDays.com and see what we’ve been doing there. We’re just about to release the videos for RxJS Contributor Days. This came about because I wanted to help framework authors get together with customers (big customers of theirs) and also core contributors, stick them all in a room and try to help them be more productive.

With RxJS Contributor Days, we are doing that once a year. It was really amazing what came out of it… You have people from the Angular Core Team coming, you have people from the Nuclide project at Facebook, who use RxJS a lot; you had people from the Slack team come, who uses it a lot, you had Matt Podwysocki, who was the original creator of RxJS, André Staltz from – well, André does so many things, but CycleJS I think is what he’s mostly known for, or Evan You from Vue.js… So sticking them all in a room and then just trying to figure out, “Alright, RxJS is Angular’s only external dependency, so what are they concerned about? What is the core team not addressing?

We had a few beginners or teachers in the room, and it was like “Okay, how can the core team actually make RxJS more attainable to learn, more approachable to learn?” Same thing with Angular Contributors Days; we’re gonna have Misko Hevery, Igor Minar, Jules Kremer and Stephen Fluin leading the charge on Angular Contributor Days, and we’re inviting a bunch of large companies that are using Angular and people who are heavy contributors.

We’re not only trying to address the fact that these people don’t get to sit with the core contributors very often, but allowing the cross-pollination of ideas to actually happen. I think a lot of times in open source one person’s hacking on one thing, another person’s hacking on another thing… If you just get everybody together, you realize that maybe you’re trying to solve the same problem, so you can distribute resources a little bit better, and then hopefully set the pace for the year of “Okay, now I have these relationships, now I can do awesome things with those new relationships.”

Glauber Costa: Oh, happy to do that. And the story - in a sense, it’s still the same story, so I think you hit the nail on the head there. It’s just really how it started. I don’t feel this is a completely different thing, what we’re doing now. The story there is that we were using SQLite in our company. Pekka is my co-founder. We founded a company… If you happen to stumble upon it, I’ll just mention it here by a name; it’s called ChiselStrike. And it was in the API space, we were doing a bunch of things, and we were using SQLite very heavily in that company. And the reasons we were using SQLite were reasons I think that a lot of your audience will understand and relate to. Look, you would just do npm install, the name of the package, and then everything’s there. Your database is there, everything is there, everything works, you don’t have to install anything… So SQLite for us always had this appeal of a database that is just always working, and then you can build stuff around it, and it’s just always there. So there was always something that was very appealing to us. However, we also stumbled upon this thing a little bit about SQLite not being open source. So let me address that.

SQLite is a public domain software, which is technically a difference that doesn’t matter. I think only a lawyer would be concerned with what is the difference between public domain and open source. For all intents and purposes, it’s the same difference between like the BSD license and the MIT license. It’s minor. Nobody cares. I don’t care, and I think nobody else does either. But SQLite is, according to their own website, by the way - this is something that they put in big, bold letters in their website - open source. Again, I’m not claiming that they’re wrong to say that. It’s all good. Open source, but not open contribution.

So when people think of an open source project, there are two things that come to your mind. The first one is the code is open. And SQLite is that. But you also think about – instinctively, “Well, the code is going to be on GitHub somehow, and then if I find an issue, I can go contribute to this project… And I have some source of ownership to the code. It’s not just that I can see the code, I can also modify the code.” So that’s usually classically one of the tenets of open source. Now, SQLite is not like that. The maintainers of SQLite, the creators of SQLite do not take contributions. This is not something that we’re saying, this is something that they’re saying.

[07:57] And yes, I mean, I think it happened in the past that people managed to contribute to SQLite in very special occasions, but it’s not a project that is designed to take contributions from other people. So that is how SQLite is.

We started running into a couple of, hurdles with SQLite. Part of that is that we wanted to deploy our SQLite databases to the edge. We wanted to deploy it in a way that was replicated, we wanted to do things like read replicas, and we wanted to put it to run on call for workers, and all things like that. And we considered this a good thing for our project. And technically, we knew we could do it. And for context here, Pekka and I, we’ve been working together.

Pekka is my co-founder. We’ve been working together for 15 years now. We met each other when we were both working on the Linux Kernel. Pekka was one of the maintainers. Each one of us worked in Linux for almost 10 years, the first five without knowing each other; I think we knew off each other, but we never met. And in the last five years, very closely to each other, where we developed a good friendship. After that, we actually joined a startup that also became a database company. We spent eight years doing that. So we felt technically like SQLite, all things considered, is pretty simple, given the scope of things that we used to do. So technically, we can make the changes that we believe we can. And so let’s just come up with our own version.

And we saw at the time a lot of people dancing around the subject. That’s what we saw. So there were some projects like LightFS that were super-interesting projects. Dqlite… We saw lots of people trying to get SQLite to work as a distributed system. The problem that we saw with all of those projects is that they were all, again, dancing around the subject that SQLite cannot be changed, so we’re going to create layers and layers and layers around SQLite to help with that. And for us, it was – technically, this is actually a pretty simple problem. You just have to change SQLite here, here, and here, and you’re good to go.

And I think the last piece of the puzzle, or maybe the two last pieces of the puzzle - number one is that we were very concerned with who’s going to want to run our SQLite plus patches, because SQLite is a very trusted project… So you make random modifications to SQLite, and everybody starts distrusting what you’re doing. So we knew that - look, the solution to this problem is that we should just create an open contribution version of SQLite, because now it’s not like SQLite plus Fink, it’s its own project, and then people come to trust this one project. And with that, the diagnosis that we had at the time was like “Hey, SQLite is great code-wise. The thing that is holding it back in our reading is exactly the fact that it cannot take advantage of the dynamism, of the creativity of a truly open source community. So let’s create that.”

And the last piece of the puzzle is that at the time, we were actually discussing “How do we do this?” And one of the options was maybe we should just rewrite SQLite. And we had a lot of experience in our previous company, Scylla… Scylla was a re-implementation of Apache Cassandra and C++, fully compatible. And maybe we over-indexed in the fears that we had going that route. We said, “Hey, the problem with Scylla is that it took a long time for us to put in the market. SQLite is much smaller, so maybe we can do it.” But maybe because we had just come from an experience where we wrote a database, we figured that forking was a better strategy.

So at the time, our decision, after much deliberation was “We’re not going to rewrite SQLite. We are going to fork it instead.” The main advantage is that you have something tomorrow, because you start from a codebase that is already working. And then we’re going to start making changes to that.

[11:59] So that was LibSQL. LibSQL was essentially at the time – and the story of Limbo, if I had to tell the sanitized version in which the fork never existed, it would be the same up until this point, because all of those things, all of that, it’s still valid. It’s just that at the time, a year and a half ago, we decided that forking was the best alternative. And now - and I’m happy to go into the why, but now we decided to try “Okay, what if we had done that, one of those options that we considered? What if we just rewrote it?” And then when we put it out there, the results were – I’ve never seen anything like that before. So that’s the story.

Yasir Ekinci: Yeah. My dad actually was a miner when he was younger. I don’t mean like “and then he grew up”, but he was actually in the mines, down in the pits… And he used to – yeah, he’d come home and he looked like he was wearing eyeliner, or guy liner, because he’d have like soot in his eyes, and stuff. Obviously, it’s not showered out properly, but… Yeah, he sort of looked like a seventies rock kind of glam star. It was great.

Gina Häußge: So it should be obvious that it’s pretty much impossible to test every possible printer, firmware, plugin, operation system, starting state of software situation. So what I do before I actually roll out the full release is there goes a long, long phase of release candidates. And OctoPrint has a release branch system built in, so if you feel fine with testing stuff that is not necessarily fully stable yet, then you can just switch over to another release branch, and then you will get release candidates whenever I push those out. And they actually get the same procedure that I do for every single release, and I will go quickly over it later as well.

But the idea behind that is that if I have something like 1,000-2,000 people out there testing a release candidate and putting it through several years of print duration over the course of the release candidate phase, and then I can be pretty sure that a lot of these combinations that I would never be able to test have been tested. And yeah, it usually takes something like three to four release candidates until no more bugs come in… And at that point, then I declare this stable.

And of course, after I’ve pushed out a stable release – so the current stable version is 1.10, but we are now already at one point 1.10.1. So there are bug fix releases that I also push out. Those do not go through a full release candidate phase, again, but they only get bug fixes and maybe small minor improvements of existing functionality. They do not get new features, they do not get big changes… They obviously also get security fixes, stuff like that, but I try to really limit what goes in there… And if it feels too risky, then it goes into the next stable release that will actually get the full release candidate phase again.

And what I do for every single release is – so OctoPrint can basically run anywhere where you can run Python. But most people run it on a Raspberry Pi, so that is also what I concentrate on for testing. And there is this dedicated image that someone else is maintaining, Guy Sheffer, for OctoPrint, which is called OctoPi. And a lot of people confuse the image with the software and the software with the image, which also causes a lot of complications in support… But anyhow, so OctoPi is the most common environment that OctoPrint will be installed on out there.

So what I have here is I built myself a little test rig that has three Raspberry Pi threes, which is the current basic option that I suggest… So get the three, because that basically is the best thing that you can get, the lowest supported version. And if you want something with more power, then of course you can get something else, but the three is like the base version that I look at.

[01:34:05.07] So I have three Raspberry Pi threes there, and all of these have a little card adapter in there that can be switched through USB, either to act as a mass storage device through a host, on the one end, or as an SD card on the other hand. So that is slotted into the SD card slot of each of the Raspberry Pi’s, and all of these then go into a USB hub to a fourth Raspberry Pi, a Raspberry Pi 4, actually, which I call the flash host. And that thing also has control over the little powered USB hub, through which I power the three Raspberry Pi’s. And now I can individually power them on and off, and I can also individually unmount and mount their SD cards, and flash them, without having to physically release the SD card, and push it into a flashing stick… That is what I did until 2020, and it was driving me nuts.

Jerod Santo: So funny story about this particular feature… This was almost entirely Cody’s idea, because I’m like a) I want social proof; I want people’s quotes on there. And then b) we do a few things differently, and they’re all minor; it’s like minor reasons. But I want to have somewhere a list of the stuff that we do, like we don’t track your clicks, blah, blah, blah. And he’s like “Well, what if you just did way too many of them, and just made some up?” And he actually stubbed it out with 36 reasons… And I took it as a personal challenge to come up with 36 reasons why you should subscribe. And in the mid ‘20s I’m like “Cody, this is too hard, dude. I cannot come up with – he’s like “I did not expect you to come up with 36 reasons. I just made up a number.” And so I finally made it to 27, and left it there. But I love that you’ve improved it, you’ve added two. I will go on record to say we are accepting pull requests on reasons to subscribe. So if you have more reasons, I would love to get that number as high as humanly possible.

Chris Krycho: And notionally, you want that to have the same basic meaning even if you’re jumping across ecosystems. Like, Bundler, and Composer, and Npm, and Cargo, and all of these can use the same basic language, and so your experience in principle - we can talk about it in practice; but in principle, it should be the same. That you can give a specifier to your package manager and say “Look, only give me minor bumps to this. And I’ll take new features if they come in, and I’ll take patch versions for sure if they come in, but don’t give me any breaking changes.” And Npm should be able to say that, and Bundler should be able to say that, and I as a user should be able to say “Okay, cool. This is gonna work the same basic way. Maybe I need to tweak the syntax a little or think about a couple different things.” But it should translate across ecosystems, more or less.

Gina Häußge: [00:15:51.11] So it should be obvious that it’s pretty much impossible to test every possible printer, firmware, plugin, operation system, starting state of software situation. So what I do before I actually roll out the full release is there goes a long, long phase of release candidates. And OctoPrint has a release branch system built in, so if you feel fine with testing stuff that is not necessarily fully stable yet, then you can just switch over to another release branch, and then you will get release candidates whenever I push those out. And they actually get the same procedure that I do for every single release, and I will go quickly over it later as well.

But the idea behind that is that if I have something like 1,000-2,000 people out there testing a release candidate and putting it through several years of print duration over the course of the release candidate phase, and then I can be pretty sure that a lot of these combinations that I would never be able to test have been tested. And yeah, it usually takes something like three to four release candidates until no more bugs come in… And at that point, then I declare this stable.

And of course, after I’ve pushed out a stable release – so the current stable version is 1.10, but we are now already at one point 1.10.1. So there are bug fix releases that I also push out. Those do not go through a full release candidate phase, again, but they only get bug fixes and maybe small minor improvements of existing functionality. They do not get new features, they do not get big changes… They obviously also get security fixes, stuff like that, but I try to really limit what goes in there… And if it feels too risky, then it goes into the next stable release that will actually get the full release candidate phase again.

And what I do for every single release is – so OctoPrint can basically run anywhere where you can run Python. But most people run it on a Raspberry Pi, so that is also what I concentrate on for testing. And there is this dedicated image that someone else is maintaining, Guy Sheffer, for OctoPrint, which is called OctoPi. And a lot of people confuse the image with the software and the software with the image, which also causes a lot of complications in support… But anyhow, so OctoPi is the most common environment that OctoPrint will be installed on out there.

So what I have here is I built myself a little test rig that has three Raspberry Pi threes, which is the current basic option that I suggest… So get the three, because that basically is the best thing that you can get, the lowest supported version. And if you want something with more power, then of course you can get something else, but the three is like the base version that I look at.

So I have three Raspberry Pi threes there, and all of these have a little card adapter in there that can be switched through USB, either to act as a mass storage device through a host, on the one end, or as an SD card on the other hand. So that is slotted into the SD card slot of each of the Raspberry Pi’s, and all of these then go into a USB hub to a fourth Raspberry Pi, a Raspberry Pi 4, actually, which I call the flash host. And that thing also has control over the little powered USB hub, through which I power the three Raspberry Pi’s. And now I can individually power them on and off, and I can also individually unmount and mount their SD cards, and flash them, without having to physically release the SD card, and push it into a flashing stick… That is what I did until 2020, and it was driving me nuts.

Chris Shepherd: [00:26:04.28] Yeah… And I think that’s another learning, isn’t it, that you kind of have to pick up as you go with Go… If you go and write JavaScript – you know, the Leftpad incident is very famous, where everyone was depending on this really small module to do something very minor… And I think that comes with people from a lot of other languages. If you do come from PHP and Java, it’s not unusual to pull in dependencies… And you get the – are they called POM files? I can’t remember what the Java things were called. I think they were called POM files. You could get huge ones, and the dependency management thing was quite well managed; there were enterprise solutions for dependency management available to you… Whereas kind of coming around to “Actually I should try and implement this myself in Go, and I should manage and own this myself…” What’s the idiom? “A little copying and pasting is better than a little dependency.” Coming around to that way of thinking definitely took me some time, and it kind of takes everyone who jumps into the Go ecosystem a tiny bit of time to get to grips with it, I think.

So especially when we were doing this, this was unusual. And also being able to type things into Google and not always finding an answer was unusual too, if you’re used to Java, where it was 20 years of experience available on the internet for you.

Daniel Vassallo: But anyway that I could, without necessarily being very picky, very choosy. I don’t care about how sustainable it

is, or how high the return on investment is; whatever I can, to at least have some baseline income. And literally, the next day, after I went through this week of anxiety, I picked up my phone, I went to my personal contacts, and I found a friend who needed some programming help, and I started doing some freelancing with this friend. But the connections wasn’t a lot; I was doing like 20 hours a month, barely enough to cover my mortgage… But it was a huge, huge improvement in my peace of mind. Now I have some income. And I knew that if I wanted to, I could maybe find another client, or maybe add some more hours… It was incredibly liberating. Now, I wasn’t relying on this project succeeding anymore; it almost became like a bonus. Now, if it succeeded, it’s great. If it doesn’t, I’m not going to be desperate, as there’s already something there.

And immediately, things started to – one thing started to come after another. I was seeing these creators, the creator economy was flourishing, self-publishing, educational products, programming books, and whatever, and some of them making decent payoffs, some of them less so… And I said, “I have some technical content in my head. I worked at AWS for almost 10 years, I know certain products inside out… What if I do a brain dump of some of the things I know, and try to sell it online?” And this became another – now I call them another small bet. I spent like a month on this project, not a long time; brain dump of everything in a Word doc, minor editing, put it out on the internet… And within a few weeks, it made low five figures, and it was, again, eye-opening to me. Like, why am I bothering with this huge, ambitious project, highly speculative, that might never – yes, the upside might be huge. If all the stars align, I might make a big exit, or have a high income from it… But there are these small, much more achievable projects that are much more attainable.

And over time, I kept experimenting with different ideas, some things worked, some things didn’t, and I started to believe that this is a sound philosophy, and then, as I kept searching it more, I realized “This is nothing new.” The operators in this world, like book publishers, venture capitalists in Silicon Valley, movie studios in Hollywood, operate on this strategy, and some of them have been doing it for centuries, like book publishers, even VCs… I mean, not the modern kind, but investors in general, since probably the Middle Ages have had some portfolio of activities; they only invest small amounts of their funds, they never go all-in on one thing… They treat ideas – I like to call it treating ideas like cattle, not like pets, so you don’t fall in love with your projects… Yes, you want them to succeed, you’ll be disappointed if they don’t… But if they don’t, it’s just business; you move your attention to something else.

[17:57] And I started to see these parallels with how these industries operate, and I think many of their principles apply even to us as individuals. Yes, we’re not investing capital, unlike a book publisher, or a venture capitalist who’s investing their capital; we’re investing our time. But it’s equally scarce and precious, probably more so… And I think we can sort of, again, reason the same way as a VC does with their funds, but in our case with how we invest our time.

So what’s our time? We only get a few hours a day, that is disposable to our projects… How should we invest it? Should we go all-in for many years on one project? Is that a wise strategy? That would be like a venture capitalist going all-in with all their funds on one idea. It’s almost never going to happen, right?

Craig Kerstiens: So it is Git, but it’s old-school Git. Go read the Postgres mailing lists. Like, here’s a patch submitted via email, and like there – nope, GitHub… I think there’s a GitHub mirror. But it’s lile, no, you’re not submitting issues on GitHub. It is old school Linux open source development.

Postgres has a few commit fests, where like “Here’s a window open for commits to come in.” There’s kind of a – it’s fascinating, because people think of the core team as what commits and controls Postgres. The core team - I believe it’s nine people; it used to be seven. That’s more like a steering committee. Like, “Alright, we’re gonna deal with the Code of Conduct complaints, and we’re gonna figure out the conference, and defend trademarks, and just make sure people abide by that.” And then there is a commit bit, and then a major contributor, minor contributor.

Major/minor is a little bit of a legacy thing. I don’t think they’re pushing that quite as much. Like, you commit a major feature like range types, or JSON - you kind of earn “Hey, you’re a major contributor”, major feature, that was in the release notes. Your commit bit, you earn over time, over the years. A few years ago, they gave them out to some people that were really up and comers. Jeff Davis, Neil Conway I think were really, really early. I think Jeff Davis may have been the youngest to get his commit bit at like 18 or 19. I don’t know if it was him or Neil Conway. But young – Jeff Davis is back, active in the community; for a little while he wasn’t. Neil Conway went on to get his PhD, and is doing other interesting database things, but not active.

So Postgres is really slow about adding new committers. It’s fascinating to watch some people that get their commit bit and get excited, and are like “I’m gonna go ship some features now, and commit stuff”, and it was not as solid, it was a little buggy, and their next year was fixing that stuff. Like, there’s kind of this unspoken role of “You did this, you maintain it, you clean it up.” So the community really watches itself, right? There’s kind of that peer pressure… Like, you’re not committing and inflicting pain on the rest of us, right?

So commit fests happen every few months. That’s an open window where patches are submitted. People sign up in the commit fest app; there’s an app. so if you want to start to contribute to Postgres, or see how it works - this is a fascinating sense of open source development - go review some patches. Pull them down, build Postgres, test it, reviewed the code quality…

I can dig up, there’s a couple presentations that were given, that we could link in show notes, to like “Here’s how to contribute to Postgres.” The best way is just reading the PGSQL Hackers mailing list. You want to become a – people ask me a lot of times “How do I become like a really senior developer? How do I get good at C? How do I become an advanced, advanced developer?” Just read the hackers mailing list. It’s fascinating how the conversation – it’s not a cordial… I mean, it’s cordial enough, but it’s not always friendly. As a beginner, you show up there and ask a question, like “Alright, where’s the patch to repro this?” and the sample test case, and all that… But it’s an amazing quality development that happens right there.

[39:50] So patches get submitted, pulled in during the commit fest… People sign up to review patches… Once enough review happens and people feel good, a committer picks it up and says “Alright, I’m gonna commit this.” Even if they didn’t author it; they’ll give it a run-through, they’ll see the reviews of it, and then they commit it.

Ganeshkumar Ashokavardhanan: Yeah, that’s a great question. And just as a background too, even when I joined AKS full-time, I did not think too much about how new versions of Kubernetes come up in AKS. I sort of thought that they just showed up. I was very naive in my thinking there. But then I learned about how oftentimes you need to make changes internally to actually support that.

[38:18] So like you mentioned, in this case I’m more of a user of what the upstream Kubernetes release teams do for versions. I guess the parts that are helpful are all the release notes, and documentation, and they provide high-level summaries of what changes are going to be available… I think that’s quite helpful, even in the process of making changes in AKS. Because especially in AKS, there are so many edge cases, and different scenarios that we need to address for customers, because there are so many different customers. So getting a good overview of the major changes in minor versions is helpful.

I think the part that I’m still thinking about and perhaps could be improved as well is in terms of making it easier to handle some of these API deprecations and flag removals, and so on. Some part of it is manual, even though there’s like various mechanisms to figure out whether an API version is being deprecated, and so on. I think some part there perhaps could be improved. And I know it’s a complex process as well, with all these fixes coming in, and some of it going through batch versions, and so on… And AKS internally also has additional backports of hotfix patch versions to previous patch versions and so on that only AKS [unintelligible 00:39:43.24] supports. So the overall process as well is like a lot of moving parts, and it’s been interesting to see how that plays overall. And for me too I think it’s a great opportunity to collaborate as well with the upstream team within both Brendan’s org, and also in the broader community, to make sure that the main features and changes that they’re doing can be fronted to users through AKS.

Gerhard Lazu: We could do. Or a Bitcoin miner. [laughter] We could. It’s there, we’re paying for it… But you know, to be honest, the reason why this is exciting is because we can spin up on-demand, and we can have maybe more than four CPUs. We only chose four CPUs because of the cost. It just hits the balance nice. But if we could use 16 CPUs on-demand, that would be a lot better. And with Fly machines we can, because they spin up so quickly… You don’t wait minutes for it to come up.

Drausin Wulsin: So your immune system is the defense industry for your body, and it is insanely good at its job, because 99.99% of bad things that happen in your body are crushed, right? The immune system dispatches them, no problem. And this is the product of hundreds of millions of years of evolution, and lots of predecessors of ours dying, in order to naturally select for this beautiful thing that is the immune system. It is actually like a defense force, where you have different specialized players that are that are good at different things

You’re not going to send a SEAL team to monitor your apartment building; you have a security guard for that. And the security guard is maybe even better than a SEAL team in some cases, for various reasons. And so the immune system has evolved to have all these really good different players that work together really well, to sort of crush viruses and bacteria when they sort of come into your body… And also to crush things that happen inside your body. So the two biggest problems that happen and that the immune system is balancing between are cancer and autoimmune issues.

So cancer is when basically there’s some mutation in your body that - it’s like a runaway train. All of a sudden, some of your own cells start replicating, and all of the brakes and emergency brakes that usually keep this from happening have broken, and there’s this sort of out of control growth that’s happening.

[20:01] Almost all cancer - again 99.99% of cancers are dispatched summarily by your immune system; your immune system sees what’s going on, it goes and kills those cells; you’ll never know. At the same time, in autoimmune issues, what’s happening here is the immune system is sort of going overboard. It thinks there’s a problem, but there’s actually not a problem, so it’s just attacking sort of like civilians, healthy cells. And this is also very problematic. So I’d say the immune system is this wonderful balance – like checks and balances. And there are lots of cell types that work and signal to each other to keep each other in control.

Okay, so what is immunotherapy? So immunotherapy is this wonderful sort of revolution, really, in oncology treatment that’s really taken place and blossomed in the last decade, where we basically really have realized that with just some very minor coaching, we can get the immune system to be way more potent at killing some types of cancer. And the coaching that I’m talking about is basically just binding one little kind of antibody to a certain type of immune cell. And when you do this, all of a sudden the immune cell sort of has like a forcefield on, and the cancer cells can’t turn it off in the way that they’ve evolved to be able to do. And so immunotherapy is really the art of coaching the immune system to be better than it already is. It’s sort of using an existing tool, just better, and giving it up the pep talk that it needs to go and fight the cancer.

Rob Barnes: So in terms of backups, we have a functionality built in there to take snapshots, for example, depending on – so the way you’ve got to think about Vault is dependent on your storage backend, you can think of Vault’s entire architecture as kind of decoupled. You have the place where your secrets are stored, so that’s where all the secrets are stored, encrypted there. And then you have Vault itself, which is the thing that you interact with and the thing that does all the encryption, decryption and secrets management for you, and it interfaces with the storage backend.

Now, typically, we used to recommend Consul as a storage backend. We now have integrated storage backend in Vault, so now it’s not as decoupled as we used to be. You never used to have kind of – you used to have Consul, obviously, as a storage backend, and you’d have to manage that additional thing. It’s another system. And if Vault is just the business value you’re trying to extract, then no one really wants to manage this additional system just to get their secrets management, right? Which is where the whole argument of integrated storage comes in.

So in terms of that, it is a clustered approach, it is highly available. If you do lose a node, there are other nodes to take its place, there are leadership elections, so on and so forth; there is replication of the secrets between the nodes as well. And then it will just come down to implementing a good, fault-tolerant design for your Vault cluster. You’d have them in different kind of fault domains, and so on and so forth.

[42:22] So that’s kind of the first part. In terms of things like upgrades and general operational overheads, I’m not gonna lie to people out there, there’s a lot of things to do. So if you’re running it on a VM, for example, then you have to think about patching the underlying operating system (that is your responsibility), and protecting the underlying operating system as well. In general, I think if I remember correctly from the Vault hardening guide, you wouldn’t even have SSH to that. So you need to think about what your consumption patterns are there. Are you going to have a thing where you kind of throw away a VM and deploy a new one with Vault, and it joins the cluster? And so on and so forth.

And then you have upgrades of Vault itself. Now, it depends on how you approach this. If you are someone that stays on top of upgrades, so when a new Vault comes out, you upgrade in some kind of development environment, you test out the application, you make sure it all works, and then you kind of roll it up to the rest of your environments, then upgrades can be, dare I say, straightforward. But honestly speaking, as someone who worked with many organizations who were hosting Vault, I don’t think I came across a single organization that were on top of upgrades.

So what will happen is they are on a specific version, then something will happen one day where they run into an issue, and they think “Oh, it’s a bug.” And maybe it is a bug, right? How do you fix that? “Oh yeah, we’ve fixed that already. It’s in this version here.” Okay, you need to upgrade to that version. How do you get through all of these minor, intermediate and major versions to kind of do that upgrade there?

Sometimes the upgrade path is not so clear to understand. Do you have to go to the next version, and then the next version, and then the next version? Or can you do a one-hop jump to your target version? And sometimes the reason why you upgrade is not even because there’s an issue, it’s because there’s new functionality and you want to utilize it to make your lives a bit easier.

So if I’m being honest, from my point of view, I don’t want to manage all of that stuff there. I don’t care enough for that type of operations. In fact, if someone will do it for me, I’d much prefer that, because all the things like your SLAs for downtime and so on and so forth become your responsibility and accountability… Whereas you have professionals who can look after these things here for you, and it becomes their responsibility and accountability. And that sounds super-sweet to me. Let me just focus on building the thing that makes the business profits, it builds a small market share, rather than thinking about things like Vault. That’s not really what I want to do. I’m just an application developer. It turns out I need to encrypt some stuff, it turns out I need to access a few secrets to get to different parts of our platform… Yeah, I just need to know where to get that information, or my application needs to know where to get that information, and let’s go. Let’s go and build greatness.

That’s why when Rosemary talks about the best way to get started is kind of HCP. It’s actually interesting, because I remember years ago that was one of my startup ideas was HCP. I was like, “Oh, imagine if I could just run people’s Vaults for them. That’d be super-cool.” And you know, I never really got down to the drawing board or figuring out what that would look like, and so on and so forth. And I’ll tell you what, the people here are HashiCorp have done such a tremendous job. A far, far better job than I ever could have imagined. But there’s a reason why I came up with that idea, is because honestly, managing platforms - there’s a lot of things to do. So if there’s one less thing that you can manage, especially when it underpins your entire security, I wouldn’t see why you wouldn’t want to offset that.

Jesse Grosjean: Yeah. So let’s see… So I wanted to do cool computer things in high school. That was my goal. And I was interested in graphic design mostly, because I liked Mac for some reason, I’m not sure why, and I wanted– Photoshop was the top of the… I graduated high school in 1995, so that’s the timeframe. And then I went to college and did computer science and studio art. Studio art was a minor, and more I like to draw and stuff, but… I like looking at visual things, but I’m not too much of an expert beyond that.

And anyway, so then after college, I did – again, interested in cool computer stuff as being visual, fancy buttons on the screen, not so much server algorithms, or things like that. I want to put things on the screen that I can click. And I went to a startup that was doing visual things. And then after a year of that I went and worked at the University of Maryland in their Human-Computer Interaction Department as a programmer, a research assistant kind of person. And that was neat in that it was lots of interface design kind of things. And in particular, they were doing zooming user interfaces where it’s a two-dimensional surface that you zoom into smoothly.

And so all of that was neat, and lots of experiments and cool demos, but in the end I also have this interest in taking notes on things, and I came to the conclusion that it’s really hard to get a zooming interface or any other fancy interface to be generally as productive as just like a plain text editor. If you put that information that you’re visualizing - and there are tons of exceptions, but I’m just talking about for a day-to-day, an average person. Well, let’s see… I put my grocery list in this text editor, and it looks just as good and as easy to manage as some fancier tool. And so that realization - which wasn’t immediate - that I came to, and I especially came to it with WriteRoom, where that was kind of just a fun idea that I did in a week and it got more attention and applause and money than stuff I’d worked on for years. So that got my attention, I was like “Hey, well…” So I wonder why that is, and stuff.

[12:20] One thing that I have come to think is that computers are magic. Maybe this is old news, but especially for like 1995, they’re the magic new thing. And I think that the real magic is pretty simple, in a lot of ways. For some use cases, it’s just like you can edit this text without retyping it, which is dumb and 50 years old, but that’s the big difference from paper. And all these things that we add on top of it are valuable, but if they get in the way of these core little things, like I can copy and paste text, or something… It’s always a trade-off and it’s taking away.

And so that’s been my assumption for a long time, is that for the kinds of things I’m interested in that are text-based, just a basic text editor is the foundation, and try to expand on that idea, but be aware that anything you do is going to have a plus and a minus, and that you might just be losing. It might be better to go back to the text editor.

Alex Sims: [35:52] You have to enjoy your day-to-day to get the most out of yourself… And not even just yourself, but your team. And one of the things that I learned over the last few years from someone that came on going “Everything needs to be shining. We need to be using this technology, and this technology.” And coming into a team with (I’d say) more seasoned developers, that have sort of weathered the storm of the shiny, and sort of said “We’ve got this tech. It’s done us solid for five years (at that point). We don’t necessarily need to upgrade.” As long as we’re following the upgrade path to sort of say “I’m on PHP five point something, and it’s now end-of-life, and I’m now bumping up to the next version, to make sure that we haven’t got any vulnerabilities at the language level.” Then I think we’re doing the right thing and we can carry on pushing ahead with it.

There needs to be a real reason to break away from doing something the way that you’re doing, in order to justify that cost of adding that extra complexity to the stack… Because every time – like, now we’re out of service. We’re adding additional complexity that someone needs to understand. If I were to walk away, who else on the team can pick that up and understand it?

And one of the important parts in my role now is everything that we’re building is being thoroughly documented. From the perspective of architectural diagrams this is how different parts of the system interact with one another, this is the contract that this service has exposed to the rest of these services… Once you understand that part, everything else just becomes black box. And if I were to walk away, then someone should be able to come in, reference the documentation and go “Okay, I can see how we push messages to this service now. This is the flow.”

Back to your original point on how do you get other people excited and make them want to learn the new things - I think that’s a hard one. Like I said earlier, when I came in I was trying to hard-sell React, and “We need to use Laravel, we need to get rid of Symfony 1.4.” Coming in and just saying those things without really being able to say “This is why” was a really hard sell. But over time, we’ve seen that maintaining Symfony 1.4 is hard, especially when we jumped from PHP 5 to PHP 7.2 [unintelligible 00:38:08.08] on GitHub, and they have been maintaining that Symfony 1.4 fork for years, and upgrading it with every major version of PHP and minor version of PHP that comes out. But that’s only gonna happen for so long. If anyone decides to stop maintaining that, then we’re in hot water when PHP 7 goes end of life and we go to 8. I think we’re already looking at how we can start moving to 8, and then fortunately I believe that fork is still maintaining that version. But yeah, there comes a point where you have to pull off the plaster and start thinking about how do you move to something that’s gonna be sustainable.

Deepthi Sigireddi: So there’s definitely a hardware cost to it when you start deploying Vitess. Typically, what people will do is they have their MySQL instances already, and they’ll just put Vitess in front of it, and start routing the application queries through Vitess. Obviously, you have to run these additional components that Vitess brings in. For every MySQL database that you have, you run a management component called vttablet, and you run a bunch of proxies which will receive the MySQL queries connections from clients, and then pass them to the tablets which will eventually execute those queries on MySQL. So there is a hardware cost associated with it.

There is also compatibility. So Vitess is MySQL-compatible, but MySQL query syntax and constructs keep evolving. Vitess started (I think) maybe on MySQL 5.5, and we now support almost all the constructs in 5.7, but not everything that is 8.0, because a lot of new syntax was introduced in 8.0. And where this becomes hard - it’s easy if you’re unsharded. You sort of just have to understand the syntax and you can pass it through. But in a sharded system, what you end up doing is that you take each query and you have to plan that query, and you have to say which of the backing MySQL should this query eventually go to? Because it’s not going to go to all of them; it’s going to go to either one of them, or a subset of them. In rare cases - yes, if you do select star from a sharded table, you have to go to all of them. But usually, you want to go to a subset or a single one. And query planning is where we figure this out. What that means is that any new syntax or construct that MySQL introduces, we have to be able to understand that, because maybe it’s a function, and the function is being applied to the results of multiple rows, and you have to know how to implement that function in Vitess in order to still provide that transparency. What’s another example…? Or it’s a join, and if it’s going to be a cross-shard join, then you have to do some of the processing in-memory, after fetching the results from different individual MySQL databases. So all of this is happening in Vitess, which means that to this date there is a compatibility gap between Vitess and MySQL, and we are trying to close that.

[48:16] We started the compatibility project formally in… I have to think. I think January of 2020. So it’s been two years. And we are still working on it. We have a sub-team at PlanetScale, all of whom are Vitess maintainers, who focus on closing this compatibility gap. And the way we did it was we said “Okay, we’ll take some popular developer frameworks, like Ruby on Rails, or try to run WordPress on Vitess and look at the queries that are being executed.” And especially with frameworks, they execute some preamble queries, some information schema, metadata queries, and so on. And then we start adding support for those things.

But to come back to the original question, which is “What does that journey of adopting Vitess look like for someone who starts on Vitess?” You may find that there are some queries that you are using which don’t work with Vitess in a sharded mode. It’s much rarer with unsharded, but there was a user on open source who said, “Oh, I’m using common table expressions. They don’t work with Vitess.” And I had to say “Yeah, they don’t work with Vitess because we haven’t added support yet.” But most of the time it’s not unsharded but sharded that causes the incompatibilities. So people will take a test environment, they’ll have some test data, they’ll put Vitess in front of their MySQLs, they will start sending some test traffic through Vitess, and then they’ll discover what other things that don’t work. At that point they have a choice of either contributing a fix upstream to make it work, or changing how their application works, so that those queries are not produced anymore.

So that’s the sort of journey people tend to go through when they come to Vitess. And then, once they know that either all the queries work, or we’ve made the - usually not big changes; minor changes needed for the application to work, then you can start moving from test into production.

Kris Brandow: Yeah, and there’s a lot of legitimacy to that… And I think part of the issue that I’ve had with adding the version into the name is that it kind of gives an extra amount of credence to semver, of saying “This is the correct way to talk about your API, to talk about your library that you’re building.” I think that’s probably not what we really should have done as a community, and I think that will inevitably just push us to essentially not use the major part of semver, as you’re suggesting, Peter, and just have basically a major and a minor version, instead of having a major/minor patch, or having a major on a patch version.

So in effect, what we’ve done is we’ve kind of taken away a common understanding of what that major version number means to people when we put it elsewhere, which I don’t think is particularly good for the long-term of our community. I understand wanting to go with something like semver, because I think it is widely standard, but I think this is also the conversation we as a community and we as an industry need to have around semantics, because they are hard.

And in some of the conversations that have come up around this too, I think that there have been arguments of like “Well, our tooling should be more nuanced. We should have ways of deprecating specific parts and pieces of our API.” Like, “This function is insecure, or this method, or this type is insecure. Please don’t use it”, which I understand…

Jean-Sébastien Pedron: So after that FreeBSD 5.0 disaster the release engineering team started to work on something so that FreeBSD never faces that situation again… And that process evolved a couple times since. Today, the FreeBSD release engineering is based on a fixed interval between major releases and also minor releases. We don’t expect to start on a very specific day at 8 AM, for instance. The OpenBSD one is sharp as a Swiss clock, but not in FreeBSD.

When we want to start to prepare the next release, we have release engineers or someone who is hired by the FreeBSD Foundation and is paid for that; he will take care of announcing to the FreeBSD contributors - not only the contributors, but the entire community. He will publish a calendar where he will state that the code [unintelligible 00:21:52.14] We expect to get the first beta at this date, we expect perhaps two betas, then two release candidates, specifying, again, the dates. He will indicate as well the date for the final release of FreeBSD. So that calendar is updated on a regular basis while we make progress in that release cycle. For instance, if we discover that there are bugs or there is a security issue, or whatever the reason, we might want to delay beta for a couple of days, or we might want to add third or fourth beta, or same for the release candidates, and so on.

So that calendar is very flexible, but it’s quite useful, because it tells to the FreeBSD contributors when to expect things, and it’s very easy for contributors to organize and prioritize their tasks. For instance, if someone is working on new features, then he knows that he has to finish by these dates, or it will be delayed to the next release. So that’s very helpful for contributors, and like I said, this is not that strict. So any contributors can communicate also to the release engineer what he is working on, so that the release engineers know that “Okay, this specific batch is incoming. It might introduce some instabilities, but we want that in the release”, so he can anticipate that and perhaps tell anyone that “Okay, we expect this to come in the next couple of weeks. This will go in that beta and we will add another one after that”, for instance.

[23:52] So that calendar tool is really useful, because it allows everyone in the community and the developers to communicate and understand what’s going on. As I said, for users who will use that new version of FreeBSD, they can plan for testing, for instance. You mentioned Netflix - they’d appreciate that, because they can test in advance the new features, so they will fetch the development branch, for instance, compile FreeBSD and then try it in their environment and see how it goes, and they will give some feedback.

So the fact that we use a detailed calendar - yeah, it really helps the communication and it makes the whole process more reliable and the outcome more reliable as well. I think that’s the main part which was introduced following FreeBSD 5. And we have some evolutions from time, but they are mostly around adjusting the timeframe between releases, so that it’s easy for end users to understand that “Okay, this will come in next September. Perhaps the release will take a bit more time, but in next September - okay, we know that we’ll have a new release.” This would have been very helpful in the time of FreeBSD 5, because we could have delayed some of the work done around looking to a future version, for instance, instead of trying to finish that huge task before shipping anything.

Suju Rajan: Building off of the standardized version of us looking at the data that we have… Daniel, exactly to your point, we do have a notion of what a member embedding needs to look like. The data for LinkedIn comes from LinkedIn profiles. So in this sense, understanding that “Hey, this particular member has a – in that space, it looks like people who are in this technical industry, in this location, and here are the sorts of jobs that they apply to”, as much as we can represent them in that member embedding space.

Likewise, we have things that are on the job and they’re inside as well; someone asking for a data miner, maybe because they were under a rock, is also still asking for an AI scientist, so how can we bring that job embeddings view into it.

And in some senses, being able to scale this. Because if you think about where – there was an earlier question of “What are the different services in which we have AI?” There is the search aspect of it, and we also organically recommend jobs to people - something that we call instant jobs, where you want to be the first to apply… So members actually sign up for alerts, to say “Hey, the moment you see a job come into our system, then send me an alert right away.” So we need this to be high, high, high precision, as an example.

Then on the flipside, on the recruiter search as an example, it’s all about matching members to get jobs. So we’re trying to build this notion of what we call these two-tower embeddings, which represent the member side and the job side, and bring that together along with more near-line and real-time features, to sort of then personalize it to what that particular member is also looking for, to do ad matching.

Of course, if it’s on the search angle, then you also have the query context that comes into play. On the recommendation side it’s a lot more just based off of your activity signals, and so on and so forth. We try to infer intent, to be able to make those recommendations.

But to your point - yes, on the underlying surface we do have these embeddings, and we are going to be writing a blog post pretty soon about this particular data layer in our talent solution systems.

Jason Warner: The most important one, if you think about how you might make a decision in this area, my lens is pretty exclusively the largest lens I’m going to use, besides the micro-lens, and it’s going to be “Does this effectively change all the software at once?” Because with GitHub’s market position - we have 40+ million registered developers on GitHub, 100+ million repositories, and all that sort of thing… Basically, all the software lives on GitHub. So we’re in a position that if we do something, it should be meaningful to nearly everybody in the world who cares about software.

So if it comes down to that it’s a feature, or it’s a nice-to-have, or it’s a shiny button type of scenario, that is not what we’re after. We’re after something that completely changes it again. It either takes software from level A and levels it up entirely to level B - that’s what we want to do.

[12:01] So when ideas come to us, is this basically worth our time to go look at this? Will this change everybody at once? Will this change the industry? Will this make something x% easier - or x times easier, which is actually what were after - or x times better? That sort of scenario. That’s how we have to think. And the truth is that most things that people come up with are friction-reduction ideas. “This thing is hard” or “This is hard” or “This is awkward”, but that’s not what we’re after. We’re not after minor percentage change friction-reduction if we wanna change everything. The minor change friction-reductions are incredibly important to GitHub, but that’s what Horizon 1 and 2 are for.

Josh Aas: [01:11:54.27] Another thing that both affects us and that I’m personally pretty passionate about is memory safety. In the same way that it seems crazy to us now that you would start a major website and not use HTTPS - we know so much about the risk of that and it just seems crazy to do that now - I think we’re also gonna come to a point where we feel like it is crazy how people say stuff like “Well, I’m running a bank and I need to do some reverse-proxying, so I’m gonna spin up an instance of NGINX or Apache and do my reverse load balancing.” Because what you’re really saying there is “Why don’t I just stick several million lines of C code on the edge of my network? And that’ll probably be fine.” That code is not safe.

We certainly should not be writing any new code in C and C++. The opportunity for vulnerabilities – I mean, it absolutely will have vulnerabilities, and we need to get that type of code away from our networks to start with, and then probably away from most other things, too… So I would hope that in 10-20 years we think it’s crazy to be deploying major (or maybe even minor) pieces of software that are written in languages that are not memory-safe.

So we’re trying to remove code written in C and C++ from our infrastructure at Let’s Encrypt. I think that’s just a basic part of diligence applied to secure infrastructure. If your stack is some giant pile of C++ or C at the network edge, followed by OpenSSL written in C, followed by a Linux kernel written in C, glibc - your whole pathway has got all this code that you just know is full of security holes. It absolutely is. You just can’t claim that those are even close to secure systems. They’re absolutely not. We’re gonna look back on this and say “That was crazy. We have better options today.”

So we’re trying to remove that kind of code from inside Let’s Encrypt, because it’s a huge liability, but we/I am looking into ways that we can try to move the needle on this problem in software in general.

Emily Robinson: Absolutely. I was lucky enough when I went to college at Rice University to be there when Hadley Wickham was a professor. For those of your listeners who might be not that familiar with R, or do more in Python, Hadley Wickham is one of the most well-known R programmers. He’s created some of the most popular packages, especially focused on data analysis. I started learning R in university, from classes he had designed.

My major was one I created myself, called Decision Sciences, which was focused on the social sciences, with a minor in stocks… And then I went on and got a master’s degree in organizational behavior.

That was actually a part of the Ph.D. program, and I decided two years in that Academia wasn’t necessarily for me, so I off-tracked, and I got my master’s degree. Then I came back to New York, where I’m from, and did a data science bootcamp Metis.

[03:54] The reason I was drawn to data science is the process I found was quite similar to the social science research process. You would come up with the question that you want to investigate, find the data, analyze it, and then present it to people, whether that’s in Academia, someone very much in your niche, or sometimes to a professor in a totally different department. What attracted me to data science was being able to do that, but for companies, making an impact, and on a bit of a shorter time scale than the seven years it can sometimes take to publish a paper.

After that bootcamp I went on to work at Etsy, and then at Datacamp, doing data science for both of those companies, and specializing in A/B testing or online experimentation. Now I’m here at Warby Parker… I started back in December, so I’ve been here a little over three months now.

Chris Wilcox: At this point it’s up to – I would say it’s three or four. The levels are a bit fluid, if you noticed from the talk… They’re based on something to do with automotive engineering, to driverless car leveling and understanding… So it’s just really a way to frame risk and reward, honestly, more than anything else. It’s about a level – probably a 3, maybe a 4 at this point.

The thing that made the change for us is we go as far now to auto-detect if we ought to publish. So as commits come into the main branch, we can detect that there are new changes, and we use a thing called conventional commits. At the front of every commit is a label - be that “Chore”, “Fix”, “Breaking” - that it’ll also detect “Is it a patch, a minor or a major release?” We can auto-generate changelogs… And from that, really the only thing you do as a developer on the team at this point is merge the PR, and everything else is done for you. So we still control whether or not we publish to npm, but the rest is fully automated.

Chris Coyier: Yeah… Except for there’s a few; they’re minor, but they have major life impact, I think. One of the points is that, is the job title thing… And this highly affects hiring, which is a big deal in people’s lives. I can look around at job boards - you could do this right now in 30 seconds; go look at the CodePen job board, or the Stack Overflow job board, or Smashing Magazine’s job board, or anything that is general web technology related jobs, and you’ll see the word “We’re hiring front-end developer” all over it. Everybody’s hiring front-end developers.

So there’s a couple of angles to that. One of them is the reason that’s happening is because companies can put them to work so well. There’s so much more stuff that front-end developers can do. They’ve been kind of unhinged, or let free lately. The front-end developers can be super-productive. So they are, but the reason they’re so productive is because of the swathe of things that falls under the front-end developer bucket is so wide that it just depends… That’s what’s so tricky about this.

And then somebody with a certain front-end developer skillset will look at those job boards and be like “Yee-ha! Look at all these jobs. I’m gonna apply at a bunch of them”, only to find out that they really don’t have what that particular job is asking for.

So there’s been people that are like, “Well, I’ll solve this this afternoon.” Fire up Medium.com, “Here’s my new take on what job titles should be… If you’re this, that means you have this, this and this skill. If you’re this…” You know, “I’m a front-end engineer, I’m a front-end designer, I’m a UX engineer, I’m a UX designer”, and they just declare what the new job titles are in the world and that’s that… And that’s been tried so many times that it’s like – that doesn’t seem to be working. Maybe that is the final answer here, but it’s gonna take more than – I don’t know what it’s gonna take, but it’s gonna be weird.

So this hiring thing, for now, what you can do if you’re a company is be like, “Okay, maybe I have to put front-end developer in the job title because it’s grabbing people, we’re getting applicants that way”, but you have to be a lot more descriptive than you ever were in the past about what that role is.

If you’re saying, “I need a front-end developer to work at my agency to design WordPress themes for medium-sized clients”, you need to be clear about that. “We use SaaS generally…” Not that these things can’t evolve, but usually when companies are hiring, they’re hiring for a pretty specific – like, they need somebody right now with some of these skills, and that person can come on and evolve those things, but hiring is tricky. You need that skillset right now.

Or I’m hiring somebody to work on my VuePress site, that uses styled modules, or who knows what. It’s just kind of a different section of front-end stuff, so you have to just use your words a heck of a lot better than you have in the past.

Erik St. Martin: And even for failures, right? Something changed and you don’t understand why you’re noticing a new failure in production. All you did was upgrade say one library and make one other minor change, right? You don’t know what changed in the library to understand why now you’re getting socket connection errors and stuff like that, right? You start looking for a hardware problem because you’re like, “This library has always worked… What’s going on with the system configuration?” or something like that, and you come to find out it’s one line of code that changed in the library and you didn’t notice that slight change in semantic version, like a patch version that broke something.