Jerod Santo: Another thing that’s happening is that there’s more opportunities now, in addition to the speculation, because there’s now more use cases. We’re also in a bull run for cryptocurrencies, so that’s gonna bring in more people, of course… But back when Ethereum first launched, you had the DAO use case, the ICO, so you had the crowd investment use case… There really wasn’t much else you could do with these things besides speculate. And then the DeFi thing started to happen - decentralized finance - and I think there’s a lot of snake oil in that whole deal, but maybe it’s just unregulated markets; I don’t know, there’s some features there for lending stuff that’s cool, maybe… And then what happened recently, in addition to the stuff you all are talking about, is NFTs really have become this interesting, wild, new use case for these decentralized networks… And a lot of people wanna mint NFTs and don’t know how to… So what do they do? They hire somebody who can do that for them.

So there’s lots of opportunities to get involved in and around building auction houses, and minting platforms… So this is really brand new. I mean, 2020, 2021… I’m not sure how long it’s been going, but the NFT thing is very real. Now, maybe it’s just a fad, but it seems like a legitimate use case that’s bringing lots of people to the table.

Jerod Santo: So, since Bitcoin is back in its bull run… It’s just been a few years in a bear market and then came roaring back from something like $4,000 a year ago, to 60k per coin, and around there today… People are interested once again in this crazy world of cryptocurrencies, and so we thought it’d be fun to explain Bitcoin like you’re five, because it is a difficult thing to explain, difficult to talk about, a lot of ins, a lot of outs, a lot of what-have-yous…

[19:57] The first way I would like to explain this is by stealing a tweet from Theophite on Twitter, which I thought that was a hilarious way to explain Bitcoin… Somebody tweeted out “I still don’t get Bitcoin”, and Theophite replied “Imagine if keeping your [unintelligible 00:20:13.18] for 24/7 produced solved Sudokus you could trade in for heroin.” [laughter] That’s pretty classic, right?

So of course, there’s your bear explanation, and there are lots of downsides with Bitcoin specifically around energy use and whatnot… But there is also some real value to the network. And I’m not gonna explain how Bitcoin works like we’re five, but I thought maybe I’d try to explain why some people think it’s valuable like you’re five… So here we go.

Imagine that you’re playing Monopoly with some friends… And one person decides - maybe they’re the homeowner. Usually house rules… They decide that they’re gonna be both the banker and the rule enforcer. Pretty typical. Everything starts off okay, but then the banker starts doing some things that you don’t appreciate as someone who plays the game. For example, they start injecting new money into the game, but not equally to everyone. Just to themselves and some of their friends.

They change the rules about how much money you get for passing Go, what you can go to jail for and for how long, they create over-the-top transfer fees and laws about who can trade with whom, etc.

Eventually, you and a few other people in the game decide that you want to transact without the banker’s rules, permissions, fees and so on. But you can’t use the Monopoly bucks, because the banker runs that whole system of things. So instead, you start a grouptext on your phones. And each time you wanna transfer money to and from each other, you put that in a text. Maybe you text “I’ll send five Schrute Bucks to Kball.” And then you text that out.

Everyone involved in this new way of sending money is in the grouptext, so that’s how you keep each other honest. Bitcoin is the grouptext. The end. What do you think?

Jerod Santo: It’s just too easy. Alright, back to some bad news… Polyfill supply chain attack hits 100,000 plus sites. This one has made the news a few times, and so it’s not exactly new news… But I wanted to bounce off you what I said on Monday on Changelog News, and get your take on it. Because in light of this, which - for those of you who didn’t hear this news, there is a JavaScript library called polyfill.js. It provides, as is the name, some polyfills for features that don’t exist in older browsers. It was served up via a CDN that the polyfill people set up very kindly. Cdn.polyfill.js, which I’m sure served tons of bandwidth and traffic for many years, probably free of charge, subsidized by whoever it is that was putting out the open source… And eventually got sold to a malicious company. The domain got sold, polyfill.js. And so they set up a CDN at the same address, that did some nefarious things. And so this is kind of your “Rug pull, not cool” situation. Different kind of rug pull. Not an open source run pull. A domain rug pool.

Preston Van Loon: We have many layers of testing. The first obvious one is the unit testing, keeping things tested in a small isolation. We have fuzz testing, we have what are called spec tests, which are basically conformance tests that are produced by the reference – we have like a reference implementation, and all of the different client implementers can download these tests. So basically, it says “With these inputs you should expect this output.” We run all of those. We run full end-to-end testing, and most of these things are done in a code pre-submit. So actually, if you’re proposing a change, we’ll run the change to a simulation for about ten minutes, and basically it’s a smoke test to see “Does the world continue with your change, or does it blow up?” That’s a great way to stop really easy bugs that way.

And then we have a pretty long process of pre-production soak. So with our software, we put it out into the test network, where there’s no real money at stake, but there is an environment that’s very close to the simulation of the real world. We’ll run it there for some medium-term amount of time, and be able to check “Okay, is there performance regressions or new logs that are appearing or anything kind of weird that would be concerning for this kind of release?” and we call that the pre-production soak. And we also do canary testing on top of that.

Erik Kennedy: Yeah, for sure. Learn UI Design is a video course, it’s got about 40 different lessons, 20 hours of content, and it’s really just the full run of tactical design advice. It’s the same sort of stuff that we’ve been talking about here, but in a format where you kind of watch over my shoulder as I design sample projects, or go through sample designs to talk about different concepts… All the fundamentals, things like alignment and spacing, there’s a whole unit on color, a whole unit on typography, a unit on process… There’s tons of stuff there, but it’s basically for people who are maybe developers, UX designers, maybe graphic designers too, but someone who says “Yeah, it’s gonna be worth my while to have UI design be this career skill of mine.”

Maybe you’re not necessarily looking to call yourself a designer, but you know that this is gonna be something that you just want to be able to do competently… And yeah, so all students get lifetime access, and that includes all the updates and stuff that I’ve been making.

There’s also a student community, and a Slack channel, and I don’t remember if I mentioned it, but there’s a bunch of homework assignments, which is a pretty significant part of it… It’s kind of just “do at your own pace”, but they’re really just a bunch of exercises to focus you in on mastering the skills that the different videos talk about. So it’s the whole deal, yeah. LearnUI.design.

Erik St. Martin: Yeah, anything that will run full Linux, so we can program in Go. But then I also like playing with the small ARM processors and things that Go does not run on. So I’m really excited to be able to try to merge those worlds…

Now I have a – I think I sent pictures out over Twitter, of this embedded board that has a small touchscreen LCD, and it has a Cortex M4 on it that I’ve been waiting to mess with our barbecue stuff on, and I was gonna have to do it in C and C++, but now I may have to play with Emgo.

Jerod Santo: Okay, so it’s not exactly the way that I was thinking that it would work in terms of the – it’s checking live, so to speak, each time it’s run on a schedule, or if there’s a webhook that comes in, of course, it knows and it sends the new version of that particular dependency. But when it does a full run on a specific project or repo that you have, it’s going to go to all the sources - in many cases, that’s just npm - and just check for the most recent version of that library. It’s gonna do that for each of your dependencies, each time you run it?

Erik St. Martin: And they are on getchip.com, if you’re interested. There should be a link that says PocketCHIP. They’re fairly inexpensive; I think they’re like $50 or $60, and they’ve got like a keyboard on them and they run full Linux, and you can install Go on there, and they’ve got some pin-outs and stuff like that too for doing GPIO… So it makes for kind of a fun hacking project, outside of using Arduino… You can actually do Go.

Brian Ketelsen: Exactly. So you can use it for just that concept, the cancellation; you can use it to run lots of goroutines in parallel and keep them synchronized, or you can use it as a pipeline to run pull data between goroutines and still capture all of the errors in between them. So it’s a neat package. I intend to use it.

John Nunemaker: Yeah. Well, the slowest part was the bank, and attorneys, and things like that at the end. The part of “Are we going to do it?” and “How much are we going to buy it for?” was like a week. That was really quick. I talked to Dan once, we hit it off, we enjoyed each other’s companies, and stuff like that… Just a phone call, nothing fancy, for like an hour on a Saturday night, Sunday night, I don’t remember… Props to my wife for all the extra time she did single-parenting while I was doing phone calls and stuff on this… But we talked, and I was just like – and then he sent me all the info. So I’ve looked at a lot of P&L’s over the years because of Box Out, and because of due diligence and stuff there… So I was like “I’m comfortable with that.” And I looked at it, I was like “I think this is what I would feel okay with.” The amount we ended up at, it was like – I’m like “I’m a nervous guy. I run the ball out in the fourth quarter in the football game. I’m not aggressive, by any means.” The whole reason I have multiple things is in case one of them goes bad. I just diversify everything. So yeah, from that standpoint I was like “I’m okay with this number. Are you okay with this number?” And he’s like “Yeah. Because it’s you guys, I’m okay with this number.” And so we got to that and we’re like “Okay, cool.” And then from there it was like “Okay, who’s going to be in and out of the company? How much ownership are they going to have? How much capital are they going to put in for the ownership, and how much is going to be work-based?” and all that kind of stuff. And then is it a separate company? Is it a stock sale, or an asset sale? All that kind of stuff.

And then fun things at the end with the bank, where they’re like “Oh, FYI, if someone has 20%, then they have to also personally guarantee it. And then we have to take it back to our bank board, literally the people that run the bank, and get it approved again. And that takes at least another week.” And I’m like “We’re supposed to close tomorrow.”

Jerod Santo: So the full dry run move - was that your second iteration, or that’s still happening now, like with your released version?

4:Ooh, so this is – it’s a little bit mixed now in our third iteration. The second one was we tried a fool dry run. We would just invoke the npm CLI using –dryrun. This is kind of the recommended way to do it by npm configuration currently. But it just tells you it added X number of packages and removed X number of packages. That’s the only information you can get out of the CLI.

John Spurlock: I’m glad you’re doing it, because there’s a huge chicken and egg problem in podcasting, because let’s say there’s a new standard, or – like, you were doing chapters. That’s a great feature. But until all the listening apps implement chapters, it’s hard to make the sell, right? It’s hard to say – you almost have to go first as being the publisher. So it is chicken and egg, but I like to say really hosts need to go first on a lot of these; and you are the host in this case.

So I’m really glad that you guys are taking the opportunity to put this info in there, and hopefully, we’ll pick up the ball and run with it and add new apps and stuff that light this kind of thing up. But you can’t do it if the data isn’t there.

Mark Ericksen: And that’s just a generic situation that we have to deal with when we’re wanting to provide non-interrupted service, or that the user is not going to all of a sudden hit an error, their query failed. So there’s strategies for doing that, and sometimes that’s a multistage deploy where you might go halfway to the migration, to the full change that you want in one deploy, and then follow it up afterward with a second deploy that finishes it. There’s strategies for that. But to answer your question, the way Fly does it is it’s a rolling deploy where a new one will go out, and health checks have to pass before it’s considered, “Okay, it’s up and good”, and then it will roll out to all the other instances. And there’s a special instance that’s run very first, that runs the migrations. It doesn’t actually run the app. It starts it up just enough to run the migrations, not to actually run the full instance. So it says, “Okay, can we run the migrations?” Because we’re only going to run them once. We don’t need to run them every app, having, say, “I’m going to run the migrations now”, and trying to do that and discovering, “Oh, it’s already been done “, or having to deal with any kind of locks or queries or anything like that.

Evan You: Yeah. So first of all, Vite covers both the development part and the build part. So during development it is a dev server, and during build we actually run a full bundling process using Rollup. So if you use WebPack, you would have to install WebPack, WebPack CLI, WebPack DevServer, then you need to configure the DevServer to do the same thing with the actual build. But Vite tries to just have all these things in one package. So when it comes to the modules, it handles TypeScript out of the box using Esbuild. So it only transforms the syntax. It strips away your types, spits out plain JavaScript; it doesn’t do type checking. One of the reasons we do that is because TSC type-checking is really slow. So having that to be part of the module transform pipeline is actually a major performance killer.

[23:57] On the other hand, if you use something like VS Code, your VS Code already runs a TypeScript language service, and you already get all of this type-checking, but you’re working on your code. So why would you wanna do it as part of your build process, to just slow it down?

Ville Tuulos: Yeah. And the interesting thing is – I mean, there are so many kind of systems that look like that, and I always say that the devil is in the details… Many systems – let’s say like the tons of systems that let you specify workflows in YAML, and oftentimes they say “Oh, you can run any Docker container, and you can run any code inside the Docker container that makes a step”, which kind of like, as a user, it kind of like pushes the hardest problem to you, that “Okay, wait a minute… So how do I define what code runs in this container, and where do I push my container? And what are the dependencies I need inside the container? And how do I move data between these containers?” So in a way, just like having a workflow, that’s kind of the easy part. That’s why I wanted to have this self-contained thing in Metaflow, that you have everything in one place; so you define the code, you define the dependencies, you define the resources, you define the workflow… Because then you get [unintelligible 00:37:43.15] that you can actually run the full thing in production and it does something useful.

And by the way, one thing that I definitely want to mention, which is really important, is that this is never a waterfall. It’s never so that you prototype, and then you deploy, and then you declare mission accomplished. If the project is successful at all, what inevitably happens is that either something fails in production, in which case you have to go back and start debugging, or the business stakeholder or whoever comes to you saying that “Okay, now we want better results. Can we improve accuracy? Can we add this new dataset?” and whatnot, so you kind of have to start iterating again…

That’s really the challenge with many systems, that even if you are able to do that one deployment, how do you come back from production. Everybody always talks about going to production, but how do you come back from production, and then keep iterating, and maybe start having multiple versions running in parallel? So it’s all these small things that really matter a lot when you think about running ML for real.

Bryan Boreham: [36:02] I work a lot with Prometheus; a lot of the Prometheus maintainers work at Grafana Labs… So that tends to be my go-to technique for looking at what’s going on, particularly at scale, if you’ve got 100 programs running and you wanna keep an eye on what the bulk of them are doing. It’s basically the same data. It’s coming out of the Go runtime, but they show up as things like go_memstats_heap_alloc_bytes. That’s one of the core ones. That’s how much memory has been allocated. If it’s going reasonably slowly, you can see that number rising. It typically looks like a sawtooth. It rises quite slowly, then it drops very sharply when we free everything up in that full garbage-collect, and then it rises again.

What might be happening - your program might be garbage-collecting like ten times a second, in which case the metrics are not going to pick that up. It’s just like a blur. And unless it’s a really, really small program, you do not want the garbage collector to be running ten times a second. Once a second is more moderate, or once every 30 seconds, or something like that… It’s a trade-off; you’re trading off how much memory you’ve got, against how much work you want the garbage collector to do, against how big is your program… There’s no hard and fast rule – well, there’s one hard rule; if you’re doing garbage collection, Go will run the full garbage-collect every two minutes. That’s the slowest it will go. That’s hardcoded into the runtime. But if you watch the rate at which garbage collections are happening, which is a different Prometheus metric, if you’re doing it that way – I’ve forgotten the name… But if you watch the rate at which it’s happening, I think a lot faster than one a second is probably too fast. And you can’t really go too slow, because you either run out of memory or you hit that two-minute point.

Kevin Ball: I feel like the web’s security model was by necessity pushed to a place where things had to be sandboxed, they had to be secure, because suddenly you’ve got all of this untrusted code that’s gonna be running, and now WebAssembly is basically allowing us to say “Hey, that’s a good idea for any type of code we might wanna run. Let’s pull that in.”

Saul Pwanson: So I’m so much more willing to work on something if someone’s subscribed by Patreon, if they have an idea. And like I said, it’s not that they’re paying me to do it. It’s not worth it for me to get three dollars to do whatever feature that takes me four hours… But just the mere fact that they’re invested in any way is motivating. So that’s the reason behind the Patreon, it being a Patreon thing.

And I don’t actually take money out of it as an income for myself. I have a full-time job, so I don’t really need that money in a certain sense. What I do with that money is I run other experiments. For instance, this last winter I was working – there’s a guy whose name is Tom Buckley-Houston (I’m hopefully pronouncing that right, Tom). I wanted to see if I could get VisiData on the web. So he makes brow.sh (browsh, he calls it), which is a text interface to the browser. You can basically run a full browser in your terminal, and you can do things like play YouTube videos through it. It’s totally great.

Jerod Santo: [laughs] That’s kind of the beauty of open source though, right? Somebody else is interested, and then they can just take the ball and run with it. It’s beautiful.

Russ Cox: Plan 9 was a lot of things. I mean, it really was its own world, and it pre-dated Linux at some level, and in addition to the operating system, it had this whole way of thinking about resources, and presenting them; then I had all these simpler commands, and UI… It was derived from Research Unix, but it was not Unix. So there was a lot that was Plan 9 that if you think about the experience of using Plan 9 – there was the whole system, it wasn’t just the operating system kernel.

People still run the full system today, and then in addition to that, when I was in grad school, I tried to use Plan 9 for a while, and eventually I realized that it just wasn’t gonna work out, mainly because everyone else in my group was not using Plan 9, so it was very hard to coordinate with people.

So I took all of the userspace software and I ported it to Unix, and it now runs on pretty much all the Unixes. That’s called Plan 9 from User Space, or plan9port, and if you do a Google search for plan9port you’ll find the source code and you can download and install it. I still use essentially all the tools from Plan 9 and Rob’s editor Acme and all of that. So yeah, I still use Plan 9 every day at some level, even though I’m on a Mac.

I actually really think that a lot of the spirit and the way that we approached structuring programming in Go owes a lot to Plan 9. Plan 9 had its own language at one point that had concurrency; we didn’t call them goroutines, but it had goroutines, and it had channels, and it was the place where we really experimented with all this stuff… And we took what we learned and we moved on to other systems; I can run the editor on the Mac, and we can run – all the concepts that we had on Plan 9, we can do them in Go now. It’s great.

Jerod Santo: Wow. That’s the beauty of open source right there - worldwide, somebody in Australia you never met, and he can pick up the ball and run with it. That’s really cool.

The Wired article – Atom was featured in Wired around the 1.8 release date, which was middle of 2015, and then that article states that Chris Wanstrath had imagined this editor; he loves Emacs… Emacs, of course, very powerful, but written in Lisp, and you extend it by writing Lisp. Chris wanted kind of an Emacs style thing with modern web technologies, and he had imagined it in 2008.

Felipe Hoffa: I would love to highlight this about BigQuery. We can find open data all over the internet - that is awesome. But what’s special about data shared on BigQuery is that anyone can go and immediately analyze it. Everywhere else you have to start by downloading this data or by using certain APIs that restrict what you can do. When people share data on BigQuery, like for example the GitHub archive that Ilya has been sharing for all this time, they say it is available for immediate querying by anyone and you can query it in any way you want. You can basically run a full table scan that runs in seconds without you having to wait hours or days to download and analyze the data at your home.

Jerod Santo: And obviously, the AI gold rush is on… And so if you think about what the pick axe is for the AI gold rush - well, you may think it’s the models. It seems like maybe that’s kind of true, but also in the long-term is it going to be false, as those become commoditized? But maybe the ability to deploy the models, the tooling around actually taking that stuff from a Jupyter Notebook into production in a way that’s reproducible and not expensive, and doesn’t run your cloud bill up insane is the kind of tools that people are going to need in order to find the gold, so to speak.

Gerhard Lazu: We don’t deploy. So we now run tests, by the way… We didn’t use to run tests in pull requests. Oh, dang it, I don’t know how I overlooked that thing…

Chris Ferdinandi: Yeah, for sure. The tooling is wide and varied, and a lot of it depends on your, I guess, really preferred stack. I tend to use Hugo for all of my things, because I’ve found the Ruby templating in Jekyll confusing. Not that Go is particularly clear either, but it renders really fast. And I’ve found that I really like old-school static site generators because they’re mostly HTML, with a little sprinkling of whatever your templating language of choice is. So Eleventy is another really good choice.

The one thing that has kept me from migrating to Eleventy, honestly, is I do not want to deal with maintaining Node.js on my server. And again, we’re shipping flat files, but I have a process that involves, like, deploy to GitHub, send a webhook, pull the code, run a build on the server, and I just don’t want to deal with the whole node on the server thing. It’s a pain in the butt enough to do it on my own machine, never mind something I have to SSH into.

But there are tools for any kind of workflow. So if you enjoy working with React - and we mentioned Gatsby has kind of fallen out of favor a little bit, but you can use React in Gatsby for templating, and that’s totally fine. These days, you also have like, Next.js. If you like Vue, Vue has kind of a – they have a bit of a static site compilation thing. And then I think the one I’m probably most excited for, just because I think it’ll reach the most people, is the new kid on the block, Astro, which is heavily inspired by Svelte, and works in many of the same ways, but lets you pull in components from all of your favorite JavaScript libraries. So if you have a drop-down menu component you like from React and a card component from Vue, and as Svelte project you started working on, you can mash them all together, and it’s going to run and spit out plain HTML and some Vanilla JavaScript, and it’s going to shed all the libraries for you.

Amal Hussein: Wow, I never thought that pull request and run was a concept. It’s a bit of [unintelligible 00:51:05.07] never come back… [laughs]

Gerhard Lazu: Definitely. I would definitely agree with that. And we were somewhat locked into Concourse, because we had Concourse do the things that other CIs cannot do as easily. I mean, sure, it’s possible. Anything is possible, but at what cost? In our case, we almost had to take a baseline, and we had to use a CI for the things that we wanted - like Jerod mentioned, PRs. Concourse wasn’t good at PRs for us. It can do them, but they’re not as first-class as some of the other things. PRs in Circle CI are very easy; they’re so easy. That was a nice feature, which we embraced… But for example orchestrating our infrastructure, I would not have put that in any CI, to be honest.

Concourse is a bit more than a CI. As Jerod was alluding to, it’s this automation platform almost. It does so many things, and some - it does them really well, but it’s a very complex system. It’s not a CI, it’s a lot more than a CI. So a Circle CI - we just use it, run our tests, as I said… When there’s a pull request, we run our tests, resolve all the dependencies, package assets, and then produce a build artifact, and that’s it.