Paul Jolly: Yeah, sure. So the simplest way to describe it is to think of – the Tour of Go is very familiar to people as a way of starting to learn the Go programming language… But once you want to go beyond the language and actually wrap your hands around something and create something, you need to understand the Go tooling. And that’s where you need to get something set up locally, generally speaking. You need to install Go, have your computer set up with an editor etc.

So Play with Go is a very low-cost way, that doesn’t require you to have anything installed locally. As I said, the guides are browser-based. So when you open a Play with Go guide, on the left-hand side you’ve got the guide, which reads as a guide, or a tutorial as usual. On the right-hand side of the page is an actual interactive terminal that is hosted on Google Cloud. So this is effectively real compute resources that are live in the browser for you. So as you’re working your way through the guide, you’ve got command blocks in this left-hand side, which is the tutorial part, and you can actually click on those command blocks and they will execute in the right-hand side, the terminal, on that, in the Google Cloud compute resources.

For example, you could have a command block at the left-hand side that simply just ran go version and that would run the command on the right-hand side and show you the output from it. So that’s the interactive nature of it.

You can also create files as you’re going along. For example, let’s just imagine you’re creating a guide that explains how to create tests in Go. You would obviously create yourself a package, and then within that package you would have perhaps a regular package file and then a test file, as well… And you would create these two files by having code blocks in the left-hand side of the guide, and then with the final command block you would click on the command block and that would run go test, and you’d see all this happening live within your browser.

So this is a bit of an experiment really to see whether this kind of interactive type of documentation works… Because the Tour of Go and Go Playground are incredibly popular as a way of getting to know the language, but then at that point there’s kind of nothing; you’re left to your own devices. And this is a way that, as Carmen said in her closing keynote at GopherCon, of trying to lower the barriers to entry, if you like, for the next one million Gophers, i.e. not place any requirements on them to have anything installed locally, and just try things out in the browser, in a nice, fun, interactive way.

Jason Bosco: Yeah, so we came up with it mainly to mirror the cost of running the service with how much we charge users. So that’s one core principle that we held on to, because - sure, from a business perspective, that probably doesn’t make the best idea, because you’re very closely tied to your costs… But that’s what we chose in service of trying to make sure that we offer something that’s as affordable as possible.

So if you were to run, for example, Typesense on your own cloud accounts, we wanted the cost to be somewhat similar. And where we get savings is from economies of scale, essentially, like running thousands of clusters ourselves, so both the management effort involved, and the savings that you get with high spend. So that’s what we capitalize on. And then we pass on – you know, some of the savings we get, you know if you want to call it that, all savings back, instead of trying to do value-based pricing, which is what I’ve seen some other SaaS companies do. Now, that does make the pricing a little bit more complicated, because people have to know how to calculate RAM, how to calculate how much CPU they need… And that’s why we added a little calculator which says “Just plug in the number of records you have, or the size of every record”, and then we’ll calculate and roughly give you an estimate of how much RAM you might need. That works out well for most use cases.

[01:00:08.08] If people choose x as the size of their dataset, Typesense typically takes 2x to 3x RAM, and that’s given out as the recommendation in that calculator. And then for CPU, we just tell people, pick the lowest CPU available for that RAM capacity, and then as you start adding traffic, you’ll see how much CPU is being used and we can scale you up from there. Or we say run benchmarks – if you already have high traffic in production, run benchmarks with similar kinds of traffic, staging environments, see how much CPU use, and then pick good CPU. So that does make it a little bit more complicated to calculate CPU.

And then the other configuration parameters, like you can turn on high availability, meaning that we’ll spin up three nodes in three different data centers, and automatically replicate the data between those, and then load-balance the search traffic that’s coming, search and write traffic between all the three nodes. So flick of a button, you have a HA service.

And then we have this thing called Search Delivery Network which we built in Typesense Cloud, which we essentially replicate the dataset to different geography regions. So you could have one node running in Oregon, one node running in Virginia, one node running in Frankfurt, another one running in Sydney etc. And anytime a request originates, we will automatically route it to the node that’s closest to the user.

It’s similar to a CDN, except that in a CDN they only cache the most frequently used data, whereas here, we replicate the entire search index to each of those nodes sitting at different locations. So it’s as good as it’s going to get in terms of reducing latency for users. In fact, this search delivery network is what prompted some users to use Typesense as a distributed caching JSON store. So instead of having to replicate your primary database, which is probably sitting in one location, out to different regions, which is a hard thing to do, they instead send a copy of the data into Typesense, and have Typesense replicate the data to different regions, and then hit Typesense directly as a distributed cache. So that’s an interesting use case that people have used Typesense for.

So yeah, these are the different pricing angles. And I think when people realize that, “Oh, if I were to host this on AWS, or GCP, this is how much the incremental spend I have to spend with Typesense Cloud” - when that delta is tiny, when people realize that, that’s when hopefully that’s a convincing case for people to let us deal with the infrastructure stuff, rather than having to spend time on it yourself, and spend that engineering time and bandwidth. However tiny that might be, we still take care of that on an ongoing basis.

Nick O’Leary: Absolutely. And we are quite consciously moving away from talking overtly in the context of IoT. That’s where we’ve come from and that’s always a strong use case for it. I said at the start Node-RED is “low-code” programming for event-driven applications. We used to call it a visual tool for wiring the internet of things. Anyway, you’re right - it’s anything event-driven, whether it’s REST APIs, integrations…

I’ve got a whole ton of Node-RED flows running on the internet, handling a whole bunch of GitHub webhooks, for example. It just makes it really quick and easy. I’ve got a whole bunch of Alexa Skills at home that are backed by Node-RED running in the cloud, just because it makes it so quick and easy just to spin something up to handle the request, and again, not have to worry about all that boilerplate code you need to do it.

Techno Tim: Yeah, yeah, absolutely. Yeah. To get the fastest machine, to get the best frame rates, to get the lowest latency. Yeah, it pushed a lot of things forward. But I also feel like, man, that kind of like pigeonholed what a machine is possible of, and everyone optimized probably for cost, to get less lanes, smaller motherboards. It’s been happening over time for a long time. But yeah, it’s crazy to think - like, everything depends on the GPU now, too. No matter what you want to do. I mean, when you think of AI, when you think of Bitcoin, when you think of video rendering, creation… Everything depends on the video card. And everyone’s competing for them, at least with the 5080, I think. The 5080, what NVIDIA did was basically say “No, this one’s specifically for gaming.” They basically gave it gaming performance, like the previous generation for the cost, I think, less than the previous generation, but without the CUDA cores for the ML and AI.

[01:48:03.27] So I think – I mean, I don’t know if it’s the right thing to do, but it was a smart thing to do, for NVIDIA to say “Nah, this one’s for gaming. If you’re going to do AI, this is not the one to get.” I think that that kind of segmented their audience. I wish they would go wider. I wish they would go wider. Say “Hey, here’s the creator edition. Here’s the AI edition”, or Bitcoin edition, whatever. Call it whatever you want. “And here’s the gamer edition.” And then optimize for those things.

Honestly, it’s hard to discern between media creator and gamer, because really, you just need GPU, you need that encoder. You don’t need the 3D aspect, but you need that encoder, and you need video RAM. So you don’t need any – well, the funny thing is you still need some AI capabilities, even for creation today. If you think of Photoshop… When you say remove, or blur, or anything like that. They’ve been doing it for a long time. So that’s offloaded to your GPU if you have one, is to be able to do Gaussian blurs, or blurs, or even replace, or auto select this person, or cut this person out of the photo. So those are still needed, even outside of video, if you’re just doing static art… But yeah, I don’t know. I wish they would – I don’t know. I don’t know what they should do.

Gina Häußge: So it should be obvious that it’s pretty much impossible to test every possible printer, firmware, plugin, operation system, starting state of software situation. So what I do before I actually roll out the full release is there goes a long, long phase of release candidates. And OctoPrint has a release branch system built in, so if you feel fine with testing stuff that is not necessarily fully stable yet, then you can just switch over to another release branch, and then you will get release candidates whenever I push those out. And they actually get the same procedure that I do for every single release, and I will go quickly over it later as well.

But the idea behind that is that if I have something like 1,000-2,000 people out there testing a release candidate and putting it through several years of print duration over the course of the release candidate phase, and then I can be pretty sure that a lot of these combinations that I would never be able to test have been tested. And yeah, it usually takes something like three to four release candidates until no more bugs come in… And at that point, then I declare this stable.

And of course, after I’ve pushed out a stable release – so the current stable version is 1.10, but we are now already at one point 1.10.1. So there are bug fix releases that I also push out. Those do not go through a full release candidate phase, again, but they only get bug fixes and maybe small minor improvements of existing functionality. They do not get new features, they do not get big changes… They obviously also get security fixes, stuff like that, but I try to really limit what goes in there… And if it feels too risky, then it goes into the next stable release that will actually get the full release candidate phase again.

And what I do for every single release is – so OctoPrint can basically run anywhere where you can run Python. But most people run it on a Raspberry Pi, so that is also what I concentrate on for testing. And there is this dedicated image that someone else is maintaining, Guy Sheffer, for OctoPrint, which is called OctoPi. And a lot of people confuse the image with the software and the software with the image, which also causes a lot of complications in support… But anyhow, so OctoPi is the most common environment that OctoPrint will be installed on out there.

So what I have here is I built myself a little test rig that has three Raspberry Pi threes, which is the current basic option that I suggest… So get the three, because that basically is the best thing that you can get, the lowest supported version. And if you want something with more power, then of course you can get something else, but the three is like the base version that I look at.

[01:34:05.07] So I have three Raspberry Pi threes there, and all of these have a little card adapter in there that can be switched through USB, either to act as a mass storage device through a host, on the one end, or as an SD card on the other hand. So that is slotted into the SD card slot of each of the Raspberry Pi’s, and all of these then go into a USB hub to a fourth Raspberry Pi, a Raspberry Pi 4, actually, which I call the flash host. And that thing also has control over the little powered USB hub, through which I power the three Raspberry Pi’s. And now I can individually power them on and off, and I can also individually unmount and mount their SD cards, and flash them, without having to physically release the SD card, and push it into a flashing stick… That is what I did until 2020, and it was driving me nuts.

Gina Häußge: [00:15:51.11] So it should be obvious that it’s pretty much impossible to test every possible printer, firmware, plugin, operation system, starting state of software situation. So what I do before I actually roll out the full release is there goes a long, long phase of release candidates. And OctoPrint has a release branch system built in, so if you feel fine with testing stuff that is not necessarily fully stable yet, then you can just switch over to another release branch, and then you will get release candidates whenever I push those out. And they actually get the same procedure that I do for every single release, and I will go quickly over it later as well.

But the idea behind that is that if I have something like 1,000-2,000 people out there testing a release candidate and putting it through several years of print duration over the course of the release candidate phase, and then I can be pretty sure that a lot of these combinations that I would never be able to test have been tested. And yeah, it usually takes something like three to four release candidates until no more bugs come in… And at that point, then I declare this stable.

And of course, after I’ve pushed out a stable release – so the current stable version is 1.10, but we are now already at one point 1.10.1. So there are bug fix releases that I also push out. Those do not go through a full release candidate phase, again, but they only get bug fixes and maybe small minor improvements of existing functionality. They do not get new features, they do not get big changes… They obviously also get security fixes, stuff like that, but I try to really limit what goes in there… And if it feels too risky, then it goes into the next stable release that will actually get the full release candidate phase again.

And what I do for every single release is – so OctoPrint can basically run anywhere where you can run Python. But most people run it on a Raspberry Pi, so that is also what I concentrate on for testing. And there is this dedicated image that someone else is maintaining, Guy Sheffer, for OctoPrint, which is called OctoPi. And a lot of people confuse the image with the software and the software with the image, which also causes a lot of complications in support… But anyhow, so OctoPi is the most common environment that OctoPrint will be installed on out there.

So what I have here is I built myself a little test rig that has three Raspberry Pi threes, which is the current basic option that I suggest… So get the three, because that basically is the best thing that you can get, the lowest supported version. And if you want something with more power, then of course you can get something else, but the three is like the base version that I look at.

So I have three Raspberry Pi threes there, and all of these have a little card adapter in there that can be switched through USB, either to act as a mass storage device through a host, on the one end, or as an SD card on the other hand. So that is slotted into the SD card slot of each of the Raspberry Pi’s, and all of these then go into a USB hub to a fourth Raspberry Pi, a Raspberry Pi 4, actually, which I call the flash host. And that thing also has control over the little powered USB hub, through which I power the three Raspberry Pi’s. And now I can individually power them on and off, and I can also individually unmount and mount their SD cards, and flash them, without having to physically release the SD card, and push it into a flashing stick… That is what I did until 2020, and it was driving me nuts.

Jerod Santo: So you wrote up this awesome article on your observations, your findings. As you said, you’ve done a bunch of these; always, startups. It’s been seven, eight years since you’ve done a lot of these, and now it’s like a time to reflect and look back, and these are your lessons learned. You shared a bunch of lessons; we’re going to talk through as many of them as we can, we’ll see how it goes. And I would love to just dive right in.

[08:10] So the first thing that you talked about, your number one finding - and I believe these are in somewhat of an order; not like best or worst, but maybe highest level to lowest level, and maybe sensational to less sensational. We’ll just work vertically down. Listeners can check the show notes if they want to read along.

“You don’t need hundreds of engineers to build a great product”, which you’ve also wrote about this. But I’m sure you had a lot of startups that had a bunch of engineers, and you probably had some that had a handful. And it wasn’t like – you couldn’t draw that correlation. Is that what you’re saying?

Gleb Bahmutov: So I already mentioned component testing. That baby is out there, it’s growing, it’s becoming a nice person on its own… But there’s something else that I would say is huge. So usually, you have to be a developer to write a test. And a lot of people will ask “Well, can’t I just interact with the website and record the test this way? No code, or low-code tooling.” And Cypress has added what’s called a Cypress Studio, where you can just interact with the site, and it spits out the code right into your spec file, you can inspect it, modify, and so on. And they removed it in version 10, and it caused chaos. People were crying, people were cursing, people were throwing furniture out of the window, like “Why did you remove this no-code kind of recording tool?” And Cypress, the company, the developers said “It’s kind of hard to fit it in with the big architectural changes we’ve made, so maybe we’ll bring it back. We have to justify it.”

I personally was always against that tool, because it’s hard just to record a good test by interacting. Because you only can record actions on the page, rather than actions and assertions, which is a big part of a test. Anyway, that thing aside, Chrome the browser team has created a recorder. If you open in Chrome the dev tools, there is a new panel. It’s experimental, but it’s there by default, called Recorder, where you can click a button and record the interaction with your site, and then export it as a JSON or as a Puppeteer, which is another test runner tool from Google. Okay… Cypress has released –

Alex Russell: I think that’s an important question. I’ve been saying for a while now that I would like fewer disasters across my desk… Because it would be great to get back to working on new APIs that people need, rather than spending a lot of time convalescing with a carefully-constructed disaster.

So the way that I think we get out of this is not to switch our tools. Teams that do not operate their tools, but whose tools operate them, are unlikely to succeed with anything but the simplest tools… Which is to say, if you don’t have a lot of organizational capacity to manage complexity, pick the simplest thing that can possibly work. And that is to say, output HTML and CSS and leave JavaScript alone, because it’s not for you, no matter how good you as maybe that one developer are.

Okay, so at an organizational level, preferring tools that are simpler, because they’re simpler, because that reduces your total costs over time is just a nice thing to recognize as a pattern that can help. But that doesn’t help you if you’ve bought into a pile of complexity that you’re having a hard time managing. And I get that. So there’s a lot of good stuff that’s happening in the ecosystem right now, that I think will help. The first, of course, is that core vitals is creating visibility for management about your progress to doing a better job. I’m explicitly excited about interaction in XPaint, because it is finally starting to put a price on the worst effects of JavaScript.

So that will help both motivate, and then create credit for improving things like task scheduling, and main-thread work blocking, and all that kind of stuff. That’s great. That’s awesome. And then the tool - again, the tools themselves are depending on their endemic complexity, their basal complexity. They are more or less of a problem in and of themselves… But I have seen extreme disasters made out of the best tools, and I’ve seen teams that could sort of field-strip and rebuild the framework themselves blindfolded in two minutes - they can operate whatever, right? And I think, I think we talk about those two extremes as though they are the only thing, but in the broad middle, the thing that’s very helpful is to learn what things cost, and then start to just put a little bit of management control around it. Because there is good news in the networks and devices that we’re seeing.

Computers are about to start getting a little bit faster at the very bottom. That’s going to start to happen again. And if we can hold the line for a bit, then the sites and things that we’ve deployed today - they will certainly feel a little bit more affordable in the future, if they’re close to the line. If they’re not, you’re gonna have a hard draft, you’re gonna have to dig out…
Other things that are helpful right now - if you have one of these big complex UIs, and it’s not scaling, for instance in terms of style complexity, Nolan Lawson has been doing extraordinary work over the last year in sort of outlining how DOMs that are out of control, coupled with style sheets that are [unintelligible 00:57:32.05] and often being perturbed by very poorly-done CSS-in-JS tools - that combination is quite toxic for interactivity. But we can get it under control, right? There’s generations of CSS-in-JS tools that will extract all your style sheets out, turn it into a single CSS file that’s not being manipulated; that’s awesome.

[57:51] I’m working with and have seen teams kind of hollow out their existing React components and replace the guts with web components, so that they get Shadow DOM in isolation, and they don’t have this combinatorial explosion of complexity there. And that helps a lot.

And then teams learning to put the marginal user at the center of the conversation is just the most transformative thing. If you are sitting there talking about how it feels on the P90, the P75 if your PMs carry these low-cost devices that are relevant to your business - and I don’t want to say either is the one that you should use, although I have a post on like the performance baseline worldwide… But if you think about what you would want to know about your marginal user, and then decide intentionally to go serve that user, there are so many good tools, right? It isn’t just going to be, as you say, one size fits all. But you can learn what size fits. We’ve gotten an entire department store full of great tools. HTML and CSS, very thin frameworks, compositional things, “server-side rendering” in the style that we do it now, old school server-side rendering in static sites… There’s a lot of cool stuff happening with Astro and Eleventy, and Hugo, and PHP, and DotNet. There’s this whole closet full of stuff that we’re not talking about now, that can be extremely appropriate, and you can use it, and you can deploy it, and it probably isn’t worse. You may not be comfortable with it right now, but if the output that it creates is simpler, your chances of success with it are significantly higher.

And so dust off the Rails, dust off the Django. There is a place for this tech, and if you learn about the kind of users you’ve gotten, the sessions that they’re working through in your experience, you can make appropriate choices, and you can hire for those skills. I promise you that you can. There’s a fire sale on talent right now. In fact, one of the most depressing things about the last decade has been how folks with JavaScript skills have pushed out people who have those skills and would like to come back to using them. They’re just sitting out there, and I promise you that if your product or your team is saying “Oh, but we can’t find people like that”, t’s mostly because you aren’t trying.

Paloma Oliveira: So if you’re a single person, lost in the world, and you think you’re powerless, you’re wrong. First thing. Understand that you have so much more power on your hands than you think… And that’s the classic “Understand your own privileges”, and put the voice of the ones who do not have voice on the table that you have access. Remember that change comes when you have everyone at the table.

So invite the ones who are not in the seat to be in the seat. And that comes in many forms. We mean speaking up for people who are not there, for mentoring people - that’s so much important, and it’s so much sad… Shout-out to Chris here for being the loveliest person to work with, and for helping me a lot. So I don’t know what it means for people, but sometimes just answering a question is enough for people to feel they belong, and they know something they didn’t know before.

[43:48] I mean, I’m a fairly junior developer, and I’m a mentor. I know the difference I make to people’s life. So you don’t need much to make the difference. If you have some sort of power, or if you’re part of a project, an open source project, for example, for Christ’s sake, follow best practices. [laughs] You can literally, literally just download models that you just follow. And what does this best practice do is that it improves the communication on your project and makes it welcoming for people. It’s not much. You just write the readme. Put the license! [laughs] These are three documents that change everything, but that will help so much people to onboard your project. It’s not much, right? So just make the decision and just make one contribution to adding those best practices there.

If you have some sort of – if you’re a manager, or if you’re a company and you have some sort of mechanisms… At Sauce Labs we have now an experimental program, a fellowship program to help people onboard and become contributors and leaders in the open source space; that’s supposed to be a replicable program. Very low-cost. Definitely very powerful for changing people’s life.

So I hope that we’ll become a practical model to push forward; I can tell you in six months how it went. We will start on December 1st. We’re currently interviewing the fellows… But that’s low-cost, it doesn’t interfere anything on backlogs, nor nothing, but you automatically make people under you hierarchically speaking become mentors, sharing their knowledge, and pushing more contributors and presence in open source.

If you’re a manager in companies, support your engineers to be part of open source. Support your designers to help with open source design. We need a lot. Help people to promote events where you help them to contribute, and ask them why they’re contributing. And for that, just easily look to the stack of tools you’re using. They’ve gotta be open source. So just contribute back to the ones you’re using. And that’s a lot.

If you’re a government - well, you’re so behind if you’re not doing it, but remember that you, the government, work for the public; therefore, that should be easily accessible for the public. So if you’re not using open source and your data is not transparent, shame on you. [laughs]

That’s a list of things that crossed my mind, but – I don’t know, that’s my head. I would love to hear your thoughts about it.

Feross Aboukhadijeh: Yeah, but I think it meaningfully makes security better. I think we need a mindset shift in the industry for how people think about their dependencies. Dependencies are not this magical thing that you can just show up and use as many as you want and there’s no downsides. It’s not this infinite buffet of open source that you can just take and there’s no costs to it. Eventually, you will pay a cost for it. It’s not a matter of if, it’s a matter of when. And I think that automatically looking for changes in a package that are nefarious is a very low-cost, no-brainer thing to do to help with the problem.

[52:20] So yeah, I don’t think that there’s really anything better than this that I could be working on to improve the security of the ecosystem. I mean, what else – yeah, I just wanna make a difference and make this stuff more secure, and I’m trying to think what… I wish that we could all just agree on the best packages to use, and we could vet those, and we could bless them and call them a standard library that we just use, but with 1.8 million packages on npm, and millions more on all these other languages, it’s just too much. We’re not gonna ever be able to read it all, so we need tooling to help us.

Tracy Hinds: We’re actually meeting this weekend to hash out the domains and the subtopics that the exam will cover. The aim is to be a low-cost certification exam for someone who’s been writing Node professionally for a year, full-time. We’re gonna be bikeshedding that and figuring out what that means. The test will be in browser and third-party proctored, and we’re aiming for it to be in English as well as Chinese.

Eric Boyd: I don’t know… We’ve definitely sort of thought about that, and that’s a possibility. The thing that we see is the capabilities that the frontier models have are definitely not commoditized. There’s just things that you can do, and their logic, and reasoning, and their ability to sort of follow multiple instructions… And as you start chaining multiples of these models together and agent patterns there’s simply things that you can’t do in other ways. At the lowest end, I think there’s always going to be that question of “Alright, but what’s the best quality at this price or performance that I can sort of have?”

So I don’t know that it’ll ever be just sort of like “Oh, they’re all the same.” I kind of don’t think there will be. I think there’s still a lot more capability coming. But there certainly are people who think that. And the people who think that I often find have some invested reason to think that. They’re trying to sort of say “Oh, they’re all commoditized. It doesn’t matter”, because they don’t have the best ones.

Zach Leatherman: [00:28:10.05] Yeah, I mean, I think that at my core, I’m a frontend web developer, so even though I work on Node.js almost exclusively now… But all the websites I build use Web Components, because I really think that is the sort of lowest overhead way to build interactivity on a website, with the least amount of dev tool overhead, and tool chain overhead.

So for me, it’s really the sweet spot of simplicity and longevity. And that’s what I love about Web Components. And I think when it comes to publishing Web Components, anytime I build something with a Web Component, why not publish it, so I can reuse it on all of my 50 websites or what I have… Probably not that many in production now, but… And all of my projects moving forward. And I really try to make my Web Components as design-free as possible. I think a lot of my demos are just like Times New Roman, the default typeface, and just as little CSS in them as possible, because I really want them to be as reusable across as many projects and design systems as possible. And I think the best way to do that is just to have these sort of just structural components that don’t take a strong stance on how it should look, just how it should behave.

José Valim: …and then you ship it to production. So the idea is exactly, you know, how can we have the best of both worlds? I still have the UI for a learning perspective, or just to be productive, but that does not limit me in any way. I even have like a whole separate rant of like how low-code should not be – at least for Livebook it should not be called low-code. Because if somebody gives you a task, it doesn’t matter if you achieved it visually, right? It doesn’t matter if you use a UI to write the code for you, or if you write it by hand. As long as you deliver the task you’re supposed to do, you’re good, right? So I’m like “It’s just code.” It’s not low-code, it’s just code. It doesn’t matter how you write it, it’s code.

Adam Argyle: Yeah, that’s a good question. One of the best examples is what I was talking about earlier - I think it’s UHD, or… No, it’s OLED. OLED is more expensive, because - you know how we have black mirror; when your TV is off, it’s literally black, and it’s like reflectively black. But if you turn it on, and it’s broken, it’s dark gray. That’s not OLED. That’s your RGB lights at the lowest setting that they have. But that’s not a rich black color, which means if you’re watching Game of Thrones, which - I watch Game of Thrones on my crappy TVs all the time, and I get tons of bad color. Because they’re so dark in a scene, and my screen can’t do it, that I see these like bands of gray. When I know the reality is if I was on an OLED display, with a really, really rich amount of dark range in it, I would see richness there, and there would be millions maybe more colors just in the dark area, let alone the light area, where the whites are better because of the nits. Nits come down to the power of the lights. How powerful can these things combine to create a white? There’s literally whiter whites. And I know we talk about like painting your home, you’re like “Oh, there’s 10 whites to choose from when painting a wall.” You’re like “There’s 10 whites?! Come on!” And then you see them all together, and you’re like “Oh crap, I see the difference. That’s annoying.” Same thing happens with the display, and nits; you get more whites, the whites are whiter, and it makes a difference having that darker dark, and the lighter lights, as well as a bigger gamut of colors. Now you’re watching Discovery planet and really getting immersed, right? You’re like “Well, the waves are crashing”, and it’s good stuff. So there’s like some background on like the displays and the capabilities that are changing, that are enabling these things to be in our homes and our pockets.

Sarah Moeller: …big grant is, what I’ve been talking with people here in the University of Colorado, who also worked on this, and were my mentors… About this opportunity to make it a human-in-the-loop for linguists. So there’s two parts here I see. One is the fact that you need to get the NLP working. You need machine learning models that are relatively good at learning from field data from the linguist. So that’s something that I worked on a lot, because there’s a whole issue there with working with field data, of just - it’s noisy. And how a linguist creates data is different than how people create gold standard data for a machine learning model. It’s noisy. I’ll just leave it there. So it’s a challenge, and finding the right models that work for multiple languages and how much data should we recommend to the linguist that they bring. So then on the other side is then how do we bring this together in a way that linguists can access it who don’t have any computational background? So that’s going to require an interface.

So we’re talking about creating a third– so there’s these software that exists, that I’ve talked about, that don’t have machine learning, and they’re popular, they’re great tools. So we’re talking about creating the third, kind of, not to replace them, but to be there where it does machine learning; so they could work a little bit, export, and then have this interface where in the background you’re doing NLP, and people are doing experiments and trying different ways, but the linguist has to see the computational stuff going on, like “What’s the best way to do this?” So you run, they’ve worked by hand in their tool, they’ve annotated the data by hand, bring it into this interface, train a machine learning model, say, to do morphological analysis, break the words into their meaningful parts, and let’s say… So I’ve seen languages, depending on how you approach it, if you have– and there’s all sorts of different languages with different kinds of words, but I’ve seen a language with 3,000 words and we got like 30% accuracy at breaking a word into its meaningful parts and giving the meaning of those parts based on what the human had done. 30% - not really great. And there was a lot more data that the human hasn’t done.

So the machine learning model is learning, based on the tests, about 30% accuracy. And then we have it predicted over, let’s say, another 10,000 words that haven’t been annotated. And we know that 70% of them are probably wrong. What we don’t know, because no human has looked at it, is what 70% of it is wrong. So then you want to figure out what’s most likely incorrect that the model predicted, and not only that, but if of those incorrect ones, which of those incorrect words, incorrect analyzed words, if they were corrected, would be most helpful for the model to learn the patterns and become better, like reach a higher accuracy more quickly. So instead of having to do 1,000 words, can we find 100 words that if the human corrected those, the model would suddenly jump from 30%, let’s say, to 40%, or maybe even 50%, if we did a really good job. So 3,000 words to get to 30%, but only 100 to, say, get to 40%.

So that challenge right there is some things I’m looking at right now. You can choose randomly, you can choose– a colleague of mine figured out that if you look at the probability confidence score on the predictions of the machine learning model, that’s better than choosing randomly.

[40:10] So one thing I’m trying to do is incorporate some linguistic knowledge into that. So, okay, low confidence, but if you have a bunch of narratives, for example, you probably, at least in English, you’d end up with a lot of past tense verbs. So you’d see a lot of that, and you’d learn pretty quickly probably that -ed needs to be separated out and labeled past. What you might not see as much of is -ing verbs, just to choose English, which hard to come up with examples because English is really simple. There’s not that many complicated things going on in the word.

But let’s say you don’t have very many -ing examples in those stories. So one thing we’re trying to figure out - okay, we’re hypothesizing if you don’t have a lot of -ing examples, then probably it’s not going to do as good at analyzing words with -ing and recognizing that sometimes -ing is part of a verb. Sometimes it’s a word like ‘sing’, and you don’t want to separate it out. So if that turns out to be true, which is a hypothesis, then we could combine these things. We could choose out of those 70%, out of those 10,000 words, the lowest confidence, 100 low confidence scored words, but also make sure that they’re distributed, so that you have more -ing examples, for example, so that the model is learning from these incorrect examples, but it’s learning in a way that’s going to strengthen its weaknesses more effectively.

So then you choose it smartly, the human does just, say, 100 words… It would be wonderful to see it jump up to like 40%. Run it again, now you’ve got another– you got 60%. Okay, pick out another 100, jump it up another 10% accuracy… I don’t know. We still have to find out how that’s going to work and how quickly that will go, but that would be like the general workflow, with some abstract things thrown in the background as well.

Woody Zuill: …by doing the shot-put, the high jump, the pole vault, as long as I could get over the lowest height, run the two-mile, and maybe do a relay. He could bring in five points, and his very best miler would never bring in more than five points.

Chris Benson: Yeah. I’m wondering if – one of the challenges is we’re addressing deployment, and recognizing that people focused for a while on model creation way more than getting the model out there in a usable fashion, and that we’re getting mature about that now, your organization, and there are others out there too that are thinking very deeply about this. Do you think that there’s an opportunity for maybe low-code/no-code approaches once we arrive at more kind of standardization? Right now it’s still very early, it feels like, in terms of different approaches. So there’s a real custom feel to how you’re going to deploy, especially when you combine it with all sorts of edge targets, with edge being a catch-all phrase that could be almost anything. And so there’s a ton of specificity to your target at this point. Do you think over time, as those get categorized and kind of best practices emerge, that there might be more opportunities? Or do you think that’s still going to be a challenge, given some of the API concerns and such that you mentioned in your previous answer?

Zac Smith: I mean, there are some positives and negatives to it, right? Back in Packet days, we were – I’m gonna call it an “arms dealer” for transformation, as it pertained to this level of the stack… Which was just like – I always thought, like - hey, there’s real estate e.g. data center as a service, which is a pretty scaled model. It’s for the efficient capital, there’s many providers, of which Equinix I would say is the leading provider, definitely by market share related… But there’s like scaled business for like – if you wanna access one of the world’s best data centers, you don’t have to build one of the world’s best data centers.

And then there was this thing called IaaS, which was everything from the computer, and the network, through databases and load balancing. It was a whole thing. It had a ton of verticalized software opinion built into that. So what Packet was doing was we were trying to democratize that lowest layer. I called it hardware as a service. How do we enable the consumption of hardware and make it able to touch software? And as an independent, we had some advantages, which were hard to realize at our tiny scale… But in theory, we had some advantages, which we could help do this anywhere. That was an interesting concept; we had put out edge deployments, with tower companies, we had done private deployments with some large enterprises… We were really trying to figure out “Hey, where would this be needed?”

Now, obviously, as Equinix, we were a little bit more focused on our own large-scale real estate portfolio… But with that come some incredible advantages. No longer do we have to think “Wow, how could we change this?” I’m like, “Well, we have 240 buildings in 65 markets around the world. We could start there.” And we can also help our existing 10,000 customers, who were really struggling, frankly, with the breadth of our platform, and also just figuring out how they are going to react and make a difference related to their ESG initiatives around climate change. And what they’re doing is they’re saying “Can you help?” And what I’ve found is that’s been a really, really collaborative engagement. And what I’ve been so pleased about working with Equinix versus just (I’m gonna call it) another large company is that Equinix has the word “ecosystem” everywhere. And in fact, Equinix stands for Equality in the Internet Exchange. That was the name of the business.

Guillermo Rauch: Yeah, I think what’s happening to, and the reason that frontend matters so much is that the complexity is significant, but the tools are getting so good. Not just code tools, but low-code and no-code tools, which we support on the Vercel platform because we have a number of platforms that expose a GUI to these technologies underneath. So as I mentioned, it’s turtles all the way down. We’re seeing the rise of all these tools that compile down the Next.js and Vercel pages behind the scenes, but the end user is facing a GUI type interface, or the writing a notion document, and then all of a sudden they have a website that has been optimized for the best possible performance that hiring hundreds or thousands of developers would have gotten you.

[12:04] So really what we’re seeing is - and this is why fronted matters so much - this is where the value is. This is the cover letter, this is the presentation to your business… Whether it’s found through a Google search, whether it’s found through an Instagram ad, because you just launched your e-commerce business, whether it’s invisible because of the power of APIs… We see so much traffic that is robot-generated today on the internet… But this is literally the entryway into everything. And I continue to think that the web will continue to become the entryway through everything… And that’s the right bet to make, I think, and it’s been super-rewarding so far.

Steve Jobs: I’ll tell you what our goal is. Our goal is to make the best personal computers in the world, and to make products we are proud to sell and would recommend to our family and friends. We want to do that at the lowest prices we can. But I have to tell you, there’s some stuff in our industry that we wouldn’t be proud to ship, that we wouldn’t be proud to recommend to our family and friends… And we can’t do it. We just can’t ship junk.

Jerod Santo: Sure. So back when I first got started I actually didn’t consider myself a programmer or a developer; I was more of a scripter. I was studying information security in school, and I liked to just script things, but I didn’t actually think of myself as a developer. When I first got out of college, I was a network administrator, so I had managed some Linux machines, mail servers, stuff like that… And there was a lot of manual tasks on a network to keep it running. This is back in the early aughts… And I had a lot of jobs I had to do every single day. And then slowly I just automated every single one of those jobs to the point where I didn’t really have a job anymore, but everything was still running smoothly. So I used that opportunity to learn web development on the job, while my job was doing the job for me… Which was pretty cool.

So I actually learned how to do web development while being a Linux network administrator, because I had automated all the things. So that was pretty powerful. But my more fun one - I’ll do this one real fast - is that I like to smoke cigars, and there is a website… It’s a cigar auction website, and they have a particular auction style called freefall, where they have these falling prices that reset on occasion. So the prices start high, and then they fall down to some sort of threshold, then they go back up. And the reason they do this is because they want you to sit there and watch it and buy it when you think it’s low, but you’re not sure exactly “Is this lowest, or could it get lower? I don’t know, I’m gonna buy it anyways.” But as a developer, I thought “You know, there’s some sort of minimum amount that it’s going to get to eventually. And it may not hit it every time through, but it’s gonna hit it at some point.”

[56:07] And one of my best friends used to have a job where he would make phone calls all day, so he would sit there and stare at the screen and watch them free-fall, and buy cigars for us at the lowest possible price. And then he doesn’t work there anymore, and I didn’t wanna stare at the screen all day, so I wrote a script that would basically imitate my eyes, watch for the lowest price, and let it run for hours upon hours on end, until it got to the lowest price, and it would log it… Then it’d come back and it’d say “Okay, here’s the lowest possible price.” Then I’d put it in buy mode and it would watch it again until that lowest price came in, and it would buy the cigars for me. I actually still use that. I just used it yesterday. It still works… And I think that’s pretty cool.

Meg Risdal: Yeah, they’re like Easter Eggs, kind of… Like what’s a good example of a golden feature…? Speaking of the Airbnb dataset, one of the golden features was to take some of the meta data about the Airbnb listings and extract the number of bedrooms and bathrooms. So it was to do a little bit of like text processing.

Other times it’s to create some kinds of data visualizations. One dataset we had them create a [unintelligible 00:37:58.14] dataset. So if they happen through the course of their two hours of live coding to create one of these plots, or engineer their features in a certain way or whatever, they’ll earn extra points.

So a lot of folks are really just kind of going for this scattershot approach of “I’m just gonna find some golden features and hope that Nick and Meg [unintelligible 00:38:22.05] or want to see me do this or that with the dataset.” So it’s a little bit like – you can tell that they’re getting creative, and sometimes you see them bend some rules of data visualization where it’s like “That’s not maybe the most intuitive way to display that data, but you’re just kind of trying to fit a square dataset into a round golden feature hole, or something like that, just to earn some points.”

So I would say that kind of summarizes the strategies that I think I’ve seen from our contestants as far as trying to earn points from data visualization… But I like it best when we see folks really looking for insights, because I think that’s the purpose of data visualization, the same way that the modeling portion, the purpose is to get the lowest log loss, or whatever the metric is.

Bharat Sandu: First of all, there’s a ton of innovation in AI in the open source world, which is awesomeness. Our goal is to make sure we can provide that to our customers in a more packaged fashion. We also have a ton of research centers around the world; we have eight research centers, and the work they do is really fine-tuning and developing the algorithms. Their charter has really been to say “Okay, let’s go and build the best text-to-speech, or speech-to-text, or image captioning (that we’ll cover later on) techniques.”

[20:02] Now, these are researchers and PhD’s in machine learning; there are more of those now than we used to have maybe many years back… But they’re really specialized really deep in the machine learning space. We take all the work they do, and then we bring it to Azure AI.

Our goal on Azure AI is to use all the open source technologies available, all the work we’re doing in that space too, and taking all the research milestones, and baking them on a platform that we can give to developers and data scientists, so then they can use it from a simple, no-cost, free API call, to kind of doing high-scale reinforcement learning techniques.

We also, by the way, test this stuff in our own products, and we have the – I think we’re just lucky to have many different types of endpoints, from teams, which if you were to join a team session now, you can do transcription of all the calls; they can do recording and transcription, and translation. So all this is powered by Azure AI.

Within PowerPoint, if you’ve used it recently, there’s something called PowerPoint Designer, that looks like PowerPoint bullets, and say “Hey, maybe you wanna actually have a better representation of the slide.” And it’s really using Azure machine learning under the covers to do all the inferencing and give you those recommendations.

And even Xbox - you know, the holiday season is here… But if you have an Xbox, if you log in, you’re given a personalized screen. So if you log in, Daniel and Chris, you’ll get a different dashboard. And that’s powered by Azure AI. The service is called Personalizer, and that uses reinforcement learning behind the scenes to do very personalized recommendations for folks. The same technology is available to developers.

So our approach to bringing AI to developers and data scientists is we take things in the open source world, we of course do our own research, and then we package it, and then the real goal is to make it usable for customers, whether they’re very professional data scientists, so give them access to the best collaborative notebook environments we can, with highly scalable compute, so they can do their highest-end training work they need to do, all the way to simple APIs that any developer can call using any of the languages of their choice, and add AI to their applications in a very simple fashion.

And then what we have also done recently is we’ve started creating what we call scenario-specific app services. We can get into that… A good example would be a metrics advisor, where we have anomaly detection models running behind the scenes, but that also have a bunch of other logic around them, doing just data from multiple time series sources of data to do root cause analysis, and to have composable UX elements that a developer can embed in the application.

So if they wanna do more with metrics monitoring, we don’t just give them a model, but we give them a higher-level [unintelligible 00:22:42.22] service that does a lot of other work and other business logic for that thing.

So that really becomes a platform… At the base we have Azure machine learning, which is used to train machine learning models; on top of that we have customizable AI models - we call them Cognitive Services. Those are speech, vision, language, and so forth. And then on top of that we have a set of services that are really tuned for separate kinds of scenarios, like form recognition, to metrics monitoring, and so forth. And that’s really a stack for Azure AI.

We do go one step beyond that and we have a platform called Power Platform. I’m not sure if you’re aware of that… It is basically our platform for business users; it’s a no-code/low-code environment, and it really allows citizen developers really to go ahead and design applications. So all the Azure AI tech that I described is provided in a low-code/no-code way in this environment for anybody with a citizen developer to build AI into their applications without having to write code.

Fred K. Schott: Yeah, I think we’re closer to – so this whole episode I feel like has been a “How do you deal with legacy”, how do you transition very large changes in a comfortable way? Again, I feel like for the first time the web is actually in a pretty lucky position here, where IE11 is getting end-of-life pretty soon. It’s kind of [unintelligible 01:02:24.26] what that distinction is, but their general mainstream support I think ends in October of this year, which a lot of people don’t realize.

So sure, you might have some larger enterprises getting an extension with Microsoft, and that still matters if you’re a Salesforce, where most of your customers are these larger enterprises… But if you’re building a general application for a general user, all of a sudden that user is not on IE11 anymore, or it’s very rare that you really have that requirement anymore. So I think by waiting for October and waiting a little bit longer, a lot of these problems on the web solve themselves, a lot of the legacy just kind of melts away.

And then that idea where it’s always harder to support both at the same time - you can kind of just start to build for that world. And the world we’re entering on the web is really exciting. Every browser is essentially following this evergreen model now, where updates are sent automatically, even enterprises are encouraged to update regularly… It’s this really nice model where now all of a sudden you can start to build for a much more modern platform. It doesn’t mean right when a new feature is launched in Chrome or in Firefox that you can start using it everywhere; there’s still, obviously, a little bit of onboarding of those new features… But your lowest common denominator is suddenly only a few versions back.

So there’s a really exciting model that’s happening there as well. That change is actually gonna be pretty influential for not just how you can build for the web, but what you can assume your users are able to use, and then not having to always find a really low common denominator to support them. Specific to Snowpack, it’s one of those interesting things where we now get this option to build an serve an [unintelligible 01:03:56.06] application. Again, this idea of default complexity versus default simplicity.

[01:04:03.13] Right now if I’m building a web application, I need a bundler. I’m so used to saying “right now”, but really, I mean in the old world - you needed a bundler to work with npm, and that means you need all the complexity that comes with “How do I configure this bundler? What plugins do I need?” The model that Snowpack is following is that by default everything should just work. Everything should just work in a way where you don’t need to do a lot of configuration to get an application that runs.

Instead of bundling being an assumption and a requirement, it’s now just an optional thing that you can do for performance, and that’s - again, we’re taking the old and the new and trying to grab the best of both of them. Bundling is an optimization, at the end of the day. It’s only in the last ten years that it became this requirement that we all had to build on top of… But really, what it is is we’re saying “I wanna combine multiple files together to speed up my site.” That’s a much different model from “I need to do this for anything to run for anyone.”

So the whole model is kind of flipping, and complexity is something you now go out and find because it solves a problem, versus it’s baked into the very foundation that you need to get started.

Feross Aboukhadijeh: So I’m gonna start at the absolute lowest level. So we’re gonna go all the way down the stack to IP, or the internet protocol. I’m sure most people have heard of an IP address; that’s the level we’re gonna be operating on here for the beginning of my explanation. So what is IP? IP is basically a way where you can take a message, a packet of data, and it’s kind of like an envelope - you can sort of stamp a destination on the outside of the envelope and say to your router “Hey, please try to deliver this to this IP address.” And every computer on the network, every device on the network is assigned an IP address, and the internet as a whole sort of just tries its best to route these little packets of information to the correct destination based on the IP address.

Okay, so that’s the lowest layer. Then on top of that there’s two main protocols that people have sort of added on top, which are called TCP and UDP. TCP is the more common of the two, and it gives you a bunch of nice reliability enhancements on top of IP. So you can sort of, as an application, say “I wanna send this giant chunk of data, in fact too big to actually fit in a single envelope”, and your operating system will split that data up into little individual packets of information and send that across the internet… But it will do a bunch of nice things like stamp little numbers sequences onto each of those, so that the recipient can assemble them in the right order.

Jerod Santo: Yes, I’m with you on that. I understand that there’s a lot of misaligned incentives. I didn’t think about it with regard to the lowest common denominator of security because you want to service as many people as possible, so that’s very interesting. I’m sure there are some of these services which are doing the best they can, and aren’t some random person in Europe, or wherever they happen to live - and there’s nothing wrong with Europe - but I definitely see where that is problematic… And I think where it becomes an issue, especially for folks who aren’t well-versed in these things, it’s like “How can you tell which ones are worth their salt and which ones aren’t?”, especially when there’s so many of them, and like you said, it’s so easy start up.

That being said, the entrepreneur in me, and the one that likes to see these things as like – it’s a perfect market opportunity. I do like when people take something that is exclusive to technical folks and can open it up to a broader user base, and say like “That’s a business opportunity.” It’s like “Here’s this thing which is very useful”, and whether you’re using it to get access to Disney+ or you’re using it to actually secure yourself while you’re at a coffee shop and you don’t want people reading your email, it’s very useful service… But it’s traditionally difficult to acquire for people who aren’t technical. And now we can all get access to it. I like that idea, but I understand how it ends up not so idealistic.

Yaw Anokwa: Yeah, that’s a good question. We live currently at forum.opendatakit.org. I always have to spell it out, because “form” as in a survey form, and “forum”… Yeah, it gets super-confusing. Maybe we should just make that an alias. But that’s where the community lives, and so anyone who is interested in participating on the project can go to that forum. It’s a Discourse forum. Create an account, and just introduce yourself. Say roughly what your interest is in. That’s one way of doing it.

And then whenever I say this, my commitment to anyone who shows up and introduces themselves and says “I’m interested in helping” is that I will take some time and we’ll talk about what you’re interested in, what your skillset is, how you wanna grow, what you wanna do, and we’ll find a place in the project for you. That’s a very hands-on approach, but it’s an approach that we wanna take, because we wanna make sure that it’s not just – that’s a two-way street; you’re helping us and we’re helping you in some way. So go to the forum and introduce yourself.

If you’re a dev and you just wanna hack on code, certainly you can go to our GitHub page, github.com/opendatakit, and there’s a bunch of repos there. Some of the starter ones are the active ones… And then each repo has good first issues, or quick wins, so you can pick any issue. We have a contribution guide, and you follow that contribution guide, submit your first PR. It’ll be reviewed, tested and merged. So that’s another way of going about it. But the lowest bar I think is just go to the forum, say hi and join us.

[01:11:14.24] I should say that everyone is welcome, regardless of your skillset. If you’re a designer, you’re welcome. If you’re a developer, you’re welcome. If you love grant writing, you’re welcome. If you wanna help us on social media, you’re welcome. Literally anything that you’re interested in, we will do our best to find a spot for you.

The forum now has (I think) almost 9,000-10,000 people on it, all helping each other. I participate in a lot of open source; I’m a little biased, but I think this is the nicest and friendliest community out there, so I would love to have anybody who wants to chip in.

Bert Hubert: Yeah, bespoke. Just the one thing. So I also do presentations, and you can hire me as a speaker, and my speaker agency wrote a whole low-code solution system just to do the logistics of that. And it’s sort of wonderful in a way, because I’m really impressed that they managed to build that mostly themselves… But on the other hand, every time I do something with the system, it takes like three seconds to go to the next page. And I’m wondering, “What is it doing in the background?” But it’s sort of – it is impressive, because it is built on this concept where you click this stuff together without having to be a real programmer. But the thing is, I would love to be a real programmer, and have the primitives that make me do this safely, and at a higher level.