Drew DeVault: Yeah, so Hare is a systems programming language, and it compiles down to machine code, and you can use it for a whole lot of low-level use cases for that reason. I’m writing this kernel with it, and we’re doing a bunch of other stuff along those lines of that.

[00:24:02.29] It has a syntax which comes from the C lineage, it has braces, and it has those fixed expressions, these kinds of things you would expect from a C derivative syntax like JavaScript, Java; these are similar syntaxes. And it gives you the tools you need to do a lot of the same stuff C does. So it can feel like you have the power of C, but it also has 50 years of hindsight that C didn’t, and so it has a lot of features which kind of address paper cuts in the space that C is occupying right now. We have better error handling through tagged unions; that feels very comfortable to use, so you can write more robust code more easily. We have things like slices, which is a sorely missing feature from C. We have better string support, and we have a handful of safety features - nowhere near what Rust does, for example, but things that were sorely missed for C programmers… And we let you take the training wheels off as well. So if you need to do something in a kernel context, writing a driver with unsafe memory patterns, it’s easy to do that in Hare.

And then we also have a standard library for Hare, which is, in my opinion, significantly better than C. I think one of the [unintelligible 00:25:13.01] parts of C is the standard library, and with 50 years of hindsight and a bunch of other languages for inspiration, we were able to come up with something a lot better.

Adam Argyle: Yeah, a lot of people didn’t get the memo. They gave that memo because people were really realizing that darken and lighten would sometimes give them weird results. And so color adjust was used to do the same work, with a very similar syntax, but in a better color space, so that the results weren’t so weird.

So now we have CSS, which can – it can lighten and darken colors. And so there’s two different ways to do it. One has great support, and one is on its way. The great support method is the color mix function. You call the color mix function, you pass two colors, you can say which color space to mix them in, and you’ll get an output result of them. You can mix a color with transparency, so to make a color more or less transparent, you can mix a color with white, to lighten it, if you want to lighten it in a sort of like mixing fashion… And you can darken things by mixing it with black, or another darker version of the hue. And that’s color mix. It’s kind of – I don’t know, a little basic in it’s way, because it only…

But there’s also one called relative color syntax, which allows you to – so let’s say you have a brand color that’s like #f10 or whatever, right? This is - again, a designer gives you a HEX color, and you need to do manipulations on it. You can now say “color” – we’re gonna use the color function, like we were talking about earlier with Display P3… color, open parenthesis from your HEX color. And when you say “color from” – well, you can even say “OKLCH from”, you’re gonna get back a destructured version of that color in the color space that you asked for it. So you could say “HSL from HEX”, and you’d get H, S and L back. You could say “OKLCH from HEX”, and you would get L, C, H back. And when you get those values, you can use the calc function to increase or decrease them, or divide them, or multiply them. So you could double the amount of saturation, you could divide the saturation, you could divide the lightness. You can also just squash it. So you could be like “Hey, I got LCH back. I don’t care about L. I want a really dark version of this color.” And so you just set L to 5%, or something like that; now you have a really dark version of that HEX color.

And so you’re getting these new functions that allow you to very, very dynamically build out variants, and derivative colors, and like derive colors from other ones, and build entire systems and entire palettes that are very robust, and consistent. Again, if you’re working inside of that LCH, or OKLCH space, you’re gonna get that consistent lightness. So if a user chooses they want the theme to be green, they get a nice, visually-consistent green theme. If they change it to red, they’re gonna get something that’s perceptually the same lightness across the board, just a different hue. And it turns out HSL does not have that power. It has a lot of oddities in terms of lightness.

So yeah, relative color syntax - often called RCS - and the color mix function are here to save the day for mixing colors, making variants, and just doing overall color manipulation inside of CSS.

Derek Collison: 12-factor apps, and then of course, the cloud-native architectures, and of course then the CNCF… All of those things - that step function from data centers to that I believe will be even bigger stepping from cloud into edge, where it’s not batteries included. You can’t bring all the stuff that the cloud is offering, and you have to kind of - not think differently about the what, meaning you’re still drawing the same triangles and squares on a whiteboard as you’re architecting, but the how would be radically different. And so that’s when we made the leap to create Synadia, at the end of 2017, based around a foundation that modern distributed systems will look very different than they do today, and they will be pushed very hard in terms of dynamic agility. They can change all the time.

Again, I probably am further along career-wise than some folks on the call, but it used to be that when a system was set up like this in the ’90s, early ‘90s, no one touched it. I mean, I can walk into any financial institution on Wall Street, and I could go into their server and point exactly where our stuff was running. And it never deviated from that. And that’s very different than what I think a lot of the audience probably thinks now, where it’s like “I don’t know, it’s somewhere in AWS East 2”, you know what I mean? “But I have no clue what machine it’s running on.”

And so our thought process and our priors were that that type of step change is going to accelerate with edge, and that the current tools of doing everything with like HTTP or HTTP derivatives just simply won’t be able to keep up, in our opinion.

Adam Stacoviak: Badly, but they do it for fun. And somehow that’s like their pastime. And they’re all Bob, which I thought was just like – the entire premise of that is such an anomaly in storylines anyways. I don’t know where else it’s happened, honestly, where you have the main character be many of the same character, but derivatives and whatnot… And there’s obviously – what did they call it…? “They…” What did you call it whenever they drifted? What was the drift word?

Allan Jude: Yeah, they’re forks of the same original code from the very, very early ‘90s. And so being 35 years later, a lot has changed in between there. So they are part of the same family tree, but the kernels are quite vastly different. And kind of the goals of each of the projects is very different. So NetBSD is really designed around portability. Especially back in the early days of Unix, that was one of the big things. Some people had like a PDP, and then they had faxes, and all these different computers that were completely different architectures. Before the invention of C, every time you had a different kind of computer, you had to rewrite the operating system to like the assembly language of that computer. It wasn’t like we have now, where the x86 and ARM are the two main kinds of computer, and we don’t have to deal with very many others. There’s PowerPC, and MIPS maybe, and a couple things like that. But we’re not dealing with hundreds of different architectures. There’s three, four popular ones.

So NetBSD wanted to maintain that, and works on all of those weird architectures that we just mentioned, and a bunch of ones you’ve probably never heard of, and a bunch that only existed in Japan, and never shipped anywhere outside of Japan. So NetBSD’s focus is really that portability, and being able to run on anything.

[00:35:57.26] A lot of their ethos from the beginning was “There were these bigger mainframe type machines, and they don’t make an OS for them anymore. We can be the open source OS, so people can still use to run these.” And that has a lot of value, but it doesn’t have quite the same mainstream appeal as running commodity servers with regular applications.

But NetBSD did pioneer a very interesting thing called package source, PKG SRC. And this is a package manager that’s not specific to one operating system. So it’s supported on various different versions of the BSDs, Linux’es, Solaris and its derivatives, Lumos and so on, and it allows you to get some kind of basic applications on stuff. People who are like “Oh, I need modern Python on HPUX”, PKG SRC is a way to do that. And that came out of the NetBSD project.

And then OpenBSD has really had a focus on security, and is kind of like a research operating system where they’re trying new concepts and ideas to make more secure computers. So they invented a bunch of common things we do. They’re the maintainers of OpenSSH, that every operating system uses as a way to securely connect to a remote machine - that’s maintained out of the OpenBSD project, along with lots of other applications. They have their own BGP server, and they’ve maintained a BSD licensed version of rsync, and lots of other applications that you might actually use.

But they’ve also pioneered a lot of concepts. They have a system called Pledge, where an application can say, at the start, after it’s done its initialization where it maybe needed root access for something, it can pledge that it will only use these system calls from now on. And if it ever tries to do more than that, because maybe somebody used a buffer overflow to try to make the application do something it’s not supposed to, it will actually – the operating system will say “You promised you wouldn’t do that, and we’ll stop the program.” And this allows you to kind of sandbox a program.

FreeBSD has something – it’s not exactly an alias. It’s called Capsicum, and it’s a lot more fine-grained, but it means it’s a lot more difficult to retrofit existing applications to that more locked down sandbox. And it works very well if you’re writing a new application and design it from the beginning to support this compartmentalization. But Pledge is a very easy way to retrofit existing applications to make them safer.

They also invented a couple other concepts. They relink the kernel, all the different parts of the kernel together in a different order every time you boot, so that everybody’s machine is different, and an exploit won’t be able to just work on every OpenBSD 7.4 machine.

Ben Cohen: Yeah, so the way I like to introduce Swift is actually - I go back to when I first saw Swift, and what really got me into Swift. So I actually wasn’t a member of the team when Swift first launched. I joined about a year or so later. And at the time I was actually working in FinTech, I was working for a bank, working on trading systems.. So we were a team, doing sort of equity and derivatives trading, we had full stack development, server-side and UIs… And we were using various different languages. So we were using, on the server side for these trading systems, that really needed high performance and low latency, we – the developers separated into two camps. You had the Java programmers, and the C++ programmers. And the Java programmers, they were really fighting the garbage collector all the time. Because every millisecond counted, they were making sure not to allocate too many objects, and using ring buffers of scalar types, and things like that… And they would look at the byte code to make sure everything was going well. And the C++ programmers found this hilarious. They were like “Why don’t you use a proper language?” But it wasn’t so funny when C++ code would segfault in the middle of the trading day, right?

So that was the server side. You had two camps of developers who had chosen a language, and that forced them into a pretty tough choice - safety versus performance. Then we had UI programmers; they targeting Windows, they were using Visual Basic, and then C#. C# - kind of a nicer language, I would say (this my personal opinion) than Java. It came out slightly after Java, and I felt like it was slightly improved, similar style. Obviously, they were enthusiastic about that, but they were, again, sitting on top of a garbage-collected language. And then we also had less latency-sensitive things; we would have Ruby programmers writing Ruby on Rails apps for things like static data maintenance.

[11:53] Ruby is such – I remember I actually listened to one of your podcasts where you had Justin Searls on, and he was talking about how Ruby, how they really focus on the joy of programming, and how it was such an enjoyable language to program in. But obviously, it doesn’t scale to the kind of latency-sensitive environments that you need for that sort of thing.

So it was really interesting that we were all using all of these different programming languages. And obviously, I think people should learn different programming languages; it sort of expands your brain, and makes you a better programmer. But it was kind of unfortunate, the choice of language really limited them in some way. And so my first impression when I saw Swift, when Chris announced it on stage, was like “Yes, this is what we need.” Because it was a non-GC language, compiles down to native, but it had that higher-level feel that actually made it really enjoyable to program in, a lot like Ruby.

So I think the sort of ambitious elevator pitch for Swift is that it’s something that you can achieve C++ level performance in. Obviously, you have to work at it; nothing comes for free when you’re actually operating that level. But the language itself feels a lot higher level, a little bit more like Ruby. Obviously, it has a static type system, but we worked hard with things like type inference, to make sure that the types aren’t in your face… We try and put an emphasis on progressive disclosure, so that – obviously, one of the key things for us is we want app development, which is the main demographic for Swift, to be easy to do. We want somebody to be able to sit down, not have the language get in their way, but also, we want the language to be powerful enough that the framework developers can create things like SwiftUI, that is like this super-high level programming language that allows people to be very productive, put together apps super-quickly. And so I guess that’s how I’d characterize it. It’s a native language, but with a high level look and feel.

David Flanagan: Well, yeah. I mean, let’s pick two example, because I think they highlight this problem very well. One of them is you wanna pick a time-series database. You’ve got many options. Apache Kudu is one that is available from the Apache ecosystem. If you’re in the CNCF’s ecosystem, we don’t just have Prometheus. We’ve also got M3, and we’ve got Thanos, and we’ve got Cortex. And from Grafana - I can’t remember the name. Apologies, Grafana folks. But they’re all very similar, and they’re all Prometheus derivatives… But how do you make that decision? You do have this impossible decision.

And then of course, if you move out of those three ecosystems, there’s InfluxDB, and there’s Timescale, and I’m sure there’s a couple more. And they’re all great projects, and they’re all worthy of adoption, but they all come with their challenges.

So yeah, I don’t know how people make that decision. Very, very tricky one. Obviously, Prometheus, and sticking within that API, you can transfer a lot of knowledge, but the operational aspects of Cortex and Thanos and [unintelligible 00:55:57.21].

Then there’s the streaming ecosystem, we’re seeing a lot more event-driven stuff in the CNCF… And again, if you’re in the Apache ecosystem, maybe you wanna go with Kafka. And if you’re in the CNCF ecosystem, you’re probably gonna be more attached to NATS. However, we have a new contender, Redpanda, which is delivering phenomenal stuff. We have a Kafka-compatible API, but writing it in C++, with a WASM engine… But you may wanna go for something that’s battle-tested, and that can be RabbitMQ. So again, how do you make that decision? I think it’s tricky for everyone. I’d say between us, we have a lot of experience here, and I still struggle to make some of these decisions. So hopefully, other people don’t spend too much time on it, and just try and pick something that leans into their ecosystem and works from there.

Charles Martin: Look, what are you trying to do? Like, what does it mean to be optimal? If being optimal means that your alphas are close to two or three, then you should adjust your hyperparameters such that the alphas go down. That’s what you do. Now, doing that analytically, typically, doing what are called analytic derivatives, meaning you try to compute the gradient from that - that’s somewhat difficult (it could be done), because you have to compute the eigenvalue spectrum, and then you have to fit it, and then you have to figure out the derivative. And that’s a very complex, nonlinear calculation. It’s very iterative. It could be done numerically, or it could be done analytically with some work. It’s a lot of work… I would love to have VC funding like Hugging Face to do that… But I don’t. It’s just me. Me and you. So you just try to tune your parameters; if alpha goes up, go the other way. If you turn your learning rate up, and you find your alphas are going up, turn the learning rate the other way, and hopefully they’ll go down.

[34:06] Obviously, it’s a complex optimization problem, because you have 100 layers, you have 100 alphas, and so you’re trying to tune different layers, you’re trying to tune your layer learning rates, and your amount of dropout, and the amount of momentum… So in principle, you could try to do that algorithmically, in a way, using like a Bayesian type approach, where you try to get your alphas to go down on every layer. In principle, you could do that. It’s a complex optimization problem, but that’s what I would recommend. And I think it’s theoretically well-grounded. I mean, the point is that you want to learn more correlations.

Typically, what I’ve found is that it’s a good tool for newbies, because you get into a model, you start doing something, and things are totally wrong. And you can go in and fix some problems. Okay, now we’ve fixed it; we’ve found – like, what did we not do? Like, I didn’t put the proper regularization on these layers. Let me add regularization and try again. And you can see that, okay, that’s much better.

So from a newbie perspective, it’s a very good tool, because it helps you get started. Now, it does work – keep in mind, the tool works at the end of training, not in the early stages of training. You’ve got to let the thing bake for a while. And once it’s about halfway through training, then you can start looking at things… It’s got to have some correlations. But this is what it’s for.

Typically, trying to do large-scale meta learning would just mean you’d have to integrate the tool into some sort of process that allows you to look at the alphas, or look at more details in the layer - the shape of the spectral density, the number of spikes, the alphas, the volume of the spectral density - and figure out how to tune from that. I mean, this could even be used in a reinforcement learning situation, where instead of the reward being something that the agent takes, the reward is Oh I got smaller alpha, So I have rewards on every layer, and I sum the rewards in some average way to try to get the optimizer to work, even in situations where I don’t know what the reward is for a reinforcement learning situation.

Obviously, that would be nice in areas like you’re trying to trade in the markets, because you can’t take actions – that trade, you can’t trade on historical data and expect to learn from that. So this gives you a way of sort of doing things in a supervised or semi-supervised way, that doesn’t require peeking at the test data to optimize. And then that’s it… I hope that answers the question, but that’s sort of the idea. And there are lots of things people I think want to try. I think it’s great if you try them.

Daniel Whitenack: [35:58] It is. Well, and I think that the structure – really the Open RAIL license is, I think, a structure. And the Open RAIL license that you would adopt for your model… Maybe you could adopt sort of Stable Diffusion’s Open RAIL model if you have a similar model, but other models are going to have other implications. So you have some flexibility in the responsible use clauses.

For the Stable Diffusion license, I’m just looking here, there’s sort of [unintelligible 00:36:27.07] all the way at the bottom, and it talks about use restrictions. And this is really this kind of clause that in my understanding is really important here. And they talk about you agree to not use the model or derivatives of the model, and then they have a bunch of things. So one of those things is in any way that violates applicable laws. That’s okay, so kind of boring… But if you if you go down a little bit, there’s very interesting ones, right? You agree to not use the model to defame, disparage or otherwise harass others. And there’s other kind of interesting ones, maybe specific to behaviors that they anticipated use of the model, but they wanted to avoid.

There’s one that says, “You agree to not use the model to provide medical advice and medical results interpretation.” So obviously, you could generate images with Stable Diffusion, or maybe interpret images with such a model, that are medical-related, right? Medical imagery is a type of imagery, right? So maybe that’s a use that they anticipated and had internal discussions about. And so there’s all of these use restrictions or clauses that go into the sort of RAIL part of the Open RAIL license. So it’s still open in terms of access and distribution type of things, but then there’s these clauses around responsibility that are explicitly included.

Jerod Santo: So when I think about platforms, Unix (and its derivatives) and the web, for my money, are like two of the greatest platforms ever created in terms of just opportunity and captured value, like people actually building things that change lives etc. And I think there’s some common things between the two. Of course, one is built upon the other, and it seems like Linux on the desktop never became a thing, but the web and web servers and server-side code really made Linux – or didn’t make Linux become a thing, but… Linux is entrenched because it was a great operating system for the web to run the server side.

So they’re related and one builds upon the other, but in terms of Unix, whether it’s in the philosophy or even in the implementation, and the World Wide Web and its design and its philosophy, do you see parallels? Are there commonalities that we can look at and say “These make for great platforms?”

Shawn Wang: [04:05] But I did the weird thing - I worked in currency derivatives in London, I had to do option pricing, and you have to use a functional programming language for that. My company used Haskell, so I did two years of production in Haskell before I was a developer…

Shawn Wang: For sure. For those who want the long history, I did a 2,5-hour podcast with Quincy Larson from FreeCodeCamp, so you can go check that out if you want. The short version is I’m born and raised in Singapore, came to the States for college, and was totally focused on finance. I thought people who were in the finance industry rules the world, they were masters of the universe… And I graduated just in time for the financial crisis, so not a great place to be in. But I worked my way up and did about 6-7 years of investment banking and hedge funds, primarily trading derivatives and tech stocks. And the more I covered tech stocks, the more I realized “Oh, actually a) the technology is taking over the world, b) all the value is being created pre-IPO, so I was investing in public stocks, after they were basically done growing… And you’re kind of just like picking over the public remains. That’s not exactly true, but…

Adam Jacob: [28:02] And that model we know works. If you build something of value and you wanna sell it for money, people will pay for it, because they want it. And it’s pretty straightforward. When we do open source, we’re making it more complicated every time. So in the most traditional one, we would do open core, where we say “Well, yeah, the main part of what we do”, which is usually the most valuable thing you do, because if it’s not valuable, then the open source users don’t need it… Is completely free. It’s zero dollars, and it’s open source; you can take the source and build derivatives, and do whatever you want with it. And then there’s some other thing we build on top of it, that’s even more valuable than the first thing was, to certain target markets. You know, analytics for the enterprise would be a good answer, or SAML support. If you want SAML support, it costs extra.

The problem with the open core model is that you always get it wrong, and you gave away the most valuable asset, which is probably whatever it was you did in open source. So in the case of Chef, the most valuable thing you got from Chef was Chef. You had configuration management, it ran Facebook for free. That’s amazing.

Adam Stacoviak: [39:59] Derivates, yeah. Authenticated derivatives.

Tim Triemstra: [20:09] Well, some of the things that the computers can do, like the instant wake and running the iPhone and iPad apps, are contingent on the hardware that’s in these new Macs. But one of the things that’s really cool is that Apple has pioneered and continues to lead the industry with this universal concept, where we’ve had Universal Apps for well over a decade… So what we do is with the tooling and with Universal Apps we’re able to make it really easy for developers to support both Apple silicon and Intel.

So people with Intel Macs will continue to get amazing software, and it’s really easy for developers to continue to support them. Something that I think a lot of people don’t really understand is how perfected that is. When we had the PowerPC in Intel, that was kind of like the signature time people looked at the Universal Apps. But over the last 15 years we’ve done the 32 to 64-bit transition multiple times. We had iOS moving to 64-bit. And in fact, in iOS we had a v6 instruction set, then we had a v7 instruction set, and there were derivatives of the v7 instruction set… There was a time when we were actually building three versions of the binary and putting them in the app, and users never noticed, because it just worked. And the App Store would even thin out the versions of the binary that came to your device so you didn’t lose space.

This transition concept is something that we’ve really had working extremely well for a really long time, and Xcode developer tools does this for you automatically. When you open up Xcode 12.2 and you open up your Mac project, we’ll convert it to universal, and the next thing you know, you’re building Intel and Apple silicon, and you didn’t really do anything. Users are gonna reap those benefits for years and years.

William Nichols: And we showed them that “You should be able to get in the test, and it should run the test first time. You’re not gonna get into here and run the test to find out where your problems were and fix the ones that hit, you’re gonna actually get it to run the first time. The thinking here is this makes the entire development process more consistent, it makes it more predictable, and it gets you a much higher quality product in the end.

So there were a handful of adopters, and the people who were in these PSP courses were from industry. There were some university classes, but 90% of the data was from industry; it was from organizations that we were working with, trying to teach them how to manage software projects. And part of the training was taking the PSP course.

So the developers on the team would all take the course, they’d learn these techniques for how to estimate it, how to improve quality… And then we’d use that to help them manage their project. And we had some very notable successes with that. I could go into a few examples, but probably the most spectacular was the rewrite of the BursaTech trading engine. You’ve probably never heard of it, but BursaTech is the stock trading engine they use in Mexico… And they were falling behind, and there really are only a handful of engines in the world, a handful of those big trading engines for stocks, derivatives and so forth… One of them was in Mexico.

Their choice was either rewrite it, because it couldn’t keep up with modern times, or buy the one they were using in New York or London. So they decided to rewrite it. And they with a relatively small team - I think maximum was a couple dozen people - it took them a little over a year. They put together probably about 500,000 lines of code. They had to run it in shadow mode before it went operational.

In the first year of operations it had a total of downtime that was in the order of minutes. Literally, minutes of downtime. And their performance was about a factor of ten. They could handle about ten times the number of transactions of any system that existed at the time.

So that was part of the benefit of doing things right the first way, doing things in a disciplined manner. That was kind of where all of this PSP training was coming from.

Chris Benson: I’m curious, from a very practical standpoint, with deep learning now being dominated by the linear algebra and derivatives that we’re always taking as we are training models, is that going to be superseded by different quantum techniques? In other words, would the current math and quantum be somehow working together, or are you essentially going to replace the current mathematics with a quantum variant of that, to get better performance where you wanna go on that?

Perry Mitchell: Yeah, it’s really cool. It’s a PHP-based system. It has a couple derivatives also - NextCloud, which I wanna give a shout-out to… Those guys have been amazing, helping us with integrations. All of it is open source, it’s as good, if not better, in my personal opinion, than some services similar to Dropbox, obviously because it’s free, but it’s also very fast. They have all the clients, mobile applications, and you pay nothing for it.

Of course, you need to be relatively tech-savvy to set it up yourself, but other than that, these services are a lot of fun to work with, and can teach you a lot about cloud storage as a whole. And it just seemed like a natural start for us.

We also added WebDev support, which many online web hosts support – I’m not sure whether Amazon Drive does anymore, but many of these cloud services support WebDev, which is a great way to integrate with cloud storage, which we also support.

Wes Jensen: That’s such a great point, because if you think about it, if they started from square one, when they found out what was hot on the market right now - like blockchain, or AI - they would never catch up. The ideas or where they think the application would be would be two years out by the time they caught up in calculus, and then understanding derivatives… That’s messy. So what they do is - you’re right; they go in in advance, they say “I’m just gonna do this, and then I’ll back-track. I’ll try to figure out that actual knowledge down the road.” And to Ali’s point - that can be dangerous, absolutely.

I would caution those “Don’t not innovate, and don’t not jump into it…” Absolutely. Because I will say something that will spawn the rest of your minds here much better than anything I could come up with on my own, regardless of what my mathematical history is, right? And I think that’s what we need - to spawn this innovation. But be careful. I totally agree with that, they’re following a trend.

Chris DeBellis: Good question. So at least for me - I’ll talk a little bit about my experience… To go from, say, traditional data science into deep learning algorithms, I think one of the big skills that you have to have is coding skills. You’re going to be doing a lot of coding, you’re going to be downloading other people’s code, probably from GitHub, you’re going to be configuring it, installing it, and then you’re going to be at a minimum tuning some parameters, but very possibly, especially if you’re doing this in a production setting where your code is going to be actually used for something, you’ll have to make code changes.

So the ability to code is really important, particularly in Python. Most of these algorithms are available in Python. And there’s a lot of debate out there… Some folks say “Oh, to do deep learning and data science, you really have to have a strong understanding of math and statistics”, and I think if you are doing AI research, that’s absolutely true; but if you are doing, like we talked about earlier, that transfer learning, a lot of the math and statistics comes from training the initial model. So if you’re using someone else’s trained model as your starting point, the ability to do the math and statistics become less important… And I know some folks are not going to like that, but that’s been my experience. Over the last six months, most of my time has been spent coding, not so much worrying about statistics and derivatives and matrix multiplications, because the software does that for you… So that’s one of the great things about frameworks like TensorFlow.

And then again, for me, to get started, I spent a lot of time watching YouTube videos. Stanford has a lot of great courses online. Their deep learning courses are online, and you can watch the lectures and really learn a lot from those. So for me, that was just enormously valuable.

Also, Udacity - I took a couple of Udacity courses. They have some free courses, they have some paid courses… Those are really helpful.

Maxime Coutté: [23:52] Yeah, he taught us those math concepts in a way that was really useful; he gave a course, then we did some exercises to be sure that we really understood the concept, and then we tried to code everything that we think that we understand, just to be sure that we really master the concepts. We’ve done this with the first component, the first accelerometer, and after having played with it and implementing some little thing, we switched from a component that is the MPU, and that of DMP, that does all the calculation and send us the Quaternion.

To be honest, we could have directly buy this component and use it with some documentation, and build the headset without knowing anything about the math concepts behind it, but we felt like we need to understand the math black boxes inside of these components. And I really feel like it was necessary to understand this math; if you don’t know about those math concepts you can build the headset, you can improve it, you can even do better than us, you can improve it in a way that we haven’t thought about, but we learned those math concepts just because it was a pleasure.

The component that we bought, the MPU - basically, it does all the calculations for us, because… In fact, that will be a little bit hard to explain with the language barrier, but I will try. What an accelerometer does is that it only detects acceleration. What is acceleration? If you akek a graph and you look at position and time, and you draw the function of position and time, and you search for the derivative function of position by time, you have what is called the speed. And if you look for the derivative function of speed, you have the acceleration. To go back to the position, you have to do two antiderivatives on the acceleration, to have the position.

Doing that, there is some drift happening because of all antiderivative works. Those two derivatives took a lot of calculation. Inside of the component there’s a part that is specialized only to do that. The core headset only asks the component for the result of this computation, and the component sends us the Quaternion, and we basically send them to the game.

Travis Reeder: Sure. The Go Tool did exist, I believe, when we launched, but it was pre-1.0. Our decision process was… Well, basically we were hitting a wall on Ruby, and we needed to change. I was a long-time Java programmer, so it was kind of that I really wanted to go back to Java to get more performance, or use something new, or one of these Java derivatives, like Scala…

Gerhard Lazu: Actually, no. That’s the edge redirects. So this is forward logs. Forwarding logs is we have to send them to Honeycomb and to S3. Honeycomb so that we understand how the service behaves, what are the hits… Like, remember all those graphs that I was able to produce? We need to be able to see which requests were hit, which were a miss… So all that stuff, I think in a day I could get the Honeycomb stuff done. I think. There’s nothing crazy about it. Some things will not be present, but most of it is fairly straightforward.

S3 is a little bit more interesting, because I haven’t seen that yet, and I’m not familiar with the format. But I know it’s a derivative of what we get in the request. So it’s just a matter of crafting a string that has everything that we care about.

And I’m going to flag any items – if they’re problematic. So honestly, I would say a few days’ worth of work, I can get the forwarding logs sorted.

[01:32:10.07] Then, moving to the edge redirects, the question is “How far do we want to go with them?” Are you okay with the current behavior, which everything expires in 60 seconds, and we can be serving stale content? Or do we want to implement what Jerod suggested?

Nathan Sobo: So if you want to build a closed source derivative work of Zed, you can’t do it. So you need to stay open source… I still think you – anyway, I don’t want to encourage it, honestly. That’s kind of why it’s GPL-ed.

Adam Stacoviak: So philanthropy is a derivative of capitalism, obviously, right? You probably couldn’t have – unless there was inherent wealth, or whatever else.

Chris Benson: What are your thoughts on the CPU derivative? What are your thoughts on performance and capability relative to its own base model and its GPU siblings?

Adam Jacob: [01:27:43.27] There’s a lot of cycles. System Initiative, you literally go to a modeling screen, you could look at the function that Jerod wrote, and edit it directly. So you could literally just open the function and start typing, and it knows that your function is a derivative of Jerod’s. And it’ll track that over time. So when Jerod publishes a new version, we know that we could automatically take your function and Jerod’s and rebase them if you wanted to. We could be like “Hey, Adam, you made changes to Jerod’s. Jerod’s like “Do you want to incorporate those changes?” You could send them directly to Jerod. You could be like “Jerod, I fixed your bug.” Right? And then Jerod could receive it, see how that’s happening, and then Jerod could decide to republish it. And all of that works because the fundamental nature of the data structure is that it’s a giant hypergraph, and it’s tracking all those things over time… Which is ba-na-nas. But very, very cool.

Chris Benson: Speaking of open source, a competing large cloud company, Google, decided to try their hand in the open source space as well, or at least the open model space, and they released a derivative of their closed source Gemini. And I say derivative because they say it was built along the same mechanisms, called Gemma. And it’s currently, as we are talking right now, in the number one position on Hugging Face. At least last time I checked, not long before this. Although that changes fast, and I probably should have checked right before I said that.

Stefano Maffulli: It’s not going to happen. So maybe - and this is my working hypothesis that I threw out there… Maybe what we need is a very good description of what that data is. Maybe samples, maybe instructions on how to replicate it… Because for example, that might be data that is copyrighted. You might have the right under fair use or under a different exclusions of copyright, you may have the rights to create a copy and create a derivative around the training. But not to redistribute it. Because if you redistribute it, then you start infringing.

So I think we need to be carefully thinking about [unintelligible 00:48:02.22] And the reason why I became more and more convinced that we don’t need the original dataset is because I’ve seen wonderful mixing, wonderful remixing of models, even splitting of models and recombinations of models, creating whole new capabilities, new AI capabilities, without having to retrain a single thing.

So I’m starting to believe, really, that the AI weights in machine learning, the weights in the architecture - it’s not a binary code. It’s not a binary system, binary code that you have to reverse-engineer. If you have a sufficiently detailed instructions on how it’s been built, and what went into it, you should be able, you might be able to create new systems and reassemble it, study how it works, and execute again. Modify. So the preferred [unintelligible 00:49:05.14] to make modifications is not necessarily going through the pipeline, or rebuilding the whole system from scratch… Which for many reasons may be impossible.