From WebGL to WebGPU

Hello one, hello all of our JS Party animals. I’m Jerod, your internet friend, and I’m joined today by my friend, Amal.

Hi, Jerod.

What’s up, Amal?

I’m good. I feel like podcasting has now become my “Oh, I’m just gonna go to the grocery to pick up something for dinner” kind of thing, where I feel like I’ve been on the show so much recently, I’m just like podcasting – I’m podcasting twice this week. That’s insane.

Oh, my goodness.

I know.

You’re like a pro podcaster now.

Yeah, yeah. I’m entering into the realm of Jerod and Adam, so… We’ll see.

There you go. Pretty soon people will be sick of you, like they are surely sick of me at this point… Well, today we have an awesome topic. This is a requested topic, sort of. It’s a requested guest, for sure - Gregg Tavares. Welcome to the show, Greg.

Hello. I’m happy to be here.

Happy to have you. The author of WebGLfundamentals.org, and I’m sure you have many other things that you do as well. This show was requested by listener Daniel Buckmaster; wants to talk about WebGL, and so I thought “Hey, I don’t know much about WebGL. I’ll email Gregg and see if we can talk about WebGL.” And he said “I’d love to come on the show. Are you sure you want to talk about WebGL?” Because WebGPU - that’s the new hotness; it’s going to be replacing WebGL 2, or kind of has, I don’t know… You tell us what you told me, Greg, and why we’re going to talk about WebGPU today.

Right. Well, WebGPU shipped in Chrome in May, and it’s going to be shipping in the other browsers as soon as they get it ready… Whereas WebGL shipped - I think it’s 11 years ago, 12 years ago, so…

Old and busted, is that what you’re saying? It’s old and busted?

It’s not old and busted, but it’s definitely done. I mean, there might be some minor modifications, but kind of time to move on. And WebGPU is the direction I think everybody’s hoping it will go.

And it sounds like it is going that direction. You said Chrome shipped it, it’s behind a flag, I believe, on Firefox and Safari. Edge, of course, Chromium-based, has green support on canIuse.com… So it’s becoming a thing. It’s not going to be a flash in the pan.

It should not be a flash in the pan, no. There’s a lot of people using it, a lot of different companies. I think Unity announced they’re supporting WebGPU. I don’t know if they’ve actually shipped anything other than a beta, but I’m pretty sure they announced that. If not, I’m in trouble. [laughter]

Breaking news.

No, I’m pretty sure they announced that. And I know other people working on other things as well.

Okay. Well, tell us a little bit about yourself. Obviously, you’ve made this website, which is a comprehensive guide to WebGL… What’s your background? What are you interested in, and how did you come to create this website and be interested in these technologies?

I am actually a game developer, at least I used to call myself a game developer… I’d like to still consider myself a game developer, but the fact that the last game I wrote was in 2007 kind of suggests that maybe I should reconsider that title. And I actually came to Google in 2008, to write a game engine. That was called O3DE; you can look it up. It probably doesn’t look quite like a game engine, but it was released and then abruptly cancelled. Google’s never cancelled a project they’ve released…

No, that’s new. That’s new. You’re the first one.

[00:07:41.17] It was totally the right decision… And I ended up working on WebGL, which was the correct replacement; it’s more it’s more flexible, as people write their own engines, instead of having to deal with ours. And that was the better decision. And for whatever reason, I started answering questions on Stack Overflow, and one of the problems with these kinds of GPU APIs is they require a lot of setup… So you can answer it as quickly as you can answer most web API questions. If you want to draw a circle with a canvas to the API. It’s like, you have a canvas, you create a context, you call arc, and then you call fill, and you get a circle, like four lines. No. It’s like 70, 80, 100 lines to do that…

To draw a circle?

Yeah, I don’t know how small I can get it, but it’s definitely not as small. It’s very low-level. It’s like assembly language, compared to a library. So people write libraries on top of technology like WebGL and WebGPU. And if you’re smart, you use one of those; if you’re a glutton for punishment, then you go your own direction, and do it all yourself.

I see.

Yeah, that’s pretty cool. Just to confirm… So WebGL, the actual spec/web standard, is something that was driven by Google from the work that you were doing? Is that correct?

No, I would say – I’m gonna get these messed up. It was either somebody at Mozilla, or somebody at Opera, or both of them together kind of pushed for a WebGL at some point. Somebody, not me, decided “Yeah, that’s the right direction. Let’s go that way.”

I see, I see. And how long has WebGL been a thing?

I think it shipped in 2011…

Okay.

So we started working on it before then, but I’m pretty sure it’s shipped September 2011… I’d have to go double-check.

Wow.

It’s been so long, and I don’t remember from yesterday.

Yeah, no, that’s pretty cool. And so now it’s on its next evolution, hopefully to be replaced by WebGPU, like you were saying.

I think that would be the way to look at it. There’s still some trade-offs at the moment, but we’re trying to get rid of those. So anyway, I want to finish a little bit… Back to that as a story. So what happened was I would answer questions on Stack Overflow; and because the amount of knowledge you need to use it effectively is so large, you couldn’t really answer a lot of the questions in the small format of StackOverflow. So I was like “Okay, fine, I’ll go write some articles.” Originally they were on my blog; after about five, I’m like “Okay, I should probably move to somewhere else, so maybe somebody else can contribute, or fix bugs, or things like that…” And it turned into WebGPU, or WebGL fundamentals.

That’s very cool. I’m super-curious about this, because there’s been this huge barrier to entry with WebGL, because it’s kind of – I don’t even know, I feel like you need computer brain to effectively use it… I always feel really dumb, and I’m like “Why can’t I visualize a matrix in my head?” There’s all kinds of stuff. So I’m just curious, can you talk a little bit about that friction? Because when you’re designing a spec, obviously, as a spec author, there’s a level of bias that you have, because you’re like the expert on this thing… So how do you get feedback from people to say “Is this thing easy to use?” Because I can count on one hand, and on one hand it’s one finger out of my huge network of the people that I know that are actively using this…

Yeah, there’s a bunch of answers to that question. So one is WebGL was based on OpenGL, which means it wasn’t – it was an existing standard, and WebGL’s goal was just “How can we bring this to the web?” So they didn’t really have the opportunity to redesign it. They were like “Let’s go with an existing standard, and then all these things that are using that standard will be easy to use.” Whereas if they wrote something entirely new, all those things would just have to start from scratch. So I think that was the thinking.

So outside of the web, there’s OpenGL, which has been used since the early ‘90s. And lots of native apps use OpenGL. It’s still popular on phones, as far as I know… And I think it’s still popular even on desktop for some apps. And so yeah, WebGL kind of had their hands tied. If their thinking was “We’re going to bring OpenGL to the web”, then they couldn’t really redesign API. All they could do is make it safe for the web and try to make it webby where it needed to be.

[00:12:08.13] Yeah, that makes sense. Thanks for sharing that context. That explains a lot. So this is a leaky abstraction in the worst of ways, where it’s just like “Here’s the JavaScript way to use this really complicated thing, that was complicated to begin with.”

The other way to look at both of these APIs is they’re kind of like Web Assembly, where they’ve come out with this thing called Web Assembly, or WASM, and you don’t generally write WASM; you use some other tool, like Enscripten, or – Enscripten lets you write C++, and that translates to WASM. So C++ programmers can write web code if they want. You see all the emulators – if you’ve ever gone to the archive and you look up Commodore 64 games, or Apple 2 games, they’re all running through WASM, which are running an emulator that somebody wrote for native, and they just recompiled it. So yeah, most people don’t write WASM directly. And in the same way, most people probably don’t write either WebGL or WebGPU directly. They use a library. The most popular one is probably Three.js. Another popular one is Babylon.js.

If you’re using MapsGL, or Google Maps embedded, both of those use it for you. You don’t have to do anything, you’re just doing something at a higher level. So that’s the most common way to get those things in, get usage. You don’t have to do it yourself.

Here’s how you know you’re low-level - it’s when you have a higher level abstraction called Three.js, that people also write higher levels for. Three.js, which I have used, and has not too bad of a learning curve… Still has one. There’s some verbosity there, and some things to understand… And that’s on top of this. So you guys are down there… Why so low-level, and why these technologies in general? You mentioned gaming, and then Google Maps… So what are the use cases beyond maybe those two examples, or in addition to? And why so low-level? Can you not bring it up a level for even the library authors?

Well, the reason it’s low-level is because you get all this speed from being low-level. So you can optimize your solution for whatever you’re trying to do. As far as examples, the hotness nowadays is machine learning, which uses these APIs. Also, everything on your display - your browser, all the text in it, all these camera images we’re seeing, those are all done through the GPU. The browser handles it for you, and there are high-level APIs to do it, but if you want to do it yourself, Google Maps is an example; that one’s all WebGL at the moment. I’m sure a WebGPU version is being worked on… I haven’t seen it, but it goes without saying that it has to happen. So this is basically the basic technology of getting anything on the screen nowadays, is these low-level APIs.

And a good example – I don’t know if this is a good example. Let’s see. Back when Chrome shipped, Chrome used the CPU to render, and they had a fast renderer, and it had its impressive demos. I don’t remember – they had like these bouncing ball physics demos that they showed off, and it was pretty neat [unintelligible 00:15:02.25] fast at the time. Then all the OS’es and the browsers added GPU rendering. Using the GPU itself will draw triangles for you, and based on that, you get all the stuff you see. Triangles with textures on them, so with images in them. And that’s mostly how stuff works today. But lately, using something called compute shaders, which - this lets you kind of do generalized computing on GPUs. There are newer algorithms that take it even further. So there’s a library called Vello, which you can look up - I think it’s called Vello - that does structure graphics, like SVG type of stuff, and you can use it to render UIs and things like that. That’s using completely software rendering; software meaning that… I don’t know how to say that; they’re not asking the GPU to use its older rasterizing functions. They’re actually computing the pixels themselves. This is like a progression.

[00:16:01.08] Another examples is - I think Unreal has a renderer called Nanite, and they’re also doing new techniques, where they’re rendering things themselves in ways that just 5-10 years ago were very less common. So if we gave you a higher-level API, those things wouldn’t be possible.

It makes sense.

If we only give you SVG, you can’t do the lower-level things.

That’s really cool. So I kind of want to take a few steps back, and maybe we can use this Google Maps app that everyone’s very familiar with to kind of walk through - okay, I’m an engineer on Google Maps. I have a ticket that’s landed in my queue, that’s like “Alright, here’s a spike to explore and estimate what it’s going to take to convert from our current implementation, which uses WebGL, to WebGPU. So here’s the spike to go investigate, what’s the effort? What’s the gains? What’s the migration path? What’s gonna break? What works across what browsers? How do we handle this gracefully, so that we can progressively update?” Walk me through that spike. Because that’s a very real use case, that a lot of engineers have to deal with, even within tools like React, for example. I remember back in the day when I was like – I think I did the React 16 upgrade for my team, and… You know, we just had to figure out how this is going to impact us, and… You know. So can you walk us through that?

That is gonna be hard… [laughter]

Yeah, it’s gonna be like a bullet-pointed answer, right? I get that.

Well, that’s his spike. He spiked it out, it’s gonna be hard.

[laughs] I love it, Jerod.

Well, I don’t know what’d be easier to understand. To me a more interesting one is when they went from the old Google Maps, where it served tiled images, to the new Google Maps, where it used WebGL. And I don’t remember if there was one in between. But that was a huge change, because it went from these prerendered tiles, to a system that – when they announced that, they pointed out they can change whichever roads are labeled or which roads are highlighted, or they can show different things, all because they switched to dynamic rendering. And I think at the time they couldn’t do it with a 2D canvas API, because it was too slow. But if you go into Google Maps now, and you zoom in and out, and you move around, if you’re lucky, everything always runs butter-smooth.

It’s impressive, actually, I have to say. That is one of the Google products that I use almost every day, and I’ve now converted my phone to – I have an Apple device. I had an Android for many years; I was like an OG Android user. I had the Motorola Droid, all the way to like the Pixel 3, and then I was just like “I’m done”, for lots of reasons, which maybe we can have another show about. But now that I’m an Apple user - obviously, who wants to use Apple Maps, right? Because I can always reliably use maps on web, on any device, and it works. It’s really, really good.

It sounds like it’s a prime candidate for cancellation, Greg. Don’t you think they’re going to be done with – isn’t it time that they just closed that project down? [laughter]

No, but I feel like people should know Maps uses tools like WebGL. And I think more so – I wish it wasn’t closed source, because I would love to see the engineering behind it.

Yeah, that one is probably gonna stay closed…

It’s probably gnarly, in my thinking.

It is probably pretty gnarly… And you know, it’s one of those things where there’s petabytes of data probably. They did something recently where you can actually pull the data out and use it in 3D libraries. And there are examples of using it in Three.js, in Babylon, and in Unity, and other things. So maybe that’d be more relevant to see that source now. I have no idea, I haven’t looked.

As for WebGL to WebGPU, nowadays - this is my personal point of view, I guess, but there’s like three or four major graphics APIs. There’s OpenGL, there’s Vulkan, there’s Metal on Apple devices, and there’s DirectX 12, 11-12 on Windows devices… They’re all different, but at a certain level they’re all the same. So porting from WebGL to WebGPU, depending on how you look at it, is just a matter of “Okay, over here I call fillRect() and over there I call rectFill().” [laughs] It’s not that simple… But at a certain level, it is.

[00:20:17.23] So what’s the advantage? What I’m trying to understand is, as an engineer, there’s two things I care about - how this thing performs, how this runs on the metal, how does it perform in the browser. The other thing I care about is the usability of the API. So what improvements do I get from using this? Is there a velocity increase in my productivity as a developer when using WebGPU? Or is it still the same complicated API?

They’re all complicated. They haven’t gotten the less complicated.

Okay.

The difference is new features have been added, that aren’t available in the old APIs. So those new features can enable huge new features that you couldn’t do before.

Okay.

So is there any reason to write WebGL today?

So… Yes. Unfortunately. These new features require new hardware. So if you’re targeting a market segment that is using very old hardware, you might find that WebGL reaches more users than WebGL 2, or WebGPU.

Gotcha.

Those users that have these old equipment go down all the time. You can also decide to fall back. In Google Maps it will run without WebGL. It just goes into a mode that loses some of these features. You can’t go in 3D; the roads and things like that might not get as dynamic. It doesn’t run at 60 frames a second, or as smooth, but it still runs. Kind of like you can still get the – well, I guess they’re deprecating that, but you can still use the plain HTML version of Gmail…

Yeah… For now.

…versus the full app version. For now, yes.

But I wonder if there’s an old .gmail.com, where you could forever use the HTML version.

Like old Reddit?

Like Reddit, yeah. So I guess - here’s the thing… I think for me, this seems like a perfect use case for a progressive experience, where you have “Here are the old APIs, here’s the new ones.” We have a codebase that’s going to live side by side until the hardware catches up more universally, or something like that… But I feel like there’s definitely a marketing problem for WebGPU… Because if it can’t run on all the same devices as WebGL, what’s my incentive to spend the engineering time on this? Because I’d have to maintain two codebases, or two implementations.

Well, I think you have to look at your particular market and decide whether the percentage of people who have these old machines, or somebody you want – because let’s say you’re making a game; no matter what you do, the game is not going to run fast enough on their machine. So then there’s no reason to target that old stuff, as an example. So it kind of really depends on what you’re making. I don’t know if I have anything else to say here, but…

Yeah, that makes sense.

What you’re making and where you’re starting. Like, are you starting fresh? And what are you targeting? Because I would actually come from the other angle. I would be like “Why not WebGPU?” That’s what I would first ask myself. And if the answer is “Well, because I need to support these devices ongoing”, well, then that’s your answer. But otherwise, I’m looking forward, and I’m seeing support coming along for WebGPU, and I’m thinking “Why WebGL?”

Another example is maybe the thing you want to do - you just can’t do it any other way. So WebGPU gives the web page access to something called compute shaders. They’re not available in the other APIs… And machine learning is pretty much compute shader-based. So if you want to do large language models in the browser, there are ways you could coerce it into the older APIs… But yeah, basically, the thing is you should probably want to use WebGPU.

Yeah, that makes sense. And so I’m curious, since these things are going to be living side by side more or less indefinitely, because of the arc of the web… Are there features going to – I guess, is there a deprecation for WebGL? Because –

[00:24:06.11] I don’t know if we’d use that word yet… I just don’t think there’s going to be a lot of effort. All the people on the browser teams that would work on the two APIs – well, they’re the same people working on both, and so they only have so much time, and they all want to move forward. If they had to split their time between the both, that would be a huge drain, so…

So new features probably will exclusively be added, like high likely exclusively be added to WebGPU.

That is my guess.

Okay, that makes sense.

So if most of us plebs are using this via libraries, are the libraries then being ported? So if I’m on Three.js, and I never wrote a line of WebGL, can I just be like “Hey, Three.js, switch to WebGPU mode” and I get to use that now?

Pretty much. Yes.

Awesome!

It’s pretty cool.

No more questions. [laughs]

Yeah, we’re done now. End of podcast.

It’s all you needed to know. Yes.

Okay, actually, I do have questions about this API. So can you maybe tell us about what’s new? You mentioned one of the shading things, and that’s – that’s the term, shading things?

So let me try to make fair – yeah, these APIs basically just run functions on the GPU. And that’s where all the complications comes from, because my analogy I like to use is there’s functions like map and reduce, where you give them a function to run and they run them… So imagine you could transfer that function to another machine, and say “Run it over there.” But in order to do that, you also have to transfer all the data over to the other machine, and then you need some way to tell that function “Okay, how do I find the data? Where did you put it? What is it called? What format is it in?” And that’s where the complication comes from.

So in the older APIs – well, in both APIs, there’s something called vertex shaders. And these compute vertices on the screen; just positions to draw triangles. And whenever a triangle is computed, there’s another function called a fragment shader, and as that triangle gets turned into pixels, it goes along and it says “Okay, I need to draw a pixel here, a pixel there, a pixel there”, it calls the second function, the fragment shader, to say “What color should I make this?” And you tell it, and you provide a function to figure that out.

And then in WebGPU there’s also a compute shader. A compute shader is just a function where you say “Call this function, this many times.” And that’s all you get; you get an iteration number in the function, and the rest is up to you. Use that iteration number to index into arrays, to write into arrays, to do some computation… But the complication that comes is that you have to write these functions in some other language that’s not JavaScript; you transfer them over to the GPU, and then you have to transfer all of the data, and then wire it all up. And in WebGPU, that all happens by index. So you say “In index one, I put this data. In index two, I put that data. In index three, I put this data.” And in your function, you have to say “Okay, this array, array number one comes from index 12, and array number nine comes from index three.” And that’s where it turns into more code than a higher-level API.

Yeah, yeah.

And then generally, you don’t want to deal with the numbers, so you’re gonna write something much higher level for yourself; so you can deal with names and all that stuff, but underneath you have to wire it all up.

Yeah. And this is why this honestly makes me –

Greg’s down there underneath, wiring it together…

…my brain is like exploding right now. Honestly, this is triggering feelings that I had when trying to play around with this a few years ago. So is it safe to say that WebGL and WebGPU are kind of like – it’s like rendering on steroids? Because HTML assumes this document type structure. And in the world of the Web GPU and WebGL everything is just kind of like - I don’t know, it’s like you’re in the matrix, and there are no linear shapes, or whatever. So I don’t know, it’s just – it’s really cool that we can do this type of rendering in the web.

Yeah… I’m not sure what the question was already…

Yeah, no, I was just curious to see, is this – I mean, how would you compare this to kind of the HTML rendering engines, essentially?

[00:28:15.25] Well, I guess I would compare – the maybe easier comparison is SVG versus canvas. In SVG you declare this hierarchy of elements, and then it renders it, and you can update the elements, and it’ll update it. That’s very similar to the DOM; in fact, it is the DOM. Versus the Canvas API, where you just call these functions to draw and they just put pixels in a rectangle… And there’s no structure anymore. There’s nothing left there.

It’s more imperative versus declarative, right?

Right.

So it’s HTML, and with SVG you’re declaring the shape of a thing. But with this, you’re actually giving it instructions on how and where to draw, and when, and to update etc.

Yeah. Except that, again, WebGPU brings in compute shaders, and now it’s just [unintelligible 00:29:00.01] It doesn’t have to be about drawing.

The whole machine learning thing I think is definitely interesting. It’s also concerning I guess for me as a guy who visits a website on my phone, and all of a sudden it’s machine learning in my browser… It seems like there are security and privacy concerns with this… Isn’t there?

Yeah, that’s a good question. And I have no idea.

Okay.

I can tell you – I think, I’m pretty positive this is true… If you go into a video chat app like we’re in now, and it has the options to replace the background, or put a hat on your head, or eyeglass on your face, that’s all compute shaders type of stuff.

Right.

They might be using WebGL kind of coerced into kind of doing the same thing a compute shader would do better, because they didn’t have compute shaders until now… But yeah, those kinds of things are things that are common to use the GPU for.

Do those things usually require permission, or like explicit from the end user, of like “Yes, you may use my GPU” or “No”?

Not currently. Maybe some browsers have that option, to say “Hey, just don’t allow pages to use this.”

I mean, most people won’t know what that is, but…

Right.

…you know, you go to a website, it’s doing some inference inside of your browser, and your phone starts to get hot, and you’re like “I didn’t know you were doing inference inside the browser.” That’s what I mean by – that’s not really security… I don’t know what it is. It’s like abuse, to a certain extent.

Yeah, that’s a good question. I don’t know that it’s worse with GPUs than CPUs…

Well, just because of this new use case of compute, versus – I thought you were saying WebGPU versus WebGL. But yeah, you could do it with a CPU, I guess, already.

Yeah, you could spawn a bunch of workers and do a whole bunch of work on a CPU. It’s not like it adds a new

capability…

Yeah. There’s plenty of ways to abuse your end user out here…

Right. Hopefully, it’s either “Hey, every time I go to the site, my battery goes down. I’m not going in there anymore.”

“I’m not gonna go there anymore.” [laughter]

Don’t reward bad behavior.

Right.

It’s hard to know these days; there’s so many battery-draining apps.

Yeah, that’s true.

[00:32:10.25] But it does bring up an interesting ethical question though, which is “Should there be some type of user consent for heavy compute?” Where it’s like “Hey, this is your device, this is your battery; maybe you’re in an airport, maybe you’re in a train station…” Imagine going to a website and just like having 10% of your battery drained in like five minutes. That would that would hurt, you know? And so I’m curious… Any thoughts on that, Greg?

I’m not the person to answer… I would ask what’s the difference between that and a native app? You go to the native app and it starts using whatever features it wants…

Yeah. I would say this applies probably across the board, just in general.

I would say there’s a sentiment that when you visit a website, that you’re not giving it what you’re giving a native app. Like, I’ve installed you, I’m launching you from my desktop, versus I follow a link off of x.com to a new website… And all of a sudden it’s just – I just feel like it’s more lightweight. It’s the web. You’ve got tabs, and you close a tab, open a tab, visit a website, hit the Back button… It doesn’t feel like you’re giving it native app experience, unless of course you install it as a PWA or something, which is all good. So it does feel slightly different… But that being said, when I open up Google Maps on my phone inside of Safari and I’m using that, I understand it’s doing some heavy stuff inside of there. And so maybe it’s not super a problem. Definitely not one that Gregg has an answer for; he’s made that clear…

[laughs]

So we’re just kind of talking out loud here…

We’ll just speculate on your behalf, Greg. It’s fine.

Which is what you do usually, versus…

Well, I mean… Yeah, speaking for myself - a lot of native apps are just a browser. And even if they’re not, they’re probably using some ad library, and they get updated every week, and you don’t know that what you installed when you installed it is the same thing you’re running now… And the ad libraries are bringing a third party, so they could have added something to it… It’s not really that different of an experience. I know that it seems like it, but if you actually drill down to the tech, you’re not actually getting a different experience. Especially with the autoupdating, all the time. Every time you go and – who knows what technology [unintelligible 00:34:25.18] and to show the ads, where they’re coming from, or what other third parties they’ve installed. Maybe even their database library is updated weekly, and it’s a third party library… So it’s not really that different.

Yeah, that’s fair.

I’m not trying to make excuses, just… It’s a problem in general, and I don’t know how to solve it.

Yes, agreed.

I’m all for if you want to be able to block it, block it. For anything. I don’t want a random website to access my camera, I don’t want them to access my mic without permission… I could see the argument that some people might want them not to access – I mean, people turn off JavaScript; there’s people who just turn it off. They say “Hey, a site should not be able to access JavaScript, unless I allow it.” And that’s a totally reasonable request.

And I do think it’s something that the browser builders can do. Because for instance, Safari on MacOS does have a feature where it’s like “Hey, this tab is using more battery or more memory than we typically expect from a tab. Do you want to close it?” You know? And sometimes you say yes, and sometimes you say no, but at least then you know what’s going on. So I think that definitely it’s something that at the browser level I think we could provide affordances for people, just to know what’s going on inside of their browser, and make decisions.

File a feature request. On all the browsers.

Yeah, totally.

Absolutely. On all of them.

And speaking of features, I’m actually curious, Greg, to hear what is your favorite WebGL or WebGPU API, and what is it, and how does it work? And what does it do?

I’m not even sure I know what that question means…

[00:36:02.10] Well, what’s your favorite API? Do you like 2D rendering, 3D rendering? Do you like playing around with lighting? I don’t know… Do you like doing stuff with textures? Shadows? I don’t know, what’s your favorite thing to do?

I like games.

Okay, you like games. You like building games.

I like games. So I want to enable games.

Okay.

And I love seeing artists use this tech to make things I haven’t seen before. So if you go look at – I don’t have one off the top of my head, but you can find… You go look on Steam, and you look at all the games; I’d like to see some of those be brought to the web, if possible. Or at least to enable them. That’s the place that I guess I find the most inspiration.

Are there tools for game developers for cross-platform kind of ports over to the web via WebGPU or WebGL?

There are. I mentioned Unity before. They already shipped long ago WebGL and WebGL 2 exporters for Unity games. And there are websites - I think [unintelligible 00:37:00.04] for example, is full of games; you can just click on them and they’ll run in the browser.

Yeah. You effectively get that for free… I mean, sure, there’s probably a little bit of –

To some degree. There might be things like networking, that you have to use the right libraries; if you’re using something that’s native only, you’ll have to figure out a way that it’s going to run on the web. The biggest issue that people have with games is that games are often - especially big ones, are often like a gigabyte download, or multi gigs… And then that’s not really a web thing. So the biggest problem I think developers have is they start just making a native game and then “Oh, I can push this button and get a web app? Okay, I’ll push it.” But did you actually design this for the web? No…

Right.

So I think the games that will actually do well are going to require web-first design. And I haven’t seen too many of those.

Yeah. I’m in the same boat there. By the way, side note, I read your recent blog post about Tears of the Kingdom, and I would love to maybe have a separate conversation with you about that game, because we have different takes on it, Greg. Different takes. Both being huge fans of Breath of the Wild. So maybe if you have time, you can stick around afterwards and chat with me about that. But I’m thinking about your teaching; you spent all this time, you’ve written all this stuff for WebGL Fundamentals… Is WebGPUFundamentals.org - is it a domain held? Are you rewriting it…?

It’s already there.

It exists, and it’s done?

It exists, and – well, it’s not done; not even close. But I’m working on it. If I get lucky, Google will give me some time to work on it… But doing it on my own, it’s gonna take a long time. I think I calculated, to get all the same lessons from WebGL Fundamentals to WebGPU Fundamentals, it’ll probably take another two months of full-time work. And so that’s a lot of work.

That’s a lot of work.

And it’s a lot of redoing it all, because you don’t want to keep it – obviously, you have to do… It’s not just like copying it over verbatim and kind of translating; you’re also like “Oh, this would be better if I did it this other way. Okay, let’s spend some time on a new diagram”, things like that.

It seems like something that Google would be incentivized to allow you to work on.

Well, I think they would be, except that there’s just too much other stuff I need to do. [laughs]

Other things that need doing.

Yeah.

So what is your nine to five, in terms of what’s your marching orders there? Are you down there implementing WebGPU implementation?

I am.

S the Maps team hasn’t – they haven’t sequestered you to help with their migration…

I think – no, they’re mostly like “Yeah, get your low-level work done, so our [unintelligible 00:39:30.26] can get their work done.” [laughter]

So you’re lower-level than the low level. So we’ve got the real low-level guy here, Amal. This is nice. What about us high-level folk? What’s the best on-ramp?

To me the best on-ramp would be go to Threejs.org, click on the Manual button, and read the first page. It will get you some cubes on the screen just as an example… You can probably pick a loaded gltf file, which is a format for 3D files. They’re easy to find online, and then once you know how to load one and display it, then you can start putting 3D together. You can deal with that in an hour or two of looking up – or you can ask your favorite machine learning thing “How do I do this?” and it’ll probably tell you…

[00:40:20.10] [laughs] Make sure it has access to the internet so it’s not outdated… Because at least with WebGPU, it’s a moving target at this point.

With WebGPU no, but if you use a higher-level library Three.js, there’s plenty of right tutorials that are not out of date.

Right. What about the gaming side? If I want to get started with gaming in the browser as a creator, what’s your go-to?

That’s a hard question. There’s something called Play Canvas; I’ve never used… It’s one of the few full web-based, web-first engines, but I haven’t actually built anything in it. Unity is obviously very popular, and you can just click the button, export, and you’ll get something. It just comes back to that thinking about the web when you’re designing for the web, as opposed to just like throwing in all of Grand Theft Auto assets level in there and saying “Hey, you’ve got to wait six hours while you download this before you can start playing.”

Plus you’ve gotta worry about that new fee structure, you know… [laughs]

The other one is – yeah, exactly. The other one is – for me, I like games; a lot of times - it really depends on what you want to make. You can prototype it in a much simpler way with the Canvas 2d API, or one of those others. Suppose you’re making a 2D shooter or something like that - just get something working before you go crazy and turn it into 3D.

I don’t know if you’ve ever seen, there’s something called a fantasy console, the most famous of which is called PICO-8. And it’s kind of emulating… It’s not emulating, but it’s kind of like “Hey, so I’m gonna give you a Commodore 64. Make a game.” And the point is that with this limited system, it only has 32k memory, the screen is only 120 pixels by 120 pixels, it only has 16 colors… You free yourself from “Oh my God, how am I going to make this look as good as Elden Ring?” You just keep it simple, get it done. And so people find that a great way to start. And then they might start there with a prototype, and then once they’ve got something, “Oh, this is fun. Fine, let’s make it beautiful, and we’ll take it to another engine.”

This is really cool. I will add this to the show notes. These are really neat games, just on their homepage.

Yeah. I’m curious, like, if you think that there’s a future, or maybe even a present where you have generative AI basically creating the next Angry Birds, you know what I mean?

It’s already happened.

Okay, it’s already happened. Could you tell us the story?

Well, this is something like ChatGPT 2 or 3… Somebody said “Hey, write me some games in JavaScript”, and they described the games, and they made – you had to squint really hard, because they made [unintelligible 00:42:50.28] They made a tiny little Zelda adventure game, they made a – [unintelligible 00:42:55.15] Okay, there’s a block on the screen that represents the player, there’s a block on the screen that represents enemies. I want the enemies to go to the player. If they touch the player, he loses hit points. If the player strikes one, she gets a gold point”, those kinds of things. And they went through all the prompts they had to use to get it. And this was old. This is like three, four, five years ago. My mind might be bad, but…

You’re way behind, Amal. Come on.

Yeah, I do remember that. I do remember that. But I guess my question was really specific to using JavaScript in the browser, creating these amazing, rich, browser-based games.

Getting it to an amazing level… Yeah, it seems obvious…

That’s the trick.

It will happen. You’ll ask Midjourney to make your art, and you’ll ask some other thing to write your code, and it’ll just work. I’m sure people are working on it. And I know there was some post where somebody made like a Magic the Gathering card generator… And I’m sure they can make a card game generator, and then glue those together.

There was also a talk - this is not AI-related, but there was a talk where somebody made a slot machine generator, and they actually published 1,500 of them… Until they got taken off the store. But they gave a talk about it at GDC…

[00:44:16.07] Wow…! That’s hilarious.

And they had made it – so they just typed in a couple keywords, like, you know, unicorns and candy canes, and it would make slot machines with unicorns and candy canes and publish it.

That’s hilarious that it got taken off the App Store. Why? What’s the reason? And this is why – I mean, for anyone listening, this is why it’s really important to bring these technologies to the open web. Like, no one should be able to take down your app, or your store. There shouldn’t be this [unintelligible 00:44:38.23]0] stopping your creativity. Like, you want to publish 1000 websites? Knock yourself out. I think you should have the right to do that, you know?

Yeah, totally. As for the app store, I think the App Store is both – I don’t know how. They have some limits… I have no idea why [unintelligible 00:44:53.21] Something’s fishy is going on here. I have no idea. I’m not the person who decides.

Yeah, but maybe that’s going to be the new normal in the world of [unintelligible 00:45:02.13] creation. I don’t know, we’re gonna greatly exponentiate the number of stuff that’s on the web, and with that we’re gonna have to figure out how we’re going to curate it. I think curation is like our next big problem. How do we surface important stuff? How do we filter signal from noise? I think that’s going to be our next challenge. In every genre: film, art… All the things.

You’ll probably just ask the AI, “Hey, what’s the good stuff?”

[laughs] Yeah, to be meta, right?

Yes. Let them curate themselves for us.

Let them curate themselves for us… [laughter] Really, Jerod?

I mean, why are we working so hard?

I agree, you know? I mean, it’s gonna be like “Alright, well, man, I’m feeling really lazy. I need to take a shower, but I don’t feel like washing my hair.” And it’s gonna be like “Well–” My first thing will be like “Oh, can I get a bot to do that?” Is that gonna be our default to everything?

Pretty much…

Can I use AI for that?

We’ll end up like the people on WALL-E. You know, open the spaceship, where they’re just like – they don’t do anything; just kind of float around on their pods, and get overweight.

Oh, good Lord…

I’m really excited about all that stuff, to be honest… You mentioned generating the games, and there are times when I want to make a game. The artists may be scared of this, but where I need assets, I need a character to move around, and I need a forest of trees, and piles, and a castle, and at some point I can just ask “Hey, make this…” I don’t know if that’s good or bad. I’m not trying to make a judgment.

And I don’t think you’re gonna get the style that you want, a cohesive style for anytime soon, maybe, where a real artist that you hire, or if you’re an artist, to have a vision; it’s probably not going to come from AI soon… But I don’t know.

Yeah. I would agree, I’m very excited about like the assistive elements of AI. That’s huge. I mean, I would love a team of engineers working with me, for free minus the cost of the planet, at any time of day. That’d be great. Engineers that are smarter than me too, that know more programming languages, and can fix more bugs than then they create… Yeah, all that sounds great.

Well, that leads me to my final question for you, Greg. A serious one. Why have you not created a game since 2007?

Wow…

Yeah, that’s a good question I ask myself every day.

[laughs]

And I talk to friends, “Hey, we should make a game”, and then yeah, we don’t do it. One of it is golden handcuffs. You come to a big tech company and you’re like “Oh, what? I don’t have to crunch, and I get paid a lot more?” Another is that the industry changed. The last I shipped it was 2007-ish… The iPhone came out in 2007; mobile games became a thing. Free to play games became a thing. When I came to Google, I mentioned that I came to write a game engine; that was because at the time Unreal actually cost like a million dollars to use it.

[00:48:18.13] And I remember – Unity might have been free, but they didn’t have the reputation yet for “Oh, this is something you should use.” So I came to Google to make an open source engine. That all changed. Now there’s two – they’re not free, but they’re free to use to prototype at least… And to even ship. It’s only if you make a lot of money that they want some money.

And there’s things like Godot, and there’s a whole bunch of other engines out there now that weren’t around back then. So that’s changed it in multiple ways. It’s made it easier… So theoretically, if those existed back in 2008, then I probably wouldn’t ever have come to Google, and I would have just tried to make something… On the other hand, there’s – I don’t remember what the numbers are. There’s several thousand new games every day. And so it’s just much harder to stick out, to look at that and go “I have an idea that’s gonna sell.” Oh, wait, I’m gonna put three years in, and it’s gonna sell five copies? I’ve had friends that do that; they made the game look beautiful, they shipped it… I think they sold 80 units.

Yeah…

And it was two years of their life. So games have always been that way. You’re like “I’m going to be a rockstar.’

It’s a moonshot.

Yeah. Finding a way to be confident that I’m going to make something that’s going to have a market… And it’s something that – I want to make something that’s unique, and having a unique idea is also… I’m not complaining about anybody else’s game, but there’s so many games that are just like “That’s just [unintelligible 00:49:38.07] Yeah, I’ve played Metroid, I don’t need to play it again. I want something new.

Yeah, yeah, yeah, yeah.

Well, that’s how I feel about modern music, modern movies… There’s just a lot of – they’re remixes, but they’re not inspired remixes. They’re just kind of like shallow clones.

Yeah. I’m not saying I don’t get enjoyment out of them, but… I would like to do something that somehow sticks out, at least in my head. I know a lot of designers, they probably think their game is completely different, but I can’t tell… At a glance. Maybe if I got into it I could tell.

Do you watch that show on Apple, Quest-Something?

I have not. Mystic Quest I think it was called, something like that?

Mystic Quest, yes.

I haven’t watched it either. Why? Is it good?

Yeah, I watched a few episodes… It seems interesting. For example Silicon Valley, that show on HBO, about the tech industry and startups - I can’t… It’s very triggering for me. I cannot watch that without thinking “Oh my God, that’s like 50 people that I know are just like that…” It’s really hard for me to sit through that show. I haven’t watched anything past the second season, because I was like “Oh my God, this is too much.” But I feel the same way about that show. It’s just too much like work, and I’m just like “I can’t…” I watch TV to like –

Because they work at like a game dev company; that’s what the show is. It’s a group of people that are creating video games.

Right. Right. Right. It’s just, there’s too much similarity with how I work, so… But yeah, I was just curious if you were a fan.

I have not watched it. There is a book that I just read, that was recommended by a close friend who’s also a game developer, called “Tomorrow and tomorrow and tomorrow.” And I was kind of shocked that it has – I don’t remember if it’s 70,000 or 120,000 reviews on Amazon. It’s like “Wow, I’d never heard of this thing, and apparently it’s pretty popular.” But it’s about two kids who meet in junior high school, and in college they make a game company. And it was really like “This is my dream! Why didn’t I get to live this life?” [laughter]

Aw… You kind of have.

Well, I did…

Yeah. Honestly, if you ask me… I mean, if you’re someone who’s contributing to the web platform at a low level like you are, you have a legacy that I think is cooler than creating games, in my opinion. Because you’re enabling others to create games. So you’re like the baseline, you know what I mean? So think about it that way. You’re upstream now.

Alright…

[00:52:07.22] “Alright…” [laughter] I don’t think he’s convinced. He’s over there thinking “Okay, I’ll think of it that way”, but he’s not really thinking about it that way. He’s over there thinking about his game.

It’s like, you want to be the director, and you’re making the canvas. [laughter]

He’s like “I don’t want to be writing scripts… What’s a director and a camera person gonna do without a script?” Greg, you’re writing the scripts, so…

I think writing the scripts is the thing – no, I’m making the pencils.

[laughs]

That’s right. He’s pretty low level.

Alright. Alright. Alright. Anyways…

It needs to be done.

Well, I think what you need to do, Greg, is take WebGPU fundamentals.org and hand that off to ChatGPT. You’ve already written the first version… Just tell it to port WebGL over to this, and I’ll spend my time creating the next great video game. I think that’s your move.

Yeah, well, I don’t know about the thought of handing it over to ChatGPT, or whatever. It has crossed my mind. But the thought that all I’m doing is writing stuff to train ChatGPT… [laughter] Because nobody’s actually going to come to the site; they’re just going to ask ChatGPT, and it’s going to tell it what it learned from my site. But you know what? That’s fine, because that’s just the way the world is gonna work in the future. All the kids are gonna ask their favorite large language thing or AI, and it’s going to train it. And that’s just the way it is now.

Right. In that case you’re not even making pencils. You’re like mining the lead, you know? [laughter]

I’m teaching our overlords.

Geez…

Yeah, exactly.

Oh, man…

Well, we’ve come full circle somehow… Or haven’t… But we need to close down. Any final questions, Amal, before we let Gregg go?

I would say that everyone should at least spend 15 minutes checking out this website. I think it’s pretty cool. WebGPUFundamentals.org. Check it out. It’s solid. No pun intended. [laughter]

Yes, please check it out, and help out if you want. You’re free to write articles. It doesn’t have to be my thing. I don’t know that anybody ever will write one, but… Yeah, it’s definitely – it’s all open source. You can submit a PR if you find a typo, or you want to write an article, or you want to translate it. It’s also set up to be easy to translate.

Nice.

There’s already, I think, six or seven articles that have been translated to Chinese, and two are to Japanese already… And yeah, people volunteer, so that’s awesome.

Dope.

Very cool. Well, here’s my final word… If you are out there building games on the web, or if you’re using WebGL, or WebGPU for anything cool, contact Greg, let him know, thank him for the underlying work that he’s been doing… And even just enabling folks like the Three.js folks to do what they do… Which perhaps you’re building on top of.

So I will submit that to you. I know that that can go a long way, when you feel like you’re just down there toiling, and maybe only the AI crawlers are going to be reading what you’re writing… That’s not a good feeling. So hook Gregg up with some props for enabling so many of us to build awesome things.

I will link up all of the – specifically the game-related stuff, as well as the book you mentioned… All the things. If you’re interested in [unintelligible 00:55:21.12] PICO-8 etc. in our show notes, as well as of course WebGPUFundamentals.org. So y’all can visit those things and check them out for yourself.

Greg, we appreciate you coming on the show, and for being a good sport, letting us pick on you a little bit, and schooling us as much as you can. I know that we couldn’t quite go as deep as you can go… But we appreciate you joining us today.

Thank you so much for having me. It was great. And yeah, check out the site if you want to actually know how to do it, because it’s not something I think I’ve explained in [unintelligible 00:55:51.03]

He was about to say “in a podcast”, but then he changed his mind. That was pretty funny.

Exactly.

You can’t see his mouth, everyone. I saw his mouth. He started to say the word “p…” And he was like “Alright… Yeah, no.” Alright, I get it. I mean, sometimes I wish our podcast had this [unintelligible 00:56:08.00] where it’s like “Oh, here’s like the AR version of our podcast. We can share screens with you.”

But I think it’s just as important for folks to understand how to talk about technology without a screen, because I think for me the most interesting conversations I have as an engineer are with my colleagues, when we’re not sharing a screen. I just want to put that out there.

Cool. Well, my last word, even though I said I was gonna have a last word, is that podcasts aren’t great for everything; different mediums for different things. What we can do though is we can expose people to new ideas and introduce them to places that they can go and extend their knowledge, such as WebGPUFundamentals.org. So hopefully, we have done that for you. Hopefully, Daniel Buckmaster at least is happy, as he requested this episode… But I think it’ll be more than just himself.

Alright, I’m going to say goodbye before I continue talking even more. This is JS Party, I’m Jerod, that’s Amal, with Gregg Tavares… We’ll talk to you all next time.