JS Party ā€“ Episode #254

Project Fugu šŸ”

with Thomas Steiner

All Episodes

Thomas Steiner (Web Developer Advocate at Google) joins Amal & Nick to talk about Project Fugu ā€“ an effort to close gaps in the webā€™s capabilities enabling new classes of applications to run on the web.

Featuring

Sponsors

Square ā€“ Develop on the platform that sellers trust. There is a massive opportunity for developers to support Square sellers by building apps for todayā€™s business needs. Learn more at changelog.com/square to dive into the docs, APIs, SDKs and to create your Square Developer account ā€” tell them Changelog sent you.

Vercel ā€“ Vercel combines the best developer experience with an obsessive focus on end-user performance. Our platform enables frontend teams to do their best work. Unlock a better frontend workflow today.

Lolo Code ā€“ If youā€™re familiar with building severless apps, think of Lolo Code as your backend with a visual editor that lets you think and build at the same time. All this without having to provision or manage servers. Use the visual editor to build your app, connect nodes, and add any npm libraries you need. You can even write your own integrations. This makes Lolo Code very Zapier-ish, but for devs. Try it free today with no credit card required at lolo.co/jsparty

Sentry ā€“ Working code means happy customers. Thatā€™s exactly why teams choose Sentry. From error tracking to performance monitoring, Sentry helps teams see what actually matters, resolve problems quicker, and learn continuously about their applications - from the frontend to the backend. Use the code CHANGELOG and get the team plan free for three months.

Notes & Links

šŸ“ Edit Notes

Chapters

1 00:00 Opener 00:39
2 00:39 Sponsor: Square 00:54
3 01:33 It's party time, y'all! 00:50
4 02:23 Welcoming Tom to the show 02:49
5 05:12 Intro to Project Fugu 03:42
6 08:54 Why was Fugu created? 03:01
7 11:54 The web vs app stores 03:31
8 15:25 What happened to PWAs? 06:04
9 21:42 Sponsor: Vercel 01:35
10 23:29 Photoshop on the web 05:00
11 28:30 The standards adoption process 02:57
12 31:26 APIs that enable Photoshop on the web 03:38
13 35:05 Explaining origin trials 02:36
14 37:40 nicksbelovedtypescript.org 01:22
15 39:02 Back to the file system API(s) 03:27
16 42:42 Sponsor: Lolo Code 01:03
17 43:45 Sponsor: Sentry 00:39
18 44:37 Fugu API tracker & showcase 01:45
19 46:22 The Clipper API 05:33
20 51:55 A quick tangent on file handling 02:24
21 54:18 Wrapping up 00:53
22 55:18 Outro 01:08

Transcript

šŸ“ Edit Transcript

Changelog

Play the audio to listen along while you enjoy the transcript. šŸŽ§

Hello, JS Party listeners. Weā€™re back this week, talking all about the web yet againā€¦ Surprise, I knowā€¦ Weā€™ve got a really special guest with us today. Before we introduce our guest, on the panel with me today is my - I would say almost regular co-pilot these days, Nickā€¦ Oh, well, I just ā€“ well, anywaysā€¦ I feel I always spoil introductions. So Nick, welcome! Welcome on the show.

Hoy-hoy! Good to see, Amal.

Good to see you. This is the first show that Iā€™m back in my own home. Iā€™ve only been home for 12 hours, and Iā€™ve already had A/V issues, and internet issues, and all the things; all the things that you can imagine, all the tech issues have concentrated into this show. But anyways, regardless, the show started now, and weā€™re super-excited for our amazing guestā€¦

Oh, yeah.

Yeah, mega-amazing. Heā€™s so amazing heā€™s got a Ph.D. Tom Steiner, Senior Developer, Relations Engineer, who works on the Chrome team at Google. Welcome, Tom.

Thank you very much. Hello.

Hello. Tom, why donā€™t you tell us a little bit about yourself, for folks who might not be familiar?

Well, my name is Tom, I work on the Chrome team, but what I like most about my job is that I work on the Chrome team, but actually Iā€™m not that much advocating for Chrome. I do as well, of course, but Iā€™m very much also advocating for the web. So the web is the product that I advocate for, and specifically, more specifically, Project Fugu these days, which is all about making new use cases possible on the web by adding new APIs to browsers. And browsers, by definition, is plural, not just one browser.

that sounds like an amazing job. I have to say, for many years - Iā€™ve been following the Chrome team for almost a decade now, and itā€™s been amazing to see kind ofā€¦ I think that team specifically set the standard for how to do developer engagement, and how to do developer advocacy, and how to do developer education well. I always think of the Chrome Dev developer relations team as a whole, and thereā€™s different roles within the developer relations teamā€¦ Thereā€™s folks that work on engineering projects, like Tom, thereā€™s folks that are really more focused on advocacyā€¦ Iā€™ve always thought about that team as best-in-class. I do have to admit though, just to be fair - Tom, I donā€™t know how often you listen to this show, but Iā€™m not sure we have a ton of Chrome users anymore on the panel. I use a Chromium flavor as my primary browser these days, Brave. I also use Firefoxā€¦ But I think regardless of our usage of Chrome, I think Iā€™m a fan of the Chrome Developer Relations team, and Iā€™m super-excited to have you on the show todayā€¦ So thank you, and welcome, Tom.

Yeah, thanks for having me.

Yeah. So Tom, weā€™ve invited you here today to tell us all about Project Fugu. Weā€™re super, super-excited to finally have this topic on the show. Itā€™s been on my to-do list for a very long time. One of my really good, very good friends, and I would say heā€™s like my frientorā€¦ I donā€™t know if you know what a frientor is, but heā€™s like a friend and a mentor to meā€¦ Alex Russell was the tech lead on this project, and helped really galvanize itā€¦ And heā€™s at Microsoft now. But he was on my list, I was like ā€œAlright, Iā€™ve gotta get Alex to come on the show to talk about Fugu. Iā€™ve gotta get Alexā€¦ā€ And we just never made that happenā€¦ And Iā€™m so glad to finally be talking about this. So can you tell our listeners, what is project Fugu, and why are you working on it? Itā€™s such an important project to the ecosystem, and thereā€™s so much to unpack, soā€¦ Educate us, please, Tom.

[06:00] Yeah. So let me startā€¦ You should definitely get Alex on the show.

Oh, for sure.

Heā€™s still around. Heā€™s still at Microsoft. He still does web stuff, obviouslyā€¦ So very much do make sure that you get him on the show. And I will tell you my view of project Fugu, but obviously, Alex has been around for way more time than I have been, so he will have a different view, so also get his, for sureā€¦ But what I will tell you is, when I started working on the Chrome team, I was looking for my niche. And on the Chrome team back then everyone was doing amazing things, like CSS Houdini, and all the hot topics were being dealt withā€¦ So I looked at ā€œWhat are people doing on the web?ā€ I noticed this tendency where more and more people were building amazing applications on the web. And in many cases, these applications were lacking certain APIs. So there were gaps in the platform that the platform just didnā€™t coverā€¦ And Fugu was his effort that people around Alex - but thereā€™s a lot more people behind this - have started. And they started by filling all those gaps.

So initially, there was ChromeOS as an app platform, and they had a backlog of all these APIs that they [unintelligible 00:07:11.07] of sorts to be on the open web. So not proprietary ChromeOS application platform, API Interfaces, whatever, but like proper, on a standards track, developed in the open APIs that people could agree on universally, where browsers would say, ā€œHey, this is a good idea. Letā€™s do it that wayā€, but browsers also could say, ā€œYeah, weā€™re in general okay with this API, but we donā€™t like the shape, so letā€™s refine it a little bit.ā€ Or - and this also did happen, where browsers would be like ā€œOh, thatā€™s a terrible idea. We donā€™t want to have this on the web platform.ā€

So with all these three different options of how APIs could play out, Fugu was or is still this effort of making stuff and use cases happen on the web. And in general, what we do see is thereā€™s developer enthusiasm, thereā€™s also skepticism when it comes to ā€œCan I really use this on all browsers? What is the right strategy for building applications today?ā€ And yeah, I guess we can discuss some of these questions during the show.

Yeah, oh my God, thanks for that great summary. Thereā€™s so much to unpack here, and weā€™re gonna get into some of the specifics on specific APIs that Fugu is enabling within a browser thatā€™s already near you, or coming to you, I donā€™t knowā€¦ But I think whatā€™s more important is to discuss why was this project created. And Alex and I have had many chats, and Iā€™m certainly going to have him back on this show, but itā€™s going to probably be a wider discussion about like ā€œHow can we save the web?ā€ Thatā€™s gonna be a three-hour conversation, weā€™re gonna need some alcohol for that, itā€™ll be probably a special edition showā€¦ But focusing this on Fugu, can you tell us a little bit about why this project was created? Can you tell our listeners about why, like what are we actually trying to do here?

So talking to a lot of partners, so customers, companies Google works with, we always got sort of the same message, which was, ā€œWe need apps, we need different apps for different platforms, and we essentially need to build the same app two times, three times, four timesā€, depending on what platforms they needed to cover, or thought they needed to coverā€¦ And most companies donā€™t really like doing that. So thereā€™s a couple of companies who actually have the resources and actually do like building apps for platforms, but most companies, they do it because they think they have to, and theyā€™re trying to avoid this as much as possible. So they try to build cross-platform, so using in the old days something like Cordova, or they would go React Native as a routeā€¦ Flutter these days is becoming very popular for this promise of writing once and then running everywhereā€¦ But yeah, in the end, people also always need a website.

[09:59] So it can be a web application, it can be just a gateway into your application that you then install for Android, or Windows, or macOS, or whateverā€¦ But essentially, yeah, this desire of ā€œWhy not just have one app that we can use universally?ā€ And thereā€™s a couple of companies who have started doing that, and most companies are not there yetā€¦ But yeah, we see a lot of companies interested in at least reducing the number of platforms that they build upon.

And I would also say COVID has played a huge role in it, because COVID made desktop important again. So between - like, 2015 was the year of mobile, and everyone was looking at mobile web, and we still do, but COVID has sort of helped us roll back a little bit, and look at ā€œHey, desktop is a very important platform, and thereā€™s a lot of applications that people build for the webā€, and they run in the browser, and people on their big 20-inch, or whatever, 27-inch laptop screens ā€“ or not laptop screensā€¦ Desktops screens ā€“

27-ich laptop screens - Iā€™m like, ā€œWe need to meet these people, we need to get them on the showā€¦ Weā€™ve got to talk about backpacksā€¦ We have a lot to talk about with them.ā€

Absolutely. So anyways, big screens on whatever device - they run web apps, in many cases. Looking at my computer right now, thereā€™s a handful of apps, and most of them are really just web-based. Everything else is ā€“ macOS Finder, and some emulation software, but thatā€™s about it. So everything else is just on the web. And they are not websites, they are web applications. So I can create stuff here. We are creating stuff here right now, so this is a web-based podcasting softwareā€¦ So thereā€™s a lot of web applications that we just use, and we live in the web, and this is why Fugu was and is being built.

Yeah, absolutely.

Itā€™s really cool.

So this idea of reducing your stack, right? And JavaScript being a universal stackā€¦ I mean, weā€™ve kind of been trying to do this for, I would say, ever since Node became a thing. We made that shift with our server-side stack, so teams kind of, okay, using one language in the frontend and ā€œbackendā€, right? I say ā€œbackendā€ because for me, what is backend even these days, really? Is it really a backend? Every backend has a frontend and a backend? I donā€™t know, at least the backends that Iā€™ve worked onā€¦ So unless youā€™re working on the data layerā€¦ I donā€™t even know, really. Is it the backend? I have no idea.

So teams are trying to say, ā€œOkay, we canā€™t write iOS apps, Android apps, and JavaScript apps. We canā€™t staff them, we canā€™t maintain them, itā€™s too muchā€, and itā€™s I agree; itā€™s a huge permutation to manage, especially across Android. Creating a quality Android application is like an act of God, in some cases, really, because thereā€™s tons of permutations that you need to support.

So can you tell us a little bit about this play for the web? Weā€™ve got React Native thatā€™s been in this space, youā€™ve mentioned Flutterā€¦ It sounds like weā€™re really trying to save the web from being engulfed by native applications, and for me, thatā€™s kind of how Iā€™ve always read this ā€“ this is how Iā€™ve smelled this project. Can you speak to that a little bit, Tom?

So I think on mobile specifically people still have the tendency to look for apps on the app stores. So on the Play Store, on the App Storeā€¦ And even on desktop, Apple tries to very hard make people look for apps on the App Store first Windows System Microsoft Storeā€¦ So I think stores and the way as a distribution mechanism stores work is still very much important. And looking at what we can do about that - so stores have advantages, you can get your applications there, you can get reviews there, people can get a feeling ā€œIs this application addressing the needs I have for it?ā€ If you have an image editing application, what do others say about this application? Can it deal with (whatever) JPEGs? Does it struggle with large files? ā€¦and so on.

[14:09] So these kinds of things - people believe in reviews. So thereā€™s a lot of advantages of having apps on the app stores. And one trend that weā€™ve seen as people want to get their progressive web apps onto app stores - you may have heard of PWABuilder, that is a community project run by Microsoft folks, that helps with that. So native apps and progressive web apps on app stores can coexist, and they do coexist pretty well, I would say.

On the other hand, we still see a lot of lower-quality apps, that are essentially very much still web applications, websites, sometimes in the worst case that are poorly wrappedā€¦ And people leave reviews about those and say, ā€œThis is really just a website. Itā€™s not a great app.ā€

I think if you go this route and say ā€œI want to get my application alongside native applications, listed in a storeā€, you need to make the effort of really making an app blend in, and make it feel an app, and not like a wrapped website. So this is my sort of rambling answer to your open question.

No, that was a really good point, too. The app store is ā€“ itā€™s got its hooks, right? Itā€™s a well-paved cow path for users, right? Itā€™s the thing that they know, itā€™s the thing that theyā€™re familiar with. And for me, I have this burning question of ā€œWhat happened to PWAs?ā€ In the sense that I thought PWAs were supposed to save us from this hellhole of native applications. And I say that because, quite frankly, Iā€™m very team pro open web. Native apps are maybe slightly better for user experience, but not really great for the health of the web, specifically user privacy as well; user privacy, user data - thereā€™s all kinds of stuff thatā€™s not so great, So browsers being this sandboxed environment, and more kind of a trusted environment, always kind of a safer place to browse.

With that, thatā€™s something I was thinking about, too. Itā€™s great adding these APIs and capabilities to the browser, but just going towards what you said, Amal, the browser is a trusted place, but itā€™s also a place I heavily distrust, and itā€™s very obvious in the amount of content blockers, ad blockers, things that I add to my browser to make it more of a safe place; get rid of those cookie banners, things that. And those things - it adds a distrust, and then the first time I try and visit something that I would just obviously trust on the web, but itā€™s broken because of my overabundance of trying to block everything, I immediately start distrusting the platform more.

I donā€™t know where Iā€™m going with thatā€¦ Just to call that out is a problem. Itā€™s a trusted place, but because thereā€™s so much distrust among the content there, it sours everything, because Iā€™m overly cautious about the dangerous stuff. And it can break my experience for the good stuff.

Yeah, I can maybe speak to that. So whatever happened to progressive web apps? So first, maybe, Amal, to discuss this point that you brought upā€¦ Do you remember AJAX? All uppercase AJAX. A little bit later, it became first letter uppercase, and the rest lowercase, so more of a regular wordā€¦

You mean weā€™re not doing asynchronous JavaScript in XML? [laughter]

Here we go.

What?! Here you are, busting dreams and facts, Tom and Nick Nisiā€¦ Some naughtyā€¦

So yeah, at some point no one talked about Ajax anymore. It just had become the way we build single-page applications. And the same goes with HTML 5. Do you remember HTML 5 and CSS 3? All these logos that we put on our conference slidesā€¦ Nowadays everyone does GTML 5 apps. Or do we do HTML 6 apps now? Or I donā€™t knowā€¦ So we just stopped talking about thisā€¦

I still remember my very long DOCTYPEā€¦

[18:03] [laughs] And a little bit the same is happening with - if I could just continue with thisā€¦ The same is happening with progressive web apps these days. So people build progressive web apps of sorts, but Iā€™m like ā€œIs it a P, progressive web app, if itā€™s, not installable? Or is it just a web app?ā€ What about the just? Do we need the just? Is it a web app? Is it just an app? Is it good?

If you go to GitHub and you browse any kind of random repository, and you press the period key on your keyboard, the dot, it all just of a sudden launches a full-blown code editor? Is that a progressive web app, or is that a web app, or is that an app, or whatever is it? In many cases, itā€™s not distinguishable from VS Code as a native application that you download and run, but itā€™s an Electron app. Itā€™s not distinguishable, in many cases.

So these kinds of experiences where we just become used to using apps in a browser without really realizing that weā€™re using a progressive web app, or a web app, or an appā€¦ So what Iā€™m getting to is maybe progressive web apps have just succeeded because weā€™ve built them and donā€™t really think about it anymore. So thatā€™s that.

Nick, for your point about cookie banners and all these horrible thingsā€¦ I guess what weā€™re seeing as people just being really careful on the web - because in many cases, all these cookie banners, or some of these cookie banners wouldnā€™t have to be. People just put them up just in case, to not get sued. Native applications, I think - and Iā€™m not a lawyer, but in many cases, there would have to be some sort of GDPR notices as well, but people just donā€™t put them up, because itā€™s just so hard to inspect what a native app does compared to what a web application does. So a lot of the things that native applications do, and spy on us, and sync periodically with whatever server while weā€™re sleeping, and these kinds of horrible things - theyā€™re happening, but we donā€™t really see whatā€™s going on there.

Thatā€™s a great point.

Yeah. So on the web, people can inspect and can see whatā€™s going on, andā€¦ Itā€™s just so easy to get in this unfair comparison between the two platforms. Iā€™m a web person, so Iā€™m very much biased, but at the same time, Iā€™m trying to understand what is going on on native as well, and theyā€™re doing horrible things as well. And the web does horrible thingsā€¦

Oh, for sure. The web is not innocent, by all means. I agree.

Yeah. Itā€™s more out of sight for the native ones, but youā€™re absolutely correct.

It is, for sure, yeah. So we are here to fix both, hopefully. So if you talk to people from the Android dev del team, they will tell you ā€œLook, a lot of Android apps do horrible thingsā€, and Android as a platform is also getting stricter about what apps can do in the background, and so onā€¦ So thereā€™s some development happening there as well. Google has the privacy sandbox for the web, but now thereā€™s a privacy sandbox for Android as well.

So what Iā€™m getting at - this is a general movement. Apps in general becoming more privacy-aware, privacy-sensitive, information not being shared in the background, and so onā€¦ So thereā€™s a lot of development going on there.

Yeah, thatā€™s a great answer, Tom, and a very, Iā€™d say, scientific and diplomatic answer as well. I like your kind of ā€œInnocent until proven guiltyā€ stance for examining these types of issues. Iā€™m just kind of like ā€œGuilty!ā€ But yeah, so we have so much to get into, and so much to unpack, so weā€™ll be right back after these short messages.

Alright, Tom, weā€™re back. So exciting to be talking about this topic, finallyā€¦ Because for me, Fugu feels like the browsers kind of breaking free. Itā€™s got legs now; this kind of traditional idea of the sandbox browser that canā€™t access file system, and canā€™t talk to the Bluetooth, and ā€œOh, we donā€™t knowā€“ā€ Connecting with sensors, and all these little things, you have to go through hoops, andā€¦ Right? This idea of your browser being limited, and consequently, JavaScript apps also being limited - this is just kind of shattered now, because Fugu is just like ā€œNope, the web can do that. Nope, the web can do that. Oh, the web can also do that.ā€ Right? So can you tell us about ā€“ letā€™s just walk through what you think are the shining stars of the Fugu project, especially the ones that are gaining a lot of traction. I would love to start with your first hit, Tom. Whatā€™s your number one?

So when it comes to apps, definitely my number one app example is still Photoshop on the web. They launched it, they made it made it happen. What I want to call out about this - in the early days, when this really just launched, people were like ā€œOh wow, they brought Photoshop to the web.ā€ And then of course, haters were quick to point out ā€œYeah, you open this in Firefox and in Safari and it doesnā€™t work at all.ā€ But - and this is where the but comes in - Adobe from the very start said ā€œLook, weā€™re working super-hard with Mozilla and with Apple to make Photoshop on the web happen on the web, and not just on Chrome.ā€ They put out the version on Chrome first, because Chrome was the first to implement the APIs and get them ready, but if you went to the Photoshop website, on Safari, on day one, you were just greeted with this message of ā€œHey, youā€™re not using a supported browser.ā€ The classic Chrome-only thing. But if you do the same today, you can open files in read-only mode, and you can see the application Chrome is there, in the sense of the UI is there; itā€™s just read only at the moment. But the message that they say at top is ā€œThis is not working yet. Weā€™re working with Apple and Mozilla to get these APIs into the browsers.ā€

And thereā€™s a double negation in this tweet that Shawn [unintelligible 00:25:40.07] sent. So Shawn works on Adobe, and he said ā€œThereā€™s not a single API that weā€™re working on that is not on the standards track.ā€ So what heā€™s saying is every API they need in their application is on a standards track. Itā€™s just not implemented yet. So I think this is very important. If you are a big company, like Adobe, of course you have a lot of say when it comes to talking to Apple and talking to Mozilla, but I think the same can be sort of leveraged for companies of all sizes. If youā€™re not a big company Adobe, you can still, in some; if youā€™re a lot of small companies and you talk to these other vendors, and you say, ā€œHey, we have this application itā€™s working fine in Chrome, and we donā€™t want to tell our Firefox users to download Chrome; we want them to continue using their browser of choice, but weā€™re needing this and this and this API. Itā€™s not implemented yet, or maybe itā€™s buggy, or whateverā€¦ā€

[26:35] So talking to browser vendors, and bringing forward use cases that you want to cover, and making sure that you communicate well why these APIs are needed can be a recipe for success. And weā€™ve seen it with Adobe, which acknowledgedly, as I said, is a big company name. But so, in the sum, a lot of smaller companies can add up and they can get the same pressure on other browser vendors to implement APIs.

Iā€™m not saying this is a recipe for universal success. Weā€™ve very clearly heard for example Apple say (I donā€™t know) WebUSB is an API that they probably wonā€™t support, ever. Maybe they will in the future. Who knows? If we bring forward the right use cases, maybe thereā€™s a change in how security on the web is handled.

Thereā€™s a lot of debate also when it comes to APIs on the web. What is the right way to ask for permission? Do people understand when you ask for permission? Is there a way to communicate really all the ways an API could be abused potentially in a small permission prompt? What is the right way there? And I think this is also a way for browser vendors to just innovate and say ā€œWe are Firefox, and weā€™re going [unintelligible 00:27:40.22] way that is different from all other vendors.ā€

For example, very recently Firefox supported or started supporting Web MIDI. The way they do this now, if I understand the concept correctly, is they sort of build an ad-hoc extension that they can then block and say ā€œIf we see that the API is being abused, we have an escape hatchā€, and they can just sort of I think revoke the ad-hoc extension that was created. I might be misrepresenting the idea, butā€¦ The core idea is they innovate on the way the permission prompt is being shown by introducing a different vendor-specific way of doing so. And yeah, maybe this is the recipe for success for getting more obscure, and maybe more powerful APIs onto the platforms.

You said you worked through standardsā€¦ I was curious, is that WATWG, or what kind of standards process do these APIs go through? Or does it vary?

We start in the WICG, the Web Incubator Community Group. But then once we have incubated, of course, the final objective is to move on to the W3C, the full standards track, where we go through all the different stagesā€¦ And I just prepared a couple of APIs, and Iā€™m reading them out now, that started on the WICG, but that are now on the W3C. So thereā€™s the Web Share API, thereā€™s Web Transport, thereā€™s the Filesystem Living Standard, thereā€™s Web Codecs, thereā€™s Clipboard API and Events, thereā€™s the Multi-screen Window Placement APIā€¦ So these are just a couple of examples of APIs that started in the WICG and that then migrated over to the W3C.

Yeah, thatā€™s a great question, Nickā€¦ Because you really said a lot there, Tom, and I just kind of want to break things down for our listeners. Thereā€™s this idea of standards adoption. We can create a standard, we can agree on how to implement thing A, but if thing A doesnā€™t get implemented across all browsers, it is kind of is useless to developers, right? To some degree. And so Apple has been ā€“ and I know you probably canā€™t say this, or maybe you can, I donā€™t know, but either wayā€¦ Iā€™ll say it; I donā€™t work for Google, right? So Apple has been a tough player to bring to the table. Theyā€™ve really been a hard player to get to adopt a lot of the APIs that supported progressive web apps, and all those other things. And they finally are there. And I think the story that youā€™re telling, of Adobe successfully launching Photoshop, that works really almost like a progressive web app, because itā€™s like, it works more fully and richly on Chrome and/or browsers that have adopted the APIs that they supportā€¦

[30:19] But this idea of actually having a huge vendor and a huge player like Adobe say ā€œHey, weā€™ve implemented this, and weā€™re going to just tell our users weā€™re working with Apple to get this working in Safariā€, it really flips the script. I love this ā€œLetā€™s be transparent, and letā€™s just be honest, and letā€™s implement stuff, and letā€™s tell users ā€“ itā€™s not that this doesnā€™t work, itā€™s just that hey, this hasnā€™t been implemented in Safari yet.ā€ If I was a business where my name was on a banner for a very popular website, saying that like ā€œOh, this doesnā€™t work on your siteā€, I would be like ā€œWhat? What?! Noā€¦ā€

So I think thatā€™s genius, so really, kudos to the Adobe team and the product team that was able to make bold bets and bet on the web, reallyā€¦ Because really, thatā€™s what the product team is doing. Theyā€™re like ā€œOkay, weā€™re gonna take a chance. Weā€™re going to be the first.ā€ Nobody ever wants to be the first. Everybody ā€“ trust me, there are so many 10-year overnight successes, right? Projects like Fugu, that have been in the works for years, and youā€™re finally starting to see the fruits of thisā€¦ So kudos to that team.

So Tom, letā€™s get into what makes Photoshop successful. So what specific APIs are they using that are really showing off the power of these new APIs? So filesystem access is one, I guess, butā€¦ The floor is yours.

Yeah. So in Photoshop, it of course heavily depends on a decades-old codebase that theyā€™ve taken and translated, or transpiled to WebAssembly using Emscripten. And in Emscripten, thereā€™s different ways of making applications happen. For example, in Emscripten they added SIMD support, so that you can have single instruction, multiple data, so essentially get more things done in parallel. They have pthread support, so you can do stuff in background threads, and so on.

So, Emscripten is a very, very big topic, WebAssembly is a very big topicā€¦ Iā€™m not an expert in any of these, but what I have more insight into is the way Photoshop works is they have sort of a big swap file. So in Photoshop, you can open very, very big files, that can reach gigabytes, sometimes. The way this works is they have internally in Photoshop sort of a swap file, where they just randomly can access memory and write data and read data in parallel also, into this file and from this fileā€¦ And this primitive just didnā€™t exist on the web. So what Chrome has built for them and what is now being standardized, and what other vendors like Mozilla and Safari implement now as well is the so-called origin private file system, which allows files to be created that are, as the name suggests, private to the origin, like photoshop.adobe.com, for example, that only live in the context of this application, and they have certain performance guarantees. You can write very performantly into them, and read performantly into themā€¦ You donā€™t have to wait for rights to be confirmed, which makes writing a lot fasterā€¦ So all of these things, if you think web as well, thereā€™s a big problem, if you will; we call it the mark of the web. So if you download a file from the web, before you can execute it or open it, you get this warning which tells you ā€œHey, this has come from the webā€ and so on. Thereā€™s safe browsing checks that need to happen, and of course, all of these checks take time.

Within the OPFS, origin private file system, these checks donā€™t need to happen, because technically, they are files that are not really exposed to the user. So they live inside of ā€“ somewhere hidden in your browser. Technically, they donā€™t even have to be files; they could be implemented as, for example, entries in a database. They just behave and feel to the developer like files. And I think this is the core thing here - the actual storage is an implementation detail that you as a developer donā€™t have to know. And yeah, this makes working with these files so nice, because to you as a developer, it just feels like ā€œHey, Iā€™m opening a file, Iā€™m writing into it, Iā€™m streaming data into itā€, or whatever. So the look and feel is like a file, but then everything else, the other problems with files that you have on the web, you donā€™t have to be concerned about.

[34:29] Yeah, thatā€™s super cool, Tom. I feel like my head is a little, like ā€“ I havenā€™t been following this specific spec, so hearing where it is now, Iā€™m like ā€œWhaat? What are we doing now?ā€ Thatā€™s intense. Also, that sounds powerful. Also, we had Debbie Oā€™Brien on the show last week, we were talking about Playwright and we were pressed on time; I was really trying to avoid tangents, and I would say the same exact situation here - there are so many tangentsā€¦ I feel we could have an entire podcast on all the things Amal did not say. I will take a really quick tangent for our listeners, because I think this one is an important context I want to set for yā€™all. So thereā€™s this concept of origin trials within Chrome, and itā€™s like a fantastic concept. Tom, could you explain what that is to our listeners?

So origin trials allows you to test new APIs with actual users. Why do we have this? So in the beginning, you might remember that the early WebKit-something prefixed APIs. The idea there was we agree this is not an API that is standardized yet, itā€™s an early test maybeā€¦ So vendors thought, ā€œWell, itā€™s a clever idea to just prefix these APIs.ā€ The problem then is, because these features were in many cases very attractive, developers started using it, and depending on itā€¦ Which then ironically caused vendors like Firefox to implement things that are prefixed with WebKit, just to be supporting all these use cases that developers had built. So learning from this, weā€™re baking stuff into the web platform that then might just become baked in forever, and with no way to get back, origin trials was introduced as a safe way of playing with things that are in incubation still. So you could then just include a meta tag on your site, and if you have this meta tag on the site, the browser would detect it, and then sort of unlock the API in question, so itā€™s encoded in the meta tag token somehow what API is visible then, if you have the token. And each token is also limited to a certain amount of time. So first, you need to renew your token, so that you really make sure that your web application stays up to date. So maybe thereā€™s been changes in the API, so if you change your token, you also need to change your code, as it will stop workingā€¦ But then in the end, of course, itā€™s also important to not bake these maybe not mature yet APIs into the browser.

So every origin trail is also just time-boxed, where we say ā€œItā€™s supported from milestone X to milestone Y.ā€ And once milestone Y is over, either we just say, ā€œOkay, the feature didnā€™t work out. Itā€™s something that will be maybe iterated again uponā€, or we say ā€œItā€™s been a success, we can use it now.ā€

More recently, on the Chrome team we introduced the concept of so-called gapless origin trials, which means the origin trial would go straight into a feature that you could use in production. So this allows really to, if we have agreement on how an API shape should look like, that then the continuation would be there. You could start from the early days, go along the origin trial, and then take your user straight into prod. So thatā€™s the idea here.

Yeah, thank you for that awesome summary. I didnā€™t know that we had gapless now. Thatā€™s great. So the idea is we have this website called ā€œNickā€™s beloved typescript.org.ā€ Somebody should buy that domain if it doesnā€™t exist, and gift it to Nickā€¦ So ā€œNickā€™s beloved typescript.orgā€ wants to use some API that isnā€™t standardized yet, but thatā€™s implemented in Chrome specificallyā€¦ So itā€™s like, do we put a set of instructions thatā€™s like ā€œUser, go to your settings. Enable this flagā€? No, weā€™re not going to do that. Right? So how do we just automagically make it work for our users? So you go to Chrome, you sign up for an origin trial, you give them your domain, and you get approved, and then it automatically just works for your users. Itā€™s basically a feature flag, but for web APIs thatā€™s specific to Chrome. And itā€™s awesome.

[38:28] That you control, rather than the user, which is really ā€“

Exactly. That you control, rather than the user. Exactly. For better or worse, right? So make good decisions for your usersā€¦

And Iā€™m gonna just call out quickly, Firefox has the same concept also.

Oh, of course. Thatā€™s awesome.

Very recently Iā€™ve participated in the OffscreenCanvas origin trial, and Edge - so Edge, the Chromium-based Edge, they also use origin trials now. So itā€™s a concept that I think people like in general, because of all the advantages that I just listedā€¦ So yeah.

Yeah. It makes sense. So now that we know what origin trials are, letā€™s go back to the File System API. So the File System API was not initially on the standards track. It is now, obviously; itā€™s been for a whileā€¦ But were origin trials kind of part of the story here for being able to actually experiment in the wild on this feature? Iā€™m curious where the teams like the Photoshop team use an origin trial initiallyā€¦? Or are they using origin trials? These are all open questions, Tom. I donā€™t know the answers. Tell me.

So we need to be a little bit careful hereā€¦ So when you say File System API - there has been definitely more than one.

Okayā€¦

There has been something like the File System Entries API, or something thatā€¦ So there have been several attempts, also add standardizing; not just proprietary stuff, but really standardizing file system access on the webā€¦

Oh, wowā€¦

What we can talk about specifically now in the context of Adobe - this is the origin private file system part, and more specifically there, the access handles; so the thing that gives you access to these files that are private to the origin. And there is, or has been an origin trial that Adobe has been taking part of. So they went through this entire process of breaking changes, that from one origin trial version to the other they needed to implement, and so on. So now, the origin trial is ā€“ I think itā€™s over, butā€¦ Yeah, definitely, thereā€™s the API that is agreed on universally now, itā€™s being standardized in a living standardā€¦ Itā€™s called the File System; I think is just a standard name.

Important to say is, if you talk about file system access, thereā€™s the File System Access API, that has the so-called picker methods: showOpenFilePicker(), showSaveFilePicker(), and so on. So this is not that. So the file system part that is being standardized now is first how do we work with files in general, as like the file methods - to rename a file, to move a file, to delete a file, to create a file in the first place, to write to a file, and so on. So this is being standardized. The way you can create a file is in the origin private file system, or by using the picker methods. And the picker methods - those are more powerful APIs, and this is also where we still have some disagreement on - like I said, it allows you to open files from your local file system. And as I mentioned before, on the local file system, the user visible files, you have all these problems, like what about ransomware attacks where someone just gets access to your whatever projects folder, and then encrypts everything, so they want to get some ransom from you to decrypt again your files, and so on. So itā€™s a little bit more interesting when it comes to those picker methods. The other part, this is whether you have universal browser agreement on.

Yeah, and thank you. And like I said, I have not been following this spec, and itā€™s so interesting to learn from you that itā€™s diverged into all of these different thingsā€¦ Which makes sense, right? Thereā€™s been limitations that have been found. So instead of throwing the baby out with the bathwater, letā€™s focus on this one narrow part of the spec, and letā€™s get this one narrow part right. And I love that approach. This is why standards work is ā€“ I think Iā€™ve said the word ā€œGodā€™s workā€ already in this podcast. Iā€™m gonna say it again. It is Godā€™s work. Itā€™s amazing. Hard work. So yeah, so lots to continue to unpackā€¦ Weā€™ll be right back, everyone.

So first off, I love the naming Project Fugu. It reminds me of that Simpsons episode, the Fugu Fishā€¦ Fan Fugu Tastic! Is there a place, like a consolidated place where we can find out more about what all is going down with Project Fugu, and the different APIs that are being experimented with, and the status of them, and all of that?

Yeah, sure. We have two places. So thereā€™s the Fugu API Tracker, that allows people to just see in the open what APIs are we working on, what is in our pipeline, what features are we implementing, and just working on. And there is the Fugu API Showcase, where we show people who have been building applications, and you can go and see, just get inspiration what people have been building with these APIs; you can filter by API, and you can see ā€œI want to get a list of all the apps people have built with, I donā€™t know, the Local Font Access API, or the File System Access APIā€, or whatever. So you can just filter by API, you can search by app name, if you know a specific app and you want to see what APIs is a specific app usingā€¦ So the Showcase is really your destination for this kind of question.

So if you want to just get inspiration, just purely browsing the API Showcase, launching the applications, testing, and playing with them, it definitely helps. In many cases, thereā€™s even source code links. So you can go there and see ā€œHow did they do it? How did they implement Local Font Access in whatever application. So you can see and get a feel for the code as well.

I was just going to call that out. I was looking at the showcase and seeing the links to not only try the apps, but also view the source code. Thatā€™s so helpful in being able to mimic what theyā€™re doing, or get an idea of how to approach using the API, which is so, so helpful.

Very much so, yeah. I think weā€™ve maybe got time for one more API. I feel every single API really requires its own dedicated show. So this is really just kind of a brief overview, ladies and gentlemen, a brief overview. Very brief; brief with a small b. So Tom, whatā€™s another favorite API of yours that youā€™re like ā€œThis one should be itā€? Regardless of what stage itā€™s in. Letā€™s just take the shackles off the standards process a little bit. What are you most excited about?

You know what, Iā€™m going to cheat. Iā€™m going to mention two.

Okay, cheat. Give us two.

So first, I want to mention the clipboard API.

Oh, yes.

The way working with a clipboard works is very interesting actually, because if you think about clipboard interactions, you can go to the macOS Finder and copy a file and paste it. So then you paste a file. You can go into Photoshop and select everything and copy it, and then you have an image on your clipboard, and you can paste that image file, in the sense of you paste the image data.

If you think ā€œIā€™m going to a website and selecting everythingā€, and then you paste it into letā€™s say Word, and you will see that you have pasted sort of HTML that then magically got translated into whatever word things should be the best representation of this HTML contentā€¦ So preserving, for example, the semantic information that something is a headline, headline one or so, would be translated into heading level one in Word. But then if you press this magic keyboard shortcut, which is Command+Shift+V on a Mac, you will paste text only. So youā€™re sort of losing this information, and just pasting the text.

[47:59] This is pretty powerful as a concept, because with a clipboard API, you can do the same thing now. You can put different representations of the same content onto the clipboard. So if youā€™re building an application that edits SVG files, for example, you can put the image data on the clipboard, but at the same time, you can also put the source code, because itā€™s XML-based, so itā€™s all readable; you can put the source code on the clipboard, and then depending on the application context, if you paste into VS Code, you can paste the source code, and if you paste into something Adobe Illustrator, you can paste the image data; you can do the same thing there on the clipboard. And the way it works is super, super-simple. If you look at the code, itā€™s just essentially creating different blocks of different MIME types, and putting them on to the clipboard.

So thatā€™s my first API that I really, really, really love. The second one is File Handling. So it allows registered applications and progressive web applications, applications, web applications, whatever, to register themselves as a file handlerā€¦ Which means if you have an example application that deals with .example files, you can say, ā€œHey, my PWA can become the file handler for .example files.ā€ Which means if people have a .example file in their file explorer, Windows Explorer, or macOS finder, or whatever, they can just double click, and the PWA will open. Very, very powerful concept. And what I like most about it is it feels so natural. So once youā€™ve gotten used to it, you donā€™t think anymore, ā€œIs it a .exe file and Word opens? Or is it a .example file and the example app opens?ā€ And if itā€™s well made, you can, in many cases, not even spot a difference between a native application and a web application. Sort of what we said in the beginning, and weā€™re sort of back to thereā€¦ If you build something that is well integrated into the operating system, people think of your applications like apps, and thatā€™s what you want to be at. So thatā€™s a point you want to be at.

People donā€™t think, or should not have to think about ā€œHow is something implemented?ā€ Especially not users. They should just be using your app and be like ā€œOh, wow, this is so cool for (I donā€™t know) editing my video files, or adding my .example filesā€, or whatever.

So I think this is where you want to get at. People should be using your app without really thinking itā€™s a web app. Itā€™s just something that fulfills their use cases, covers their use cases well.

Thatā€™s so important too, because youā€™re covering these minute things that users really wouldnā€™t think about, right? And probably when youā€™re implementing this, you donā€™t think about it too much. But when you click on something, you want it to open in the app that you would expect. And that just gives it so much more of a native feel that it starts to become indistinguishable. So bravo on focusing on these little things that really add up to the expected user experience, which is so important for adoption.

Itā€™s that polish, right? And itā€™s all this invisible work, like youā€™re saying, Nickā€¦ Itā€™s all the stuff that you ā€“ this is how you think it should work, but actually, in order to get it to work in the way that you think it should work, thereā€™s actually a lot of gaps that needs to be filled. Thatā€™s just not the way it was designed to work, right? And itā€™s this example, Tom, that you shared, of copy-pasting text with header texts, and things with bullets, and just being able to copy-paste that seamlessly throughout different applications, and Windows, and whatever elseā€¦ You think that that should just work, right? Like, ā€œOh, what do you mean? Text is text. Isnā€™t there a standard?ā€ No, itā€™s not. Actually, text is really, really complicated, and blah, blah, blah, blah, blah.

Soā€¦ Amazing to hear that. I think thatā€™s also to highlight ā€“ going back to Nickā€™s point, this idea of ā€œIt should just work this wayā€, I think thatā€™s one of the reasons why standards take as long as they do. People donā€™t realize just how things donā€™t actually work the way you think they do, basically. Iā€™m glad that weā€™re doing that hard work.

[51:54] This may be is a nice tangent as well, a very quick tangentā€¦ File handling, as I said, is for installable applications that they can register themselves to become a file handler. But at the same time, if a browser doesnā€™t have a concept of installation to begin with, so for example, Safari, for example, Firefox - they donā€™t have proper installation on desktop. So on mobile, you can add applications to the homescreen in Safariā€¦ But if you donā€™t have this concept of installation, if you donā€™t have this concept of file handling, you can still build the same amazing Progressive Web App, and people can still use your application in a tab, and be very happy users. And if you have a browser that supports installation, if you have a browser that supports file handling, you can add these additional featuresā€¦ But in many cases - yeah, they are a progressive enhancement; they will make the experience better, but you can still live and use the application without. And I think this is a very important concept to take home with - progressive enhancement, in many full API cases, is very important, and once a browser then eventually maybe starts supporting a certain feature, your app will just magically work.

Weā€™ve been using progressive enhancement for a long, long time, but the idea is still fresh, and maybe even fresher than ever, thinking just about something like WebCodecs, that allows you to do video and image and audio coding codecs stuff in the browser as a progressive enhancement; you can do it as a fallback on the server, and if you have a browser that is capable of WebCodecs, you can just do it on the client. So I think this is where we can go to.

Thatā€™s amazing, Tom. And itā€™s very hopeful to hear about all of the work thatā€™s happening, and all of the supercharged superpowers that are going to continue evolving, and continue being added to the web. Itā€™s really ā€“ as somebody who deeply cares about this platform, I could not be more delighted to learn about all of this. I would say that itā€™d be great ā€“ I donā€™t know if the Fugu showcase site has a list of ā€œHereā€™s all the things a native app can do, and hereā€™s all the things you still canā€™t do in the on the web, and hereā€™s where we are, hereā€™s how weā€™re doing. Hereā€™s how we stack up.ā€ I donā€™t know if thereā€™s a stack rack comparisonā€¦ But weā€™ll have to discuss that maybe with Alex on another show, because thatā€™s all weā€™ve got time for todayā€¦ But itā€™s been an absolute pleasure to have you on the show, Tom. Thank you so much for enlightening us, and I hope our listeners are inspired by all the really great things that are coming to the web, that are on the web in some browsers alreadyā€¦ Itā€™s really been very exciting to learn, so thank you, Tom.

Yeah, thank you.

Thanks for having me. Itā€™s been an honor.

Yeah. Been an honor. So where can folks connect with you, and learn more about you, and all of that jazz?

Iā€™m tomayac on most places on the internet, so you can find me on Twitter still, but also on Mastodon Iā€™m toot.cafe/@tomayac if you want to reach out thereā€¦ But yeah, if you just search for Thomas Steiner on Google, you will find me. Iā€™m pretty searchable on the web.

Yeah, I would expect nothing less than for you to be googleable. [laughter] Alright, everyone, itā€™s been an awesome show. Thank you. Weā€™ll be back next week. Take care, everyone. Bye!

Thank you very much. Bye-bye!

Changelog

Our transcripts are open source on GitHub. Improvements are welcome. šŸ’š

Player art
  0:00 / 0:00