Web development's lost decade

Hi, JS Party listeners! Amal Hussein here, doing a solo show with the one and only Alex Russell. Super-excited to have Alex on the show here today, because Alex has been causing a little more trouble than usual on the internet these days… We’re here to talk about a really important article that he put out into the universe recently called “The market for lemons.” And I say important here, asterisk, because it’s important in the sense that I think there’s an important message, but it’s controversial, and a whole lot of things, and we’re gonna get into unpacking all of that, and more. But before we get into anything, hello, Alex. Welcome. Finally, on the show.

Hey, howdy. Thanks for having me. It’s very kind, especially on a week like this… [laughter]

Yeah, I’m feeling a little under the weather for those… So just please forgive me for any brain farts I may have during this conversation. But thank you again for coming on the show, Alex. We’re huge – I mean, I’m a huge fan of your work. I’ve been friends with you for many years, and have really appreciated how you’ve always kind of advocated for the user. I think you’re kind of a complicated person, quote-unquote, in the sense that I think you’re – I think your passion for the user, and I think passion for kind of web performance and all those good things, I think… You know, they can get animated. I think you can be a lot on the internet.

Spicy.

Spicy. There we go. That’s the word. Yeah. So that being said, I think what I’m excited about doing today is getting to, I think, unpack something that you’ve been really spicy about recently… And I think give it the nuance, and give it the context, and give it the wider picture that I think it really deserves, given the importance of the topic. And so before we kind of dive into that, I’d love to kind of – if you could just share with our listeners a little bit about your background. Who are you, random spicy dude on the internet, and tell us a little bit about how you got to where you are today.

Sure. So my name is Alex Russell. I’m a product manager, I guess, now, on the Edge team at Microsoft, and I’ve been working on browsers for 14 years, something like that. I joined the Chrome team before 1.0. But prior to that, I’d spent something like a decade building extremely large JavaScript applications, often for folks who had real problems. So I was the lead for the Dojo toolkit project, which was the conglomeration of a bunch of other sort of proto DHTML libraries back in the early 2000s… And that became kind of the corporate standard for IBM. We used those tools to build things like business intelligence dashboards, Ajax business intelligence dashboards for folks like Informatica; we built entire PIM suites out of it, helped Sun build stuff… I think some of those systems are still running. For a bunch of years, they were – the AOL email client was based on our technology, calendaring, chat, email, the whole thing.

So around the same time, a lot of my friends in the early kind of proto JavaScript days had been getting hired by Google, and eventually they pulled me in, and so then I spent the next 14 years or so working on browsers, but specifically the web platform. So I helped lead projects that developed various aspects of ES6, things like arrow functions and classes, promises, async and await… We developed at the same time what became web components… We’ve prototyped a bunch of new things in CSS that unfortunately had to wait a much longer time to make it into the world… And then from there, develop PWA’s and service workers, and helped lead the team that did push notifications, and then we did project Fugu, where we’ve been expanding the power of the platform to do things like serial, and HID, and USB, and Bluetooth, and NFC, and access to low-level camera stuff, web transports, web codecs… Oh, and I spent about a decade on TC39 in the middle there, and then ran standards for the Chrome team.

So these days, I spend about half my time working directly with Microsoft’s largest web products. And before that, since 2015 or so, I’ve spent a ton of time doing this kind of incidental consulting with people who are building PWAs. Not because I want to care about performance. In fact, I actually kind of don’t want to think about it; it’s just that whenever we sit down to work with teams, trying to figure out what hurts, that’s always at the very front of the list, whether or not the team knows it. And that’s been extremely depressing, kind of Groundhog Day thing, since about 2015.

I remember we’d sit down before every Google I/O, or for every Chrome Dev Summit, with a set of teams that we wanted to highlight their incredible work on a product, working with these incredibly smart engineers, and they will have built it out of the new hotness in JavaScript land, and it will perform like absolute dog***t. Just not okay. Just absolutely not okay. And so then it turns into this remediation process.

[08:07] These are - and I cannot stress this enough; these are not bad engineers. These are people whose houses that they’ve constructed are extremely flammable, not because they didn’t know how to join the corners, and put up joists, but because the materials that they were sold were not fit for purpose. So a lot of things ended up in their hands…

And so then we would sit down… And a lot of this work turned into sort of more incidental consulting. And again, I can’t stress enough - performance is just the first thing on the way to getting to where we really wanted to go, which is to say, “Okay, have you tried out these cool, new APIs we’re trying to make the web more powerful with? We want to like help you learn about what you need, and then identify the next set of problems.” Because that kind of consulting, that collaboration is how – if you’re a platform maker, you can go around and like you could give everyone a survey and say, “What do you need?”, and you’ll probably get a bunch of answers that are just like “Please make me a faster horse.” And the only way that you really learn about what people actually need for their products is to go sit with them.

So it was an attempt, and it’s a continued attempt to try to learn directly from people building products about what’s hurting most, so that we can go and like add those things to the platform. But this performance thing is just this constant irritant. It’s an ankle-biter at every moment. We just can’t seem to get past it… So here we are. Yet another team at work was extremely let down by a bunch of tools that have been sold to them on scalability, and the only way that those tools scaled was directly downwards, like a lead balloon. And it’s not the team’s fault. They are extremely strong engineers; extremely strong engineers. I can’t stress this enough. I work with great people. I’ve been lucky enough to work with like more than 100 organizations over the last eight years… They’re filled to the gills with good engineers who have been absolutely let down.

There’s so much to unpack right there… So you’re like JavaScript kids, slinging JavaScript, working on some pretty kind of, I would say, foundational and monumental projects on the web platform many decades ago, and I think you’ve really been very influential on the standards community, in all kinds of spaces, and a lot of things that developers use today you had you had some influence in… Which - I’m always amazed at how that list like grows every time I speak with you. It’s like “Wait, hold on… What? You worked on what, Alex?” So I think you’ve kind of shifted into this really interesting role of being in product at Microsoft, but your product really is the web platform. Is that fair to say, that that is your product.

Yeah.

Okay. Super-cool. And also just super-challenging. And so you help work with these teams, and this is something that you’ve been doing – this predates your move to Microsoft… Which, by the way, you’re like the only developer I know who when they change their job, it made mainstream news. There’s like articles of you; it was like “Top web advocate at Google lease to join Microsoft”, or whatever… I was just really amazed at how that actually made mainstream media, but…

Well, all the people who actually do all the work are still at Google, so don’t worry.. Chrome’s not gonna start to rot. [laughter]

Right. Right. That’s true. But essentially, you’ve been working on helping teams really understand looking at stack traces, and understanding how to improve their performance, and this has been something that you’ve really honed in expertise on. I think people not just at Google, but I think people outside of Google have been reaching out to you for years to just say, “Hey, could you help me with this stack trace?”

[11:46] Well, and a quick plug - if you work on a public sector website, if you work in public utilities, and you have web performance questions, my DMs on Mastodon are open. I do performance consulting for public sector entities for free. So just – no one really pays me to do this, aside from my boss, to go and sit with the office team, or whatever… But don’t worry about it, we’ll find time, we’ll make time, we’ll get you the analysis that you need.

Awesome. And be careful what you wish for, but also, that’s cool, that you just –

I’ve seen all. It’s okay. [laughter]

Alright. So kind of fast-forwarding to where we are today - like, you’ve been looking at traces, they’ve been getting progressively worse… And so what kind of – this article titled “The market for lemons”, which we’ll link in the show notes, and everyone should take a read… What got us to kind of things hitting the fan, where you kind of had to just call the frontend framework marketplace a market of lemons?

So I should note that that kind of public advocacy is the thing you do after you’ve exhausted every other potential port of call. So you’ll note that there are some teams that, you know, were shipping way too much scripts, and like actually generating some pretty bad results… So for instance, when I was at Google, the Angular team will tell you that I did not mince words; or when the Polymer team would come to me and say, “Hey, here’s the thing. I put it on the bench”, and I’d be like “Not good enough. Not nearly good enough.”

So the thing about taking the user’s side is you can’t play favorites. You actually just have to play the user as the favorite at every moment, and do it honestly; otherwise, you don’t have any – there’s nothing else to do. You’ll have undermined the whole point of the exercise, right? And the whole point of exercise is to be delivering a web that is effective and competitive, as a counterbalance to closed, owned and operated ecosystems that I think are worse for society.

So I want the web to win. And for the web to win, it has to deliver value; not for just the businesses that make websites, or the people who get paid to make the websites for the businesses, but for the users, right? There has to be a social license to operate for the technology that we deliver. So for many years, I would backchannel the bad results that I was seeing out of various framework concoctions back to the framework vendors. And some of them were extremely frustrated, of course. They didn’t want to see that. But some of them responded constructively, right? So I had a bunch of back and forth with, again, the Angular team, and I can say that I don’t think they appreciated me very much… But eventually, it sunk in, and they made changes, and they built a whole new renderer, and they started to signpost the value of having less scripts… And they started to do something that wasn’t exactly strictly technical. They started talking about what it meant to succeed in the constraints that users actually experienced. And that is, I think, the beginning of a change in our orientation towards what it means to do a good job about engineering… Which is not just to say to botch together and ship your Frankenstein’s monster out the door the minute that it can crawl. It is to try to understand how our products work for users, specifically the marginal user, and then try to build within those constraints. Because product success on almost every axis is determined by whether or not it succeeds for the user.

So they made changes. They started talking about it, they started changing the orientation of their community to start thinking about these things, which initially weren’t part of their narrative. And I have been extremely proud to see the way the Sveltes, and the Preacts, and some of these other frameworks over the years have continued to keep the user front and center, to pitch the thing that actually works.

[15:36] And I am apoplectic that so many teams, despite this wealth of really good options - like, there’s Lit, there’s Fast, there’s Qwik, now that Miško has sort of had a [unintelligible 00:15:43.25] conversion… There are all of these tools that - they’re not the silver bullet, but they are at least narratively not misaligned with what’s mostly going to work for most users, most of the time. And they have not had success in the market, in part because they’ve been crowded out by significantly worse alternatives. And I say significantly worse not just at like the technical level, although that’s also always a problem. I promise you, there’s no reason to have a synthetic event system in your JavaScript library in 2023. You don’t need it. So there’s stuff like that. But the way in which teams get sort of swept up in a low-quality ecosystem, and they can’t differentiate the low quality from the high quality, seems to me to be a contributor in the kind of –

FOMO-driven –

…narrative misformation…

…development, to some degree?

Yeah… Which eventually becomes momentum-driven, or kind of like path of least resistance-driven… And again, I want to highlight that I think a lot of what’s going wrong here is going wrong in the product owner and product management class. I am sick to death of walking into teams, engagements, and then seeing the management not realize that – like, trying to blame the engineers.

I remember this one meeting… Per usual, the names of events changed or obfuscated to protect the guilty. But the one trick I’ve got is to show people what their experiences are actually like on the phones that people actually carry. Or the laptops. Like, this is a Surface SE. This is the high-end version, so it’s four-core; I think it’s maybe eight gigabyte. Those are four slow Celeron cores, with eMMC for storage. These things aren’t fast. And so like when you show product owners how their situated experience for a marginal user actually goes, a lot of times they kind of get upset… And then there is a maybe natural, but there’s certainly an instinct sometimes to say, “Oh, I knew those dirty engineers were getting one over on me”, when in fact what happened was they simply assumed that the new generation of technologies that they were buying would embody all of the properties of the last generation, plus the benefits that they were sold. Right? So it’s like the Caeteris paribus - the all else equal, now I get these things. And in fact, the all else equal - there’s a quantity that has to be managed. You can’t assume that generation after generation, especially if you’re doing a rewrite or a replacement, that you’re going to preserve the same properties that you had come to either value in the past and then bake in, or create processes around, or all that.

So a lot of times, teams will get ambushed by huge piles of JavaScript, because they, as a product owner, or PM, did not put any constraints on that, and the engineers went HAM with the new popular thing. They hadn’t been told that these tools don’t scale without huge amounts of infrastructure… Despite the fact that, from my perspective, I’ve been talking with the teams who are pitching these things for many years, showing them extremely copious evidence of uncontrolled versions of these systems not working well, knowing full well that they know from their own experiences deploying systems at scale that these things don’t actually work without a lot of constraints. And when asked over and over and over and over and over again to just signpost what they already know about the situation, and the kind of discipline that they’ve had to put in place to keep these systems in line, inside their own product, the worst of them have been extremely unwilling to do it. They won’t even tell you what they know.

[19:40] And so when I’m frustrated, it’s not because this is like a new thing, and I’m telling people like for the first time, “Hey, by the way, this thing is slow.” It’s like, no, we’ve been quietly in the background trying to work for many years to try to improve this without kind of explosions. But we’re 10 years into this, into the Cambrian explosion of JavaScript as the first, most, and only way to build frontends. And I think we have to mark it to market and say “It’s not going well. It’s just not going well.”

So Alex, it’s fair to say that the problem here is just too much JavaScript, right? Too much JavaScript, and all the “popular” tools that we’ve been using for the past decade or so we’ve been developing kind of desktop-first; that’s kind of your core thesis here, that’s backed by data and metrics that you’re seeing in the wild… How do you think we really got here? In particular, where did the shift on end user performance - where did we lose that story?

Yeah, it’s a really deep question. So my perspective is that a couple of things stopped being true that had been true in the past… And if you look at the 2010, ’11, ’12, ‘13 moment of frameworks coming in from the cold as like this brave new way to do everything, they had mostly been built in a desktop-first world. The mobile web was not a sidelight at the moment at the time. Post 2008, post iPhone, mobile users were a real thing. Every business was thinking through, “Do I do responsive? Do I do [unintelligible 00:21:19.11] What is my approach to doing all of this?” Those conversations started to fall away at some point, because the tools that people would pick up, that they preferred, had been built for this desktop era, and they just weren’t working on the mobile side of things. So a lot of businesses actually split their vote, and said, “Okay, well, we’re going to do apps for mobile, and we’re going to do web for desktop, and never the twain shall meet.” And of course, this creates a whole new series of product problems. But the key thing about this is that the JavaScript community, the web community was kind of high on its own supply by 2010, and had started to furiously discount the reality of mobile.

And then as soon as 2015 happened, and Apple started to pull away, like really pull away on CPU performance for devices, the fact that the wealthy developers who were the JavaScript intelligencia all carried iPhones started to create an ever tighter privilege bubble around their experiences, in which they couldn’t understand what people like me were on about. I gave this talk in 2016, talking about - I’ll find you a link - how the performance of the stuff wasn’t working, and trying to put a finger on exactly these factors. But Moore’s Law was working, in the sense that every 18 to 24 months we would continue to see a doubling of transistors for a given price, right? Like, that’s kind of the outcome of Moore’s law. But because the price of devices were just falling as they entered mass market, that meant that we had a fixed performance budget every year, because we were in an explosion of distribution to ever poorer users, year on year; the next marginal user who’d get a smartphone that for the first time, the person who was somewhere in rural Indonesia, or rural India, or a tier three city in India, who were swapping out their flip phone that they’d had for a decade, for the first smartphone, they weren’t paying $400, $500, $600.

For the last decade, that median price of a smartphone has basically been the same, at about 300 bucks; it’s been hovering – like, it fluctuates a little bit. At the same time, I think if you’re wealthy like us, if you live in a house in the United States, like your wealthy world historically - if you’re wealthy like us, then the price that you will pay for a new unlocked device has only continued to escalate, as has the performance of that device.

[23:48] So we have been experiencing a discontinuity, where our previous alignment as developers to something that kind of felt a little bit like the lives of our users - like, maybe we’d have slightly faster computers, maybe we’d have slightly better network access… But you know, we had friends and family whose experiences were basically kind of in line, if they were online; that is now a yawning chasm. People talk about the digital divide - this is the digital divide, is the median to low-end smartphones versus the wealthy people devices. And that has also kind of, by the way, started to take hold in desktops as well, although it’s a much smaller market. Desktops are important, obviously; Microsoft thinks desktops are important, but they are not the same thing as where the puck is going.

And so web developers didn’t adapt to that new evidence, and as the evidence started to show up, it became pretty clear that there was a whole pile of people who didn’t want to engage with it, because it was going to invalidate some of their preferences and priors. So this post I wrote this last week is – like, I gave a talk in 2016 that said, “Hey, we can fix this yet.” And I gave a talk in 2019, where I went through a bunch of the data about how it still wasn’t going well. This is the final straw. This is “Yeah, done f****d up. It’s not working.” And in that sense, I have seen so much over this long arc of so many teams get stranded in exactly the same way, and they all think that there’s something wrong with them.

Yeah, yeah. So let’s unpack that. So teams thinking that “Hey, we are really trying our best, but we keep kind of ending up in the same situation of really not being able to deliver performance user experiences…”

They think they’re the first. That’s the first problem, they all think that they must be holding it wrong.

Right, exactly. And your thesis – I keep saying the word “thesis” here, but I can’t think of a better word… Your –

Experience?

Yes, your experience has been that “Hey, it’s not you, it’s the tools.” And it’s not just the tools, it’s the culture”, and there’s no single culprit here. We’re all guilty, myself included. We’re on a podcast called JS Party… I mean, come on. But to kind of peel the layer back for how we got here - first iPhone came out in 2007; you’re talking 2010, in terms of timelines… There’s still a pretty desktop-heavy bias, and for me, momentum is momentum. You have developers kind of trudging along one way, and as someone who’s been a lead engineer for, I would say, a few years now, trying to stop a bunch of determined engineers and like getting them to turn left instead of right is really quite a challenge, right? It’s not easy. And that discernment, and that kind of discipline – like, the discipline of being self-restricting, right? It’s just not – I feel like frontend engineering is just a big creative space where people just want to do the things, and do all the things in JavaScript… And for what it’s worth, yeah, JavaScript may be a drug, and we’re all on it, you know what I mean? And so – yeah, it’s tough. It’s tough to break out of that bubble and momentum. It’s a very strong pull.

And I think you’re right to flag this as a lack of constraint, right? Because that’s what I’ve experienced. And I mentioned this memory of going to this meeting with a team that had flown all the way in from somewhere in Europe to Mountain View, making a price comparison website, right? But you’re not at a price comparison website if you’re shopping for Prada handbags; if you’re trying to get your next Birkin. Maybe you are, but that wasn’t their market. And so we showed these folks their experience on the actual kind of devices that their actual users have. And they were astonished, and then they started to try to blame the engineers… And I said, “Stop. No. This is on you. Show me your phones.”

[laughs]

[28:05] Right? You got all these people with C in the name to get the phones out of their pockets and put them on the table. It’s all the latest iPhones. They don’t live like their users; they have not done the work to develop an appreciation inside of their product organization, that the product is the experience that the users have of a digital service, and that you have to be there with the user if you’re going to do a good job at the product.

And this is a million miles away, by the way, from how the most successful digital brands operate. Facebook, for instance, since we’re on the topic, famously had kind of like 2G Thursdays, I think it was, where all the Wi-Fi APs in the entire company would suddenly go down to slow 2G speeds. If you ask what framework google.com uses, I think it’s still some JS action stuff that was built in-house in like 2009… Because it can’t afford more than that.

What is the framework that Amazon is telling you all about? Right? They’re not. Why is that? Because that would absolutely destroy their numbers; that would destroy their business. So they’re not doing that. I’ve heard so many tales out of eBay, for instance, about people coming in and being extremely excited to get rid of Marco. “We have to move to React, because blah-blah-blah.” And then someone pulls out the spreadsheet and says, “Okay, is it worth X?”

[laughs] An acquaintance was telling me recently that they had thought that each kilobyte of JavaScript on their mid-market (again), not high-end product, like their mid-market SPA eCommerce website was costing them, at scale, about 100k a year in lost revenue. So they did the study. It turned out it was 700. Every kilobyte was worth $700,000 of revenue. Right?

So when I say that I’m getting called in, and people have this kind of terrified glint, because they’ve launched the big React rewrite, and the numbers are f*****g tanking; like they’re going – like, I’ve worked with so many teams that either can’t move product now, they are dead in the water, because everything they do feels like it’s making things worse… Or if they’ve got a kind of enlightened management organization, someone will have gone “Oh, I tried the new product. It’s no good; you can’t ship it.” So the thing they’ve been working on for 18 months is now blocked and dead in the water. Right?

So sitting with these teams who are in real pain, because the business has finally connected the fact of the flammable construction materials with the fact that it looks like there’s an arsonist on the loose, right? Like, those two things actually have a relationship. And when the business makes the relationship, it’s a bad time for the team.

So some teams can skate, right? Maybe you are selling Prada handbags. Although, okay, cool - are you doing it over an airplane Wi Fi connection? There’s just a lot of like situational stuff that you really want to work through if you’re really going to make a high-confidence assertion about who your users are, and where they’re situated.

There are some teams, who I’ve been lucky to work with, who absolutely have full control over their deployed client base. It’s amazing to work with them, because you can actually agree on what the business wants. But a lot of times, you’re in this sort of counseling session, trying to get the product owners to enunciate the things that they thought were constraints, that they never told anyone were constraints. They didn’t sign up to lose a ton of money when they did the rewrite. They knew that these JavaScript people were much more expensive than the HTML and CSS and PHP people that they had been employing before… And they thought that was going to yield a higher-quality product, not that it was going to tank revenue, right? Not that it was going to turn into a UX problem. Now, they do get more interactivity on the other side of it, but they didn’t sign up for that specific deal.

Alright, so Alex, I feel like that was a really great setup into the problem and the kind of “how we got here” situation for me. Listening to you, there’s two culprits here. It’s not just our product management organizations not thinking diligently through actual software deployment, and monitoring, and all that jazz, and like setting budgets etc. But then there’s kind of – and I’d say engineers are just as guilty. We’re the ones kind of advocating for certain tools, and patterns, we’re the ones who’ve really pushed maybe SPAs for longer than we should have… We’re kind of seeing some general consensus now within the community that “Yeah, single-page applications may be not the best solution for everything.” And depending on your use case, and what type of application, what part of the application, what your session time is - there’s so many different factors that maybe go into a decision like “Should this be a SPA?” And we’re kind of, I think, hopefully starting to understand that “Hey, there’s room for multiple architectures within one application, depending on which part.”

But again, when you look at what’s out there, and what most teams are doing etc. it is a very one-size-fits-all approach, for the most part. And not only is it one-size-fits-all, but you have startups using the same exact technologies as really large corporations, and everything in between. Like, is the technology also just generally one-size-fits-all, between companies who can afford a world-class staff and companies who can afford a fraction of that world-class staff? There’s, there’s a lot of different constraints and elements at play here. I’m curious to get your thoughts…

I think the way you’ve framed that in terms of trying to understand which architecture is going to make the most sense is really a pressing problem for our community. So I’ve been talking with colleagues and friends about this sort of little tiny model - I need to write it up in a blog post, but sort of like, every system, every interaction that you do… I wrote something about it last year, a kind of generalized theory of web performance, unified theory of web performance… And it kind of includes something like this. Basically, the idea is you start with a system being responsive, that is to say it’s not doing anything, right? It can take input at low latency; so you provide it input, it starts to do the work, and it acknowledges to you that it’s doing the work. That can be like, I don’t know, starting a loading spinner, or something. You do updates about that work, you do part of the work, you learn more about how it’s going, and then you continue to update the user about its progression. Eventually, you retire the work, and then you present the results of the operation, and then you’re back to square one. You are interactive again for the very next thing. And that loop, that core interaction loop, where you start quiescent and you end quiescent… Like, you start being able to take input, and then you return to being able to take the next input - that describes clicking on a link, or typing in a URL, just as much as it describes some update through a local data store or through a templating system to generate some diffed DOM. And so you can model the same interaction as either a full HTTP round trip, or as a fully local interaction, depending on where the data model lives.

[38:00] You remember the old Mapquest, where you had to like click north, south, east and west, and maybe you could get northeast, northwest? …this very painful thing. And then Google Maps, of course, the slippy maps made the world a lot better for folks who are doing that kind of interaction, because each incremental interaction involved much less variance and lower latency as you took each individual turn through that loop. Because grabbing it and moving it was a whole turn through that loop. And you could – again, at an intellectual level, you could model all those interactions as being full round trips. And we all came to agree that that was less good. But we didn’t say under what conditions is that less good. And so I’ve been trying to enunciate this with folks at work, and the closest thing that I can come up with is session depth weighting. So that is to say – think about a distribution of sessions, right? Because we’re never talking about just one user doing just one thing. We’re talking about users doing a series of things, in a session. And they have a distribution of actions they take. And there’s a distribution of different kinds of users through a site. But if you’re thinking in terms of, say, an eCommerce site, you have a couple of sort of prototypical session types that you think most about. So you’ll think most about a user who comes into the site door through a product page, from a search result; they add to their cart and they checkout immediately. Okay, that’s one flow.

There’s another flow, which is they go to your homepage, they go to the search box, they go to a search results page, they sort and filter a bunch, and then they go to a product detail page, and then they do some configuring and some futzing around there, and then they add something into the basket, and they do a little bit more shopping, and then they come back to the basket, and eventually they do a checkout. Those two sessions are extremely different in maybe their composition, and the question that I think we can try to tease out is, “If we know about the kinds of sessions that a site or product has, we can learn a lot more about what trade-offs are going to be better.”

I like to bring up the example of Gmail. I can click on a link to an email from Gmail and see just one email. Or I can load up Gmail as like my daily driver email client, or Outlook, or whatever; I can load up the thing to do 1,000 interactions through an extremely long session. Now, I think you’re right to be cheesed every time you see a loading bar to get to a single email, right? That 10 kilobytes of text shouldn’t require 10 megabytes of JavaScript, or something; or 3 megabytes, as the case may be. We can do better. So that’s related to session depth. And so if we think about SPAs being appropriate for long and deep sessions, especially where there’s like high frequency and low latency interaction, editors - editors are a perfect example. The Figmas, and the Photoshops of the world - you need a local data model, because otherwise network variance and network latency in the critical flow are not great. Whereas for lots of other kinds of applications, which are either not editors, or have much shorter sessions, even if you could imagine the most engaged user, fetishize the user who will spend all day on your thing, the reality probably isn’t that that’s who most of your users are… And we can know something about most applications, because honestly, most software projects today are not greenfield. Like, we’ve built these on the web before. We were building email on PIM clients, again, with tools that I used to help build back in 2006. Full business intelligence suites with Ajax in 2006-2007. The stuff isn’t new under the sun. We can say that we know something about these classes of applications now. And so as a result, we can start to characterize them by those session lengths, and then we can start to decide whether or not we need a local data model, and all the tools that are premised on operating over a local data model.

[42:01] When we talk about this SPA technology, what I think we’re really talking about is, “Do I have a local copy of some subset of the data that I’m going to be manipulating frequently, applying optimistic commits to, and then updating my UI about as quickly as I possibly can in order to make it feel better?” And that’s extremely subject-dependent as to whether or not the app itself is going to feature long sessions on average.

I love WordPress as an example here… So WordPress is two applications, right? WordPress is an editor; so you sit down and you want to write a blog post, you go into the editing UI, and you have a very long session, probably. It’s a drilled-in editor, that editor does lots of stuff, loads of stuff… You can sit there and preview, and refresh, and all that kind of stuff. Whereas if you’re a reader, that experience is mostly going to be thin consumption; like, one to two clicks.

And so in those sessions, the user who has a very long session can divide the total session costs in terms of latency, or payload, or whatever you want to call it - because those things kind of blend into each other - over the number of interactions that you take. So if you can drive down the resulting fractional number, if you can get that to some low number because you front-loaded stuff, and therefore each subsequent cost was much lower, versus the sort of full-page refresh model, then you’re winning. But if you’re a reader of a blog, you can’t afford any of that, because the denominator is one, maybe two. Scrolling doesn’t count, because the browser does scrolling for you, and it’s magic, but–

One chance to make a first impression. And yeah, depending on the context, you definitely need to meet your user sooner rather than later. And yeah, I think this whole concept of – like, the nuance of use case is something that really hasn’t been fleshed out in our community as much, right? And I’m eager to see your post on that topic. But there’s just also gaps with the way we generally build products. We’re not really thinking through – like, our whole model for software development is just very one-size-fits-all, build the thing, ship it… This kind of nuance of really building for use case - it just isn’t there. So is this something that will – is this alone enough to solve our problems? I’d say a bunch of them, probably, but I think that still kind of leaves us short in terms of what are the most “popular” tools today, and how do they help us easily meet that gap. Because the reality is developers have 101 things to do on any given day, product teams are strapped on time… Right? There’s that whole time thing, “This needed to be delivered yesterday.” And so this is why frameworks are popular, right? There’s a turnkey element to the experience of using a framework, and also buying into a community of existing utils, and whatever, that are part of that frameworks community… So where do those kinds of real world problems kind of come into helping kind of shore up this gap in how we deliver software?

Yeah, I’m reminded that engineering disciplines that kind of survive maintain their social license to operate by not naming their operators on a regular basis, right? [laughs] And you get there lots of different ways; you get there through labeling, maybe, you get there through training, lots of training… Most serious engineering disciplines involve a great deal of training. And I think it’s fair to say that we’ve made the front end so complex now that anyone who can hack it writing JavaScript in a kind of complex frontend today can easily train to learn CSS and HTML, and the basics. So I think that we have a skills gap, not a capability gap at some level.

[46:04] But then, as you were sort of focusing back on the product side, we have a gap in appreciation for evidence. So I hear an overwhelming sort of assertion that the developer experience is now significantly better than it was before. And I just keep waiting for someone to prove it… Because the teams that I work with have astonishing build times, sometimes. I’ve seen teams with 40-50 minute builds, 2-hour builds, for a website. For a website. Right? And that’s on modern hardware, that’s not on the hardware that we were building several thousand or a million page websites out of in 2006 or 2008, right? That’s on SSDs. Disk got 1000 times faster in terms of latency. That’s astonishing.

Right, but we have like the JavaScript build ecosystem and tooling ecosystem has also exponentially grown… And as it should, in many ways, because I think we’re able to do better in terms of shipping more optimal code across the wire, or linting, and whatever else. There’s pros and cons to that, I think. But, we don’t like you don’t want people to be doing that by hand, right?

I mean, I think that’s true when you are operating your system, and it’s not operating you. A lot of teams that I’ve worked with are in this place where they have bought, even though they didn’t realize it was a transaction going on – they did the starter project thing, they took the next thing, they took the Angular starter kit, they took the whatever, and now they’ve inherited a great deal of complexity… Complexity that they now own, and are the one and only stewards of. They’re plugged into the community for dear life, because that is complexity that they probably don’t fully comprehend, certainly not in every area, and each additional place that they have been sort of granted that complexity in a stack without an affirmative choice, and a measurement, is a place where – they’re taking it on someone else’s word that this is the right choice, which is often a thing that has to be re-excavated later, and investigated.

So the thing that I think I’m most frustrated about with the product side of the world is that we haven’t done a good job of creating a class of product managers that are willing to do bake-offs, that are willing to actually try to ask the question, “Okay, what is the evidence to say that this is actually improving my developer productivity?”

In aggregate, if I build a product, and teams get dead in the water for a month before or after launch, the wind in the hair feeling at the first month - you know, when you had two engineers on it, not 20… Is that faster? Is that better? Are we actually going faster?

So I’m extremely disappointed right now, with the web development community’s sort of poor characterization of its very bold assertions. Maybe they hold water, but the fact that no one’s looking and no one’s asking, it suggests to me that we’ve entered a kind of evidence-free zone, and you just – you don’t get out of that except by accident. So we have to be a little bit more intentional, I submit. And this is where, you know, when we’re working with teams to try to get them out of the ditch - because there’s so many products in the ditch - the most important thing that we wind up doing in this sort of technical coaching kind of thing is to help the teams learn what choices they have embedded inside their stacks, and then learn to operate those choices. Because in many cases, those choices are just latent. They’re not things that they are thinking about in relationship to the other parts of the system, so that they can make trade-offs about them. Because when the system is just kind of there, and everything you do to it does something incremental to it, it’s kind of spooky action at a distance. And that’s a really bad feeling for the team. And so to get a team into a place where it can approach mastery of its own system, you kind of have to walk through a very long period of acknowledging what you don’t know about the system that you’ve inherited.

[50:17] And these are all complex systems, right? A web browser is an incredibly complex system. And so part of this process is very often also teaching teams about how browsers think about their content, like how browsers actually work… Because once you can understand the choices that you’re making in context of like how browsers think about the content you send on the wire, you can start to, with a little bit more confidence, evaluate and then maybe reevaluate some of your choices incrementally. Because the thing that I’m never going to do is say “Blow up your thing and rewrite it.” Maybe if it’s actually so bad… But almost always - and in one of these consulting kind of situations, it’s going to be a “Here’s how we’re going to make the next increment of steady progress towards getting this thing out of the ditch. We’re gonna put some planks under it; we’re gonna put it in low gear, we’re gonna get a tow truck, we’re gonna pull it out gently, we’re gonna inspect it for damage, and then we’re gonna see if we can actually steer it without the axle being bent”, right?

There’s just a series of concrete steps you go through, and they’re all very painful when you’re in that position, because nothing that you’re doing at that moment is going down the road. So when people tell me that they’re having a great developer experience, I kind of want to see it in data.

Dense in all the right ways; lots of vegetables in those previous two segments… And so I’d really like to kind of see if we can take this messaging to kind of give developers some hope, right? Is all lost? In your blog post, you call out something that’s really made me chuckle… You said “Nobody wanted to turn the house lights on while the JS party was in full swing.” We’re on JS Party, right? [laughs] …and I’m ready to turn on the house lights. But I’m more specifically interested in hearing what’s kind of our way out of this. So we’ve been in this Hungry Hungry Hippo mode for JavaScript. We’ve also been in this one-size-fits-all kind of paradigm for quite a while, it’s not working… So what’s our way out? You kind of name a bunch of really great tools in our community, everything from Stencil, to Solid, to Qwik… A bunch of great tools that are good, lightweight alternatives, that are really kind of not so JavaScript-heavy, and also offer full-class support for IE, and legacy browsers… So where do we go from here?

I think that’s an important question. I’ve been saying for a while now that I would like fewer disasters across my desk… Because it would be great to get back to working on new APIs that people need, rather than spending a lot of time convalescing with a carefully-constructed disaster.

So the way that I think we get out of this is not to switch our tools. Teams that do not operate their tools, but whose tools operate them, are unlikely to succeed with anything but the simplest tools… Which is to say, if you don’t have a lot of organizational capacity to manage complexity, pick the simplest thing that can possibly work. And that is to say, output HTML and CSS and leave JavaScript alone, because it’s not for you, no matter how good you as maybe that one developer are.

Okay, so at an organizational level, preferring tools that are simpler, because they’re simpler, because that reduces your total costs over time is just a nice thing to recognize as a pattern that can help. But that doesn’t help you if you’ve bought into a pile of complexity that you’re having a hard time managing. And I get that. So there’s a lot of good stuff that’s happening in the ecosystem right now, that I think will help. The first, of course, is that core vitals is creating visibility for management about your progress to doing a better job. I’m explicitly excited about interaction in XPaint, because it is finally starting to put a price on the worst effects of JavaScript.

So that will help both motivate, and then create credit for improving things like task scheduling, and main-thread work blocking, and all that kind of stuff. That’s great. That’s awesome. And then the tool - again, the tools themselves are depending on their endemic complexity, their basal complexity. They are more or less of a problem in and of themselves… But I have seen extreme disasters made out of the best tools, and I’ve seen teams that could sort of field-strip and rebuild the framework themselves blindfolded in two minutes - they can operate whatever, right? And I think, I think we talk about those two extremes as though they are the only thing, but in the broad middle, the thing that’s very helpful is to learn what things cost, and then start to just put a little bit of management control around it. Because there is good news in the networks and devices that we’re seeing.

Computers are about to start getting a little bit faster at the very bottom. That’s going to start to happen again. And if we can hold the line for a bit, then the sites and things that we’ve deployed today - they will certainly feel a little bit more affordable in the future, if they’re close to the line. If they’re not, you’re gonna have a hard draft, you’re gonna have to dig out…
Other things that are helpful right now - if you have one of these big complex UIs, and it’s not scaling, for instance in terms of style complexity, Nolan Lawson has been doing extraordinary work over the last year in sort of outlining how DOMs that are out of control, coupled with style sheets that are [unintelligible 00:57:32.05] and often being perturbed by very poorly-done CSS-in-JS tools - that combination is quite toxic for interactivity. But we can get it under control, right? There’s generations of CSS-in-JS tools that will extract all your style sheets out, turn it into a single CSS file that’s not being manipulated; that’s awesome.

[57:51] I’m working with and have seen teams kind of hollow out their existing React components and replace the guts with web components, so that they get Shadow DOM in isolation, and they don’t have this combinatorial explosion of complexity there. And that helps a lot.

And then teams learning to put the marginal user at the center of the conversation is just the most transformative thing. If you are sitting there talking about how it feels on the P90, the P75 if your PMs carry these low-cost devices that are relevant to your business - and I don’t want to say either is the one that you should use, although I have a post on like the performance baseline worldwide… But if you think about what you would want to know about your marginal user, and then decide intentionally to go serve that user, there are so many good tools, right? It isn’t just going to be, as you say, one size fits all. But you can learn what size fits. We’ve gotten an entire department store full of great tools. HTML and CSS, very thin frameworks, compositional things, “server-side rendering” in the style that we do it now, old school server-side rendering in static sites… There’s a lot of cool stuff happening with Astro and Eleventy, and Hugo, and PHP, and DotNet. There’s this whole closet full of stuff that we’re not talking about now, that can be extremely appropriate, and you can use it, and you can deploy it, and it probably isn’t worse. You may not be comfortable with it right now, but if the output that it creates is simpler, your chances of success with it are significantly higher.

And so dust off the Rails, dust off the Django. There is a place for this tech, and if you learn about the kind of users you’ve gotten, the sessions that they’re working through in your experience, you can make appropriate choices, and you can hire for those skills. I promise you that you can. There’s a fire sale on talent right now. In fact, one of the most depressing things about the last decade has been how folks with JavaScript skills have pushed out people who have those skills and would like to come back to using them. They’re just sitting out there, and I promise you that if your product or your team is saying “Oh, but we can’t find people like that”, t’s mostly because you aren’t trying.

And when you say “they”, just to clarify, you mean people with skills, specifically skills with –

Semantic markup and modern CSS, yeah…

Exactly. Got it. Great answer… Lots to kind of unpack there. I think for me, the one thing that really stands out the most with making progress on some of the things that you mentioned is, first of all, I think product teams have gotten really used to the speed and the tailwind of open source. For example, you’re using React, and there’s a React thing for everything. And so whip me up a piece of that React app, or whatever it is, right? Whether it’s build tools, utils, whatever. There’s all kinds of options out there. And I think with some of these other libraries you’ve mentioned, there’s a smaller ecosystem… There’s also like the issue of skills, and developers feeling like “Oh, I only have the ability to maybe specialize in one or two frameworks at the most. I’m going to pick the biggest, most popular thing”, which Josh Collinsworth had that really great blog post a few months ago, that was called “The self-fulfilling prophecy of React”, where he talked a little bit about – his thesis was that React was only good at being popular.

So that’s a hard thing to – people thinking that something like React equals potential marketable skills, and then the ecosystem behind it… Those are two heavy things to fight, and I’m curious what your thoughts are on that.

[01:01:48.18] Yeah, so I’ll try to break them apart a little bit, and say that if you’re an individual who has skills in JavaScript, and you’re looking for how to improve, again, the most important thing you can do is learn to be putting the user that you’re trying to serve at the center of it, and then start to think in terms of hitting frames, every 16 milliseconds; hitting that interaction target that you would like to have hit on your extremely expensive device, on that cheaper device, and for that marginal user. I promise you can do it. Computers are fast. And you’ll find out where your framework might be a stumbling block, or where React will generate way too much GC in a critical section or something, and it’ll hurt you. And that will turn into a challenge and you’ll learn to solve it… Maybe inside the framework, maybe you’ll learn to go around it a little bit… But those are incredibly important learning opportunities for you to understand the trade-offs.

And then I would suggest, again, as an individual, if you’ve been living up in JS land, where you’re kind of doing everything in user land, the most important hours you can spend are not learning another framework; they are learning the system underneath you. Every hour you spend learning how the browser actually processes your content and actually thinks about it, actually thinks about the networking and actually thinks about how we take DOM and CSS and we construct them together, and we create the layout tree, and then we paint out of the layout tree, and then we rasterize that out in the GPU - that flow and how we do that every 60 hertz, every 16 milliseconds, learning the pieces of it and the sensitivities of that system to what you do in CSS, and JavaScript, and through your HTML, will make you a better developer in every framework.

So it’s extremely important that you learn to think about that knowledge not as something that you don’t have to think about anymore, but as something that now that you’re not a beginner anymore is extremely valuable to you… Because you’re not at the end of your career, so it’s going to be important that you be able to adapt on the basis of shared knowledge across all of it.

And then for teams and for organizations, I’d say that that hiring thing - again, that’s trope; it’s not reality. I promise you there’s a fire sale on talent right now. Go look. You could just pay for it, and you’ll pay less than you think you would. But in terms of the choices that you make in terms of stacks, I would encourage folks to be a little bit more data-oriented, right? Again, step back and look at the actual experience. How many of these tools that you think are necessary are there to solve problems that the tools that you thought were necessary created, right?

if you start with simpler output, one of the cool things about it is – okay, let’s just say you move from taking your static marketing website out of Gatsby, which is going to go with some huge GraphQL query to return a bunch of React components, and render that on the server side, and do all this stuff. And then you have to think about how you’re gonna smush all these things out, and like do the CSS minification, and all that stuff… Whereas if you just output the same basic HTML, with the SQL query, and some PHP, or some Python, or some Ruby, how much more headroom would you have in the output in terms of the cost of the output, before you ever had to think about applying a tool to start tamping down on those costs, right? Because once you start thinking in terms of budgeting, latency budgeting, complexity budgeting, you can start to think about the trade-offs as being not about like for like, but it’s about being different in ways that structurally set you up for success if your system is simpler.

Now, once you get to a very complicated product, when you know you have a really complicated product, then you will need sophisticated tools, and they will need a lot of management. But going through that process of evaluating those things, and doing bake-offs, and learning the properties that emerge out of those simpler versus more complex systems will teach you something, it’ll give you a finger feel for whether or not one style of construction is going to be more appropriate here or there, and maybe help you make more informed trade-offs in the future about the style to build in for a particular kind of use case.

[01:05:43.08] And then I’d say at a community level, we need to step back, and I think we need to think about our relationship to society. And I don’t think that that’s going to be a thing that is traumatic, or even hard, right? We’re all here to build great experiences. I believe that. I truly believe that. I think Laurie’s piece in response to mine said something about how I think that developers are stupid, which is hilarious, on a couple of levels. But I just don’t. I work with extremely talented people all day long, and I’m grateful to get to work with them. And if we take seriously that it is evidence and science that are going to get us to a place where when we apply our skills, they turn into good things for the people that we wanted them to, that’s a fundamentally optimistic way of thinking about this technology.

We don’t have to accept what has happened, or accept that it happened for good reasons, or even accept that the people who are so attached to it are correct, or worth having in the conversation, in fact. But once we put the evidence at the center of what’s working and what’s not, I think we can start to reevaluate our relationship to the technologies, and acknowledge on the one hand we’re more than our choice in Java framework… [laughs] I promise you, there’s life after the framework that you love right now. And also, that as a discipline, the more we are reliably delivering good things for users and for businesses, the more likely it is that there’ll be more work for us, more good things that we’re going to be able to create in the future, and more users that we can help. And for me, that’s the goal, is to be able to help users. So when we put that at the center, and we put the marginal user at the center of that question, then I think we get to a good place. So I’m optimistic.

Amen to that. Yeah, Alex, I mean, that’s really deeply insightful; thank you for sharing that. I think that kind of discipline around really thinking through everything that we do in the in the lens of our end user experience is something that I hope we can all embrace. I do really think there’s large kind of social pressures on teams that are developing; the kind of discipline that you’re speaking to for me seems like “Oh, in a perfect world, if I had cycles, I think every engineer would want to put their best foot forward, and really kind of unpack these things.” And this is why I think I personally would like to kind of shift this to like a higher-order problem, that we can solve at a community level, and then kind of then pull in at the team level… I think both in terms of training, and like socialization, and kind of thinking about things from our product management perspective, having a new way of building products, and then also our engineers having the cycles to kind of make decisions that are informed by data. I think that’s something that really is a missing discipline for us. So often we’re looking at things on Reddit, or npm download counts, or GitHub stars, or whatever… And those are certainly good metric metrics, but they’re not the only…

You asked a really insightful question earlier, “How did we get here?” And part of how we got here was that engineers enjoy a lot of autonomy, and a lot of power inside their organizations, in ways that I think are not – they’re not always legible. But it is the case that as an individual engineer, you have a great deal of authority over how the task is accomplished, and you can bring your values to that question. Not all the time, maybe not in the best way that you could possibly imagine, maybe not in the purest form… But it is worth trying, because we got here because some of us were confused about what that world looked like, who we were serving, how it was going to work… And we don’t have to remain confused, but we still have that power. We can help shape whether or not products are succeeding more often or not. And then we can learn to socialize the value of data and evidence in evaluating whether or not it’s working, not just to make ourselves look better, or because we did a better job, although hopefully you will get promoted more effectively if the organization also learns to value evidence and data, but because that’s how we preserve our social license to operate.

[01:09:59.14] We will continue to be able to make websites for a living, if and only if making a website for living results in something that people want to use. So we all have skin in the game here, and I think it’s really important that we acknowledge that we have some incremental - not absolute, but real influence on the kinds of decisions that our organizations are making. You can push back on the nonsense; you can call it out as nonsense when it isn’t going to work for those users.

So let’s not use a blanket kind of “These people are good, these people are bad”, until the evidence is all the way in about behavior changing or not… But we definitely need to put evidence at the center of who we’re serving and why, and I think we can do that.

Yeah, that’s a great point. So to kind of close this off - I mean, honestly, we could talk about this for hours; there’s so much to unpack. I think for folks like Laurie (Seldo) who we’ll put a link in the show notes to Laurie’s kind of counter to Alex’s article… For folks who really maybe don’t understand where you’re coming from, what’s your nuanced kind of response to that? And for folks who also think, “Yeah, you’re being too harsh, you’re calling people out, you’re singling things”, what’s your answer to that?

People are getting hurt; people I care about, users and teams that I care about were actively misled in some cases, and so that’s bad… So that needs to change. And we have not been able to affect it with any other mechanism. Like I said, I think at the front, this was like plan D. You don’t do this first. You quietly gather the evidence, you share it around with other people assuming goodwill, you try to advocate for better outcomes, you try to advocate for centering users, you do all of those things. You preserve the confidence of the people who bring you in to observe how things are going in their organizations, because they’re taking huge risks by admitting in the first place that it’s all going to s**t, even when it’s observable from space. [laughs] Even when the experiences are just clearly not very good, there’s a huge organizational risk to anyone sort of talking with me, or being willing to talk with other web performance consultants about how to improve, because it admits at some level that it’s not ideal.

Defeat. It’s defeat, to some degree.

It’s not always defeat, but it’s a curiosity about how you can improve, at least, and hopefully a recognition that you can. So I think there is a way to think this through as a time that we can eventually talk about as having been confused. And that will be good, because on the other side of confusion hopefully isn’t just another fad, but is instead measurement and bake-offs and an argument about how we’re going to situate things for these users, but not these users… Like a real engineering discourse about how the trade-offs we’re going to make are going to be more optimal in some cases rather than others. And I believe in this community, we can get there, we can go there; we don’t have to put religion at the center of things. We can center evidence, and we can keep users at the heart of everything we’re

doing.

[01:13:05.23] Wait a second, are you saying that Twitter polls aren’t a form of like science and/or true metric collection? [laughs] What’s going on, Alex?

Look, it’s right next to my bucket of leeches in terms of the scientific method. It’s just an evolution.

Hey, leeches? What’s wrong with leeches? Leeches work. [laughs] No, I’m just kidding. Alright, so Alex, just because I know people are going to be curious, hopefully inspired, where can folks start? I’ve been kind of on the fundamentals train for a very long time; I’m curious to hear from you, for folks who really want to deeply master how the browser works, right? You’ve been working on browsers for a long time, so what’s your advice for average web devs to kind of get started with kind of mastering the web platform?

So there’s a couple of really great blog posts over the years; I’ll find them and we’ll put them in the show notes. I think one of them is called “How browsers work.” But there’s also – and Chris Harrelson on the Chrome team, and a co-author whose name escapes me at the moment, recently put together a little kind of like “How to build a browser in Python” thing. It’s actually not a lot; it’s a very simple render loop. You can learn a lot about how all that works from that example, and I think it’s worth looking at if you can read Python…

And this will sound a little bit more daunting, but we have incredible training materials for new engineers who come from the classical computer science world and are just being plopped right into browser engineering. That’s not great. So it’s called Chromium University, because I promise you that browsers are as alien to people who work in C++ as C++ is to people who make websites. [laughs] So there’s a lot of crossover in this educational material. My favorite of them is called Life of a Pixel. You don’t have to understand anything about what it’s talking about in terms of the CPU, but it will outline for you how your inputs turn into these internal phases, and what happens next. Steve Cove has been giving this talk for a series of years now. Let it wash over you. You don’t have to understand it. It takes a long time.

And then lastly, as you go through and start learning about the inputs and outputs of your system, you’re going to want better tools. I would recommend - Web Page Test is the gold standard of these tools, but the dev tools that are already in front of you are pretty great. They can help you explore and learn and start to create correlations between this input in that output. There is great documentation online about those tools today; you don’t have to become a performance expert, right? But it really helps to develop an appreciation for how your marginal user is actually going to experience it.

So think about learning about how the system under you works, and then getting yourself cheap devices, cheap, cheap, cheap devices; get yourself a cheap phone. The good news is it’s cheap, the bad news is everything else. A cheap laptop. And it will do you right. It’ll serve you well. I’ve had this Moto G4 that I still test things on since 2017. These things will last a long time; they’re not going to get any faster, but hopefully, they will help you to level-set much more effectively. And I’ve got some recommendations of good, cheap devices to buy in recent post on the Performance Inequality. I’ll also put that in the links.

Yes. Yeah, lots of links. Thank you again. Devices is also another, I think, opportunity for product teams socialization where, you know, let’s normalize people testing on actual devices, and like have teams budget for that as well. But anyways, Alex, it’s been a pleasure. Thank you for taking us to school. Thank you all for listening. I’d encourage you all to listen and relisten to this, as I will… And we’ll catch you all next week. Thanks again, everyone. Bye-bye.