JavaScript sprinkles in Basecamp turned Stimulus

So David, I guess the thing we’re here to talk about is how the JavaScript sprinkles that were inside of Basecamp turned into a full-fledged framework, Stimulus JS. Can you tell us the story of sprinkles to Stimulus?

Absolutely. So since the very first version of Basecamp, since we started dabbling with JavaScript, both in the form of sort of pre-Ajax, and then post-Ajax, and then through Prototype, and jQuery and all the other libraries, we’ve always adopted a sort of distanced approach somewhat; what we were trying to do was we were trying to build some HTML documents that were linked together - I know this is terribly novel… [laughter] At the end we would adorn them progressively. Progressive enhancement is sort of the fancy term for it, but that doesn’t even fully describe our philosophy, because it’s not so much just progressive enhancement in terms of “Oh, your entire app should be able to be fully functional if someone turned off JavaScript.” That wasn’t why we did it, that wasn’t why I was interested in progressive enhancement.

Progressive enhancement was more an architectural technique of how to build great applications, which is actually similar to how I came to REST, and the REST principles for building HTTP applications. It wasn’t so much that I had this pure affection for representative state transfer and the thesis behind that. It was simply because it was a wonderful architectural pattern for guiding development and guiding how I should put an application together… And so it is with progressive enhancement - using HTML and having the server side generate that, and then sort of sprinkling little bits of dynamic dust all over the application. “Oh, this little button, instead of being a full page change, we’re just gonna have it dynamically update a tiny bit of the page.”

[04:06] It was really appealing to me, because the kind type of applications we build at Basecamp, including Basecamp itself, lends itself very well to that style. And I think that’s perhaps where some of the differences sometimes come in.

Basecamp and applications like it, including GitHub, that use a very similar style even if we don’t use the same frameworks, they have something called PJAX, that was a predecessor to Turbolinks, and that idea (which we can talk about later, too) - they actually go well together, Turbolinks and Stimulus; so that’s part of a two-pack punch. But in any case, applications like that, where it’s not about presenting a let’s say desktop-like, super high-fidelity, super-connected UI, where you’re sort of changing a little thing over here and then there’s five different other points that need to update at the same time… That was the world we were in, right? And some people are in a different world.

If you’re trying to make some sort of dashboard in a phase where you’re tweaking little dials and having all sorts of ramifications and updates, you have a different domain, and you should actually use different solutions for it. That’s one of the things that long has annoyed me, both with software development in general, but JavaScript development in particular - this idea that there’s one template for applications for the web, and that we should all embrace React and Redux as a lifesaver here. Or before that it was Angular, or before that it was something else. We have all the same kinds of applications and they all fit in the same mold, and I just couldn’t relate to that at all. So that’s why for however long it’s been that we’ve had these heavy JavaScript frameworks that used client-side MVC and tried to do all the generation of HTML on the client side, we’ve rejected that; we didn’t see the fit.

First of all, it didn’t fit obviously with progressive enhancement, but it also didn’t fit with what we were trying to do or made anything better. Over the years, we’ve had our dabbles in using frameworks like that, and it never ended up that the code was better afterwards. One of the techniques that I’m very fond of is doing VAB. You take a piece of real production code, a real feature, and you write it five different ways. That just tells the truth incredibly quickly. And the truth that I derive from that –

So did you try–

Sorry, go ahead.

In the process of time, did you try out like an Angular or a React and dabble with the Basecamp source code, and do these different things?

Yeah, I tried a bunch of different frameworks. Actually, just before going head into Stimulus, I did another broad search, essentially, evaluating all the frameworks that were already out there and saying “You know what, I don’t necessarily want to create another framework.” We have frameworks coming out the wazoo at Basecamp, right? I have my hands plenty busy with just Rails and Turbolinks and whatever. Is there a way that we can just adopt one of the existing frameworks, as we actually did with jQuery.

When jQuery was the hotshot, we just decided “Okay, we have this other framework prototype that we had extracted from Basecamp”, but it wasn’t really different enough from jQuery. And jQuery clearly had the momentum… Let’s just switch to jQuery. And we did the same thing with Rails, right? For many years, Rails shipped with jQuery in the box because that had the momentum, it was close enough, and it was actually a great framework. I, to this day, still like a lot of things about jQuery and I think its focus on ergonomics and API design were wonderful, and in some ways lost in this transition to the so-called “modern era” of JavaScript development.

In any case - where was my train of thought here? Did I try these things, was that what you were asking?

Exactly, yeah.

I was down this rabbit hole…

[08:13] Well, you’re the kind of person who has your own thoughts, has your own viewpoints on elegance, especially when it comes to code, and it would make sense that maybe you wouldn’t try it, but it seems you did.

Oh, absolutely, because I get inspired by it, too. Oh, that was where we were coming from, whether there was another existing framework out there that could serve our needs… I mean, I’m sure there would be; it’s not like the most terrible thing in the world would be if we used React for some things, for example, right? I don’t have any fundamental opposition against that. In fact, I think React in particular had an incredibly correct core inside, that instead of maintaining this complicated sense of state, you would just blow away the world and rerender it every time. I think that’s a great inside and actually quite similar to our views on how to do things. It sort of then went downhill from there, in my opinion. That was the sparkling moment where I thought “Oh, there’s something really interesting there”, and then I started seeing the mismatch of smashing all the considerations and concerns around components into one big file. We’ve spent I don’t know how long trying to separate things out such that we can be on different rates of change and all these other good things, and now here we go, React, HTML smashed in with code, smashed in with state management, smashed in with everything, and I thought “Meh, I don’t think that’s actually progress.”

So in any case, I reviewed all of these things, and then I also reviewed the JavaScript ecosystem at large, and I actually liked a whole lot better what I saw there than what I saw in the particular framework expressions that came from that world. I love absolutely the fact that ES5 onward was such a better language, especially for the type of JavaScript that I wanted to write, which was a more object-oriented style, and obviously influenced by the fact that Ruby is my true love, and I like that style of programming where it’s mostly object-oriented, and then you sprinkle in some functional programming techniques where they make sense. But at the core, I like to structure my code in an object-oriented manner, and ES5 forward just made that not be this horrible hoop-jumping exercise to do so, where we had to use CoffeeScript essentially go get an even half-way same syntax for declaring classes.

So I saw all that project in the JavaScript ecosystem and thought “This is wonderful, and it’s extra-wonderful that we don’t have to wait for the browsers to catch up.” The fact that we have Babel, the fact that we have Webpack, the fact that we have this whole toolchain that allows us to use almost all of the future-heading features of JavaScript today, in existing browsers, that people are actually using. That was what was so exciting.

So I basically took that exploration and said “Alright, I’ve examined both the ecosystem at large and the particular frameworks. I don’t care so much for the particular frameworks, I care for the ecosystem improvements. What can we do here? Can I just try to get my sense of progressive enhancement, my love of sprinkles into a more cohesive structure?” At first, it wasn’t even about the rest of the world, it was just about how we write JavaScript at Basecamp, it was about reforming that way. Because what I had come to dislike was we had such a high standard at Basecamp for writing beautiful Ruby code, and then we had a very uneven standard, in some ways, for writing JavaScript. On the one hand, we had some wonderful, deep-diving JavaScript explorations with Sam and Javan.

[11:59] They’ve made both Trix, and they’ve made Turbolinks, and it’s beautiful, wonderful code, and it’s really pure and wonderful… And then we had specific support sprinkles for a feature here, a feature there, and we had like four different styles of doing that, four different ways of attaching event handlers, and so forth… And someone coming in new to the Basecamp would go like “Which path should I follow again?” and then usually they’d just open some file that vaguely resembled what they were trying to do, and kind of follow that… Which wasn’t great, right?

You probably had heaps of CoffeeScript code at this point too as well, didn’t you?

CoffeeScript as well. CoffeeScript is not one of those things I look back on with any sense of regret. I think CoffeeScript was a monumental step forward for the syntax of JavaScript prior to ES5, and in some way still is, but the gap has now narrowed so considerably that I don’t know if the CoffeeScript path is necessarily worth it anymore, but it was totally worth it for those years when all we had was ES3, right?

How were you guys managing that migration inside of Basecamp? File-by-file, converting it to plain ES5 or ES6 JavaScript, or were you leaving the CoffeeScript in place? What’s the progress on that and how are you guys doing it?

We’re basically looking at things in sort of new horizons. When we make something new, we will make it to the best of our abilities, to the best concepts and whatever that we have, which today is Stimulus. So when we write new features, we write Stimulus controllers and that’s how we go. If we go back and revisit existing features and we have to update them in some way, we will sort of weigh whether the change is large enough to warrant a full conversion to Stimulus, or we should just leave it in place. Because the thing is you don’t really need to do this mass migration. The wonderful thing about having Babel and these transpilers is the fact that we can mix and match as we please. In fact, we do, even with modern code; Stimulus itself is written in Typescript, not in just vanilla ES5 or ES6. That’s because Sam and Javan enjoy using Typescript when they’re writing framework code… And I’m not a huge fan of sort of statically-typed or explicitly-typed (I should say) code.

So that’s not what I really wanna write and it’s not what we need to write when we write features, but they enjoy doing it for the framework itself, and since they’re doing the work, I’m like, “Of course you guys get to pick what you wanna do.” It doesn’t matter, it all just compiles down to the same thing in the end, so what do I care whether the parts of Stimulus that I didn’t have to write is written in Typescript, or CoffeeScript, or whatever?

There’s something to be said for having a general style in your application code perhaps, but these aren’t drastically different languages, right? They’re like dialects. I’m from Denmark, and in Copenhagen we speak one dialect, and I mostly still understand people from Jutland and the dialect that they speak. It’s actually not a big deal. I think people have a tendency to make it a much bigger deal than it really is.

That makes a lot of sense. I know as somebody who has dug into the Turbolinks source code, and I’ve written a fair bit of CoffeeScript back in the day, but hadn’t for maybe a year, 18 months… As a casual contributor or potential contributor, I maybe just opened a bug report or something, but looking at the Turbolinks code, which was CoffeeScript, I know that there was definitely some cognitive overhead, some catch-up I had to do in order to read that dialect.

I think on the other hand Typescript for libraries actually makes it easier to contribute because of the documentation available and the types being so obvious and stuff like that… Whereas CoffeeScript may have been a detractor for third-party contributors. I think Typescript might actually make it even easier for people to get involved.

[15:54] Maybe… [laughter] I’m not too convinced that that’s the main barrier to entry. In fact, I think that technologies have the tendency to overstate the differences in the technology choices versus the actual cognitive barrier there is to understand systems.

To contribute to, say, Turbolinks, there’s a fair conceptual model that you have to understand before you can contribute…

Yeah, that’s true.

…and I think climbing that hill - it’s a taller one than the subtle dialect differences between CoffeeScript and either ES5 or ES6 or Typescript. But to each their own… I think what’s just great is the fact that we don’t have to go through these violent transitions, where in order to have new features, for example of our application written in Stimulus, that uses Typescript, written in ES5 or ES6, we don’t have to rewrite everything that we did before that, because I think those types of transitions are incredibly painful, and I think in many ways it’s actually a blessing that we’re using transpilers all over the place, and it all just boils down to whatever you’re compiling to, whether that’s ES3 or ES5 or whatever your final output target is… It just matters less and less these days. I think that’s a real progress.

Since you mentioned Sam and Javan, and from what I understand, the process was – it sounds like you were doing some evaluation of JavaScript, the ecosystem, you’d done some research, and one day you prototyped something. And that kind of worked, and then Sam and Javan took that and rewrote it into what is now Stimulus, and that’s what you’re talking about, which is Typescript is the language that that’s written in. Can you kind of walk through that process of discovering this and kind of getting to the point where you could prototype something, what that was like, and then the transfer from you to them and how that played out?

Sure. So as I said, I did this huge survey of the landscape on all the major JavaScript frameworks, and I read through all of them and I played with all of them, and I tried to see which ideas I could get inspired by. And then I started up basically just play with all the new features in ES5 and ES6. This was sort of the first time I had really dug back in and tried what’s at the forefront of the JavaScript ecosystem development in quite some time… And I started using all the Babel packs for all the in-progress proposals for all sorts of things, and I just wanted to sort of like – if I could use the latest and the greatest, what would that feel like?

So I played with a bunch of different things. At first it was just general experiments, and then pretty quickly thereafter I became focused on “Okay, let me review all the JavaScript we have in Basecamp, let me try to see if I can pick out and extract the best styles that we’re already using, and kind of jell that into a framework that will guide people to stick with that style. Because a lot of what we had was basically just sort of subtle conventions of writing code, which I’m a big fan of conventions, but I think they’re much stronger when those conventions are backed up and enforced by a framework.

So that was basically what I was trying to do. I wasn’t necessarily trying to invent anything here, as I usually do with frameworks; I was trying to extract the best ideas that we already had, from this progressive lean hands to a sprinkles approach to JavaScript. I’d spent about two weeks doing that, and I pretty quickly came up with basically the DSL, the design of what Stimulus was going to be, and wrote it in ES6 with a bunch of plugins… There was enough there, we sort of stopped out in some ways; it didn’t have the full mutation of server and so forth, but my sense of the design was really focused around “Let me take an existing piece of JavaScript code in Basecamp that supports a given feature, that I don’t really like, that I feel like is messy or smelly or whatever, and then let me rewrite it in this proposed format that I have for stimulus and see if it’s better.

[20:13] I rewrote a bunch of different pieces of functionality and I was astounded by just how much nicer they were to work with when I had rewritten them in Stimulus. So after going through that process, I sort of chimed up Sam and Javan, who have been doing most of our deep-dive science projecty JavaScript development for a long time, including both Turbolinks and Trix. So they were sort of the natural team within Basecamp (we call them Research & Fidelity) to take this on.

Basically, I wanted what I had written up to really work, and not just be stopped out and not just be a prototype, but be the thing we actually wrote new JavaScript in. So they took this on and integrated another framework that Sam had been working on around MutationObservers called Sentinella and we ended up merging the two directions, and it became Stimulus. As I said, they rewrote the whole thing from scratch. In fact, since I wrote the initial prototype of the framework, I haven’t written a single line. I think if you do a GIT blame on the current repo, you’re not gonna find my signature on any of the lines written in Typescript.

My role - I sort of just shipped it from “Okay, I built a prototype, and I have a very clear idea of where I want the final API to go and what the client code should look like, so let me provide that guidance and let me help sort through some of the conceptual issues that we then faced on how to design the API, especially around targets and so forth”, and here we are.

Alright, so let’s talk about the architecture or the concepts of Stimulus, how you designed it, how it works and how you use it.

So one of the things I really wanted with Stimulus was I wanted to solve a couple of specific problems or bad patterns that I was seeing in our sprinkles code at Basecamp. One of the first things I wanted to address was the notion of how do you find the elements that you want to mutate or work with? We had a bunch of different styles.

[24:07] Sometimes we were using a hierarchical approach where you’d say “Oh, give me the parent of the parent of the parent here. I know that the structure of my DOM tree is like this, and I know I want the third parent up.” That’s a pretty brittle way of targeting elements, right? Someone reorganizes things or puts them in a different way and all of a sudden you’re getting the wrong element. So that wasn’t a good pattern.

Another pattern that we’ve used was targeting elements by finding them through CSS classes. So we’d say “Give me the elements that match this CSS class”, which on the one hand seems okay, because a lot of times the CSS classes are explanatory. They say “Oh, this element is about the title of a person, or something”, so when you query for that in the code, it kind of explains what you’re trying to get, but it’s also pretty brittle.

And it’s also not compatible with a lot of the modern CSS styles of writing classes, which are things like BEM, that have a very particular way of writing the classNames, sort of the BEM format, that actually detracts a bit from when you’re trying to use them as code identifiers, but they’re very helpful on the CSS side, right? The designers at Basecamp are really happy with BEM and the advantages that BEM affords them. But it’s brittle. Well, it’s two things - it’s ugly, and then it’s brittle. It’s ugly when you’re trying to target elements that have BEM classes in JavaScript code, because BEM has this format that makes sense for CSS and doesn’t make sense at all for code.

So I didn’t like the ugliness of the code, and I also didn’t like the brittleness of it. But BEM often times – like, you add another –down or a –pad, or whatever it is that you add to a BEM class to ever so slightly tweak it; well, the designers were doing that. Well, again, it broke the code. So that sense of brittleness I wanted to sort of get away from, and that’s where the concept of targets came up.

Basically, Stimulus implores you to only find DOM elements you want to work with through the concept of targets, which is basically just an explicit name that says “This element is going to go by this logical name that belongs to this controller.” And then we can move that name around and it’s not tied to this specific type of element; this could be an input element, it could be a button element, it could be spam element, and as long as it has a data-target that’s of a certain thing, you can always find it… Which not only gives you this sense of clarity around what elements when you read the HTML code are actually used by the dynamic behavior, and in the code itself, it’s very clear when you’re referencing a specific target what that target is and what the purpose is.

It also gives you a sense of generic distance from the specific structure of your DOM tree. As I said, a data target of let’s say hello.name can be applied to an input text, a text area, a span element… So you can write these generic controllers that work with all sorts of different kinds of specific DOM tree expressions, which is really neat… Because that was the other thing I noticed when I was reading through the Basecamp codebase, that we had a lot of feature-specific JavaScript that really was quite generic in its essence, and it was only tied to a specific feature because it was naming certain types of DOM element types, or it was specifically naming certain types of CSS. So you couldn’t really reuse these pieces of feature from one area of the application to another because they were tied to a specific screen and a specific layout.

[27:58] And I wanted to get a bit more generic around that, such that we could build up a library of generic behavior that we could apply to any sort of structure, regardless of how the designer wanted the page to look. So I wanted that distance between those two things.

That’s one of the other reasons why I’m not really a big fan of tying the specific DOM layout to the code, as React does and what a lot of component-style development does, because it doesn’t afford you this sense of reuse that you can develop generic concerns and generic aspects of your dynamic behavior that you can tie to any DOM tree.

That was one of the considerations, so that’s the target concept, that’s a prime motivator, and then the other thing – there’s really only two other things. There’s controllers and there’s actions. And actions are quite similar to targets. They are the triggers. A lot of code we had in Basecamp was using explicit event handling, where we would tie an event handler to usually somewhere up the tree, to some parent, and then we would sort of interrogate that event as it bottled up, to see if it was relevant for the behavior we were trying to match, and then do some code.

Usually, what we were doing - we were using the attribute called data behavior, and then we would scan these data behavior attributes as the events bottled up, and then if the attribute was a match, the data behavior was a match, we would trigger the behavior. Well, that does not provide very readable code always, I’d say. Sometimes we would provide a data behavior on a parent element, and then there would be specific DOM elements inside of that parent that would trigger behavior and we would catch these events and we would do something.

One of the things I really disliked about that was I wanted to get to the point where if someone opened up a piece of the HTML, they could see what was going on, that it didn’t have this shadowland of event handlers living somewhere in some JavaScript files far, far away, so you couldn’t tell what the HTML actually wanted to do. I wanted it to be explicit, such that “Hey, if you click this span, if you click this button, you know what’s going to happen.” That’s the concept of the actions. We make those explicit in the HTML itself; we don’t hide them away as event handlers in the JavaScript code. I mean, underneath there’s an event handler and that’s what Stimulus provides you. It provides the plumbing to do that. But what I’ve found was a lot of JavaScript code was very, let’s say, low-level. Yes, event handling is the way that we process this way and we deal with the UI, but that doesn’t have to be the way we actually write it, and it certainly isn’t the best way to provide understanding of how a system works.

So now you can do data-action on a button or a span or an input or a submit button or anything else that you want when the user clicks that or submits that or hovers on that, or some other sort of event trigger - you can trigger a specific action, and that’s spelled out. So it’s data-action equals, for example, click points to hello@greet. So you read what that button does. You don’t actually need to read the code, you can read “Okay, if a user clicks this button, I’m going to call the greet method on the hello controller.” That’s super explicit, and it means that most of the time I don’t even need to look up what the code actually does, I don’t need to look up the underlying JavaScript controller to understand what actions are available on this view, and I found that that was just so liberating.

[31:52] The second part of that was that just like data targets, the actions were generic and they could be moved around. So if we currently have an action on, let’s say, a button, and we move that action to a link, the controller could remain unchanged, because the controller didn’t actually care or know what type of element invoked its functions, which again is quite different from when you marry the HTML structure with your component and with your dynamic behavior. Then you’re kind of locked in step and you can’t reuse these things and you can’t move them around. So that seemed like a big advantage.

Then finally, the last concern or the last concept is this notion of the controller, which is basically an encapsulation of all the behavior that relates to one aspect of one feature up the system. That’s where we basically just use JavaScript classes, and these classes have methods on them, and these are the methods that we call through the action triggers, and those actions and the methods that they correspond to interact with the targets that we’ve made, and that’s it.

There’s three basic concepts, and it doesn’t take a long time to learn it. Even without learning it, you can read the code, you can read the HTML structure and you can understand what’s going on, and that’s really all we needed. In fact, I was kind of shocked when I first started extracting this stuff; I was kind of thinking “Oh man, there’s gonna be so much stuff here” and it was the real epiphany of seeing that those three basic concepts - the controllers, the targets and the actions - were enough to extract such a wide body of behavior from Basecamp.

Going back to the targets for a second - I guess I have a comment and then a question as well… I guess the comment would be that you seem to have formalized – honestly, David, this is what I love about what you do in the open source community, because you conventionize things that often times people are doing informally… Because I’ve been doing a very similar thing for years in kind of a half-hacked way, which is trying to balance between selectors, classNames, dealing with the design side, dealing with the functionality side… So what I’ve been doing for a long time is just using like a js-prefix on a className, and saying js-, and then you’re basically doing a target. That is then a signal to the designer that “Okay, this class does not have to do with the look and feel. This is a JS-specific thing”, so therefore they won’t get changed based on someone trying to change the way something looks.

Then secondly, it also signals that this thing has some JavaScript attached to it somewhere, because like you’ve said, we’ve detached the handling to some place in the sky, and so there’s a little bit of a signal there. But I like the data-* way of going about it; there’s no drawbacks… And I guess the question comes in, when it comes time to actually modify things - so you mentioned BEM, and the problem is classNames pulled out from underneath you or what have you… These are issues, and this is genericizing that and pulling it out of the className and putting it onto a data attribute… But how can you genericize it when it comes time to actually modify the elements? For instance, “This click handler actually hides an element, or changes it to a variant that’s a larger version or a smaller version.” Aren’t you still dealing with classNames in terms of the current look and feel of elements, or is there somehow that you guys are also genericizing that so you don’t have to worry about the BEM classNames in your controllers?

I love this question, because this is literally the fourth concept that we’ll be introducing shortly and that I’ve been working on for some time. It’s exactly as you say, we do almost all of the mutation of existing elements through classes, whether we want to hide something - well, that’s just adding in a class that hides it; or we wanna show something, or we wanna play an animation, we use CSS animations… Classes are really the way to mutate the DOM the vast majority of the time. And those classes are explicit, right? They are BEM-ed, or whatever they are…

[36:14] Exactly.

… and you shouldn’t let those BEM-ed classNames leak into your code, because you don’t want your designers to have to open up some JavaScript controller just to change a BEM class to add a bit more padding, right? So what we’re going to basically do is a similar construct where you can declare a data attribute that includes the BEM class in the HTML, which is where the designer would be changing it around and then you can reference that BEM class in your code and say “Hey, I want the CSS class that’s for hiding. Give me that class” and then apply it to this element. We return it to this sense of genericism.

We’re gonna be working on that next. I’m hoping that Stimulus 1.1 will include the abstraction of classes into this structure, such that we can use logical classNames that make sense for the code, rather than concrete classNames that make sense for the designer, that adhere to BEM, that do all these things.

Does that mean that the className will get set through a data attribute?

Exactly, yes. So there will be, for example, somewhere on the controller, so they can be changed by the designer in the HTML. So the designer does not need to monkey around with the controllers, at least as long as it’s a stable set of logical classNames. If they add additional classNames or whatever, you might still have to have some involvement, but in many cases, especially with BEM - BEM has this idea that you can mutate and you can combine a single presentation of an element through a mutation of one className.

That is really something we want to abstract and get away from. There should not be BEM classNames inside of a controller. They should be logically referred to.

It would be rad if there was some sort of declarative way that you could set that all up upfront, maybe in one place, where you could map these things, whether it’s just a map in your JavaScript code or something, or it’s like these logical representations… Because there’s a handful of them; like you said, you could cover 80% with defaults, and these map to these particular classNames… Just so that the designer is not necessarily having to add data-* className to all of the elements all throughout the HTML.

Well, you only have to do it for the CSS classes that you need to dynamically apply, remember that. Most of the CSS classes you don’t need to do this work with. It’s only for the ones you need to dynamically apply to something, like if you have a specific class for hiding an element or something else like that.

I don’t actually think that in most cases you can go generic with them, especially if you follow BEM. A specific presentation of one feature might have a logical className for hiding things, but the concrete implementation in BEM for hiding something might be slightly different than it is somewhere else, because it adds or removes padding or margin in some ways.

We’re still sort of feeling that out for now. I’m pretty confident that a huge step forward would simply be to go abstract with the CSS classes you need to dynamically apply, and then declare either the same element that holds the controller name, or on the target or somewhere else where it makes sense. Just disconnecting these two things so we get on two different trains of changing them.

It’s interesting that, Jerod, your version and David’s version is essentially not the same, but it’s very explicit; you were prefixing JS, and David’s solution is reusing the concept of data attributes, and again, being explicit and saying “This is different, this is not your normal class. It’s for a special purpose.”

Yeah.

[40:11] I think it’s coming from the same motivation, right? We want to be able to read a piece of HTML and know what it does. We don’t want it to sort of just magically have things happen to it because far, far away there’s some JavaScript class that declares an event handler that just happens to match this thing. We want the explicitness and we want to be able to read it. I think that’s just good code practice. So whether you do it one way or another, as you’ve found, there’s just a clear motivation to solve that problem.

Yeah, and I think I was trying to get back to something that we lost when we went from writing your click handlers right there in the HTML, which is decidedly too low-level to be in your markup… Moving away from that, when we switched to that style of markup to “Now we’re going to put everything into jQuery click handlers” or what have you; we lost that connection. Now there’s a lot of people rightly saying “Hey, this is actually a step backwards, because now you have random things happening that you don’t know about. There’s no connection in the code, so this kind of bridges that gap and puts a nice happy medium in place.

Which really comes back to why we’re doing Stimulus, and why aren’t’ we just picking an existing framework that’s out there. There’s certainly plenty of them in JavaScript land already… And I think some of it is because I just saw regressions; I saw that we took certain steps forward in certain domains with this new set of JavaScript frameworks, and then we took huge steps backwards in all sorts of different ways, that apparently people just didn’t care about - and that’s fine; we don’t all have to care about the same things… But I cared about those things, and I could live with those regressions. So Stimulus is a way to not take those regressions. Get back to in some ways a simpler time, which is – I mean, it’s always dangerous once you get into arguing what’s actually simpler and what’s not simple…

Right, the good old days…

…because a lot of it is tied up into – exactly, first of all, good old days. The good old days were often not that good. And second of all, just that we have different applications and they work in different ways. If you’re trying to make this very intricate UI with tons of related things and blah-blah-blah, then maybe these heavy frameworks do make sense. If you’re trying to make an application like Basecamp or GitHub, they don’t make sense and they are overly complex for what we’re trying to do.

And I just felt like there was an under-representation of frameworks that were trying to tackle the class of application that GitHub and Basecamp finds itself in, which is not a small class. I would argue, in fact, that it is the larger class, and it’s been severely under-served by this recent advent and explosion of JavaScript frameworks; they’re all focused on the same paradigm, that the server-side was now just tasked with producing JSON, and then you would have some client-side agent that would take that JSON and turn it into an HTML and build up a DOM, whether virtually or otherwise, and… All the frameworks are just the same, which is why I was sort of chuckling a bit when there were such furious wars between like “Or, are you using React or are you using Angular or are you using whatever…?” If they all come through the same paradigm of server-side generated JSON that’s then dynamically translated into HTML. I mean, come on, guys. It’s the same idea, right? We’re really furiously arguing about the small details.

Stimulus provides a completely different paradigm, a paradigm where the server continues to create the entire HTML document, and then we sprinkle this remaining behavior that we need onto this through a progressively-enhanced approach. But even when we do dynamic stuff - so we have Stimulus controllers, for example, that will trigger a behavior on the server side; what the server side will return is a fragment of HTML. We use HTML as the transport protocol. Very rarely - although sometimes - we will use JSON to do it.

[44:22] So when we want to update a part of the page, we ask the server-side, “Hey, can you give me this fragment of it?” Which allows wonderful things like fragment reuse, which you’re familiar with. In Rails we call those partials, and the fact that you can use the partials, the fragments of the HTML, both to render the initial version of the page and the subsequent updates, is a huge step forward.

That’s interesting. I’m curious about this. I know that in Ruby code, David, you like to remove comments and rewrite code, or replace comments with more readable code. And in your process of evaluating Basecamp and looking through things of this transition to Stimulus, I’m curious how much of that happened in your code, like how many comments were joyously removed in replacing with this prototype version that you created?

A fair amount of it, and I think that that is exactly the attraction to adorning the HTML with this very explicit tie-in with how things are called… Because when you have a button that says – it has a date or action where if you click this button we’re gonna call the greet action on the hello controller, that’s incredibly self-documenting. You do not need a code comment to explain what’s coming on. And on the controller side, the same thing, right? When you have a declaration of action methods upfront, you don’t need to comment on how those are hooked up. You’re relying on the conventions that Stimulus affords you, and those are already documented in Stimulus, so we can really cut down on the amount of needless documentation that we need in our application code… Which for me, whenever you have copious amounts of code commenting, it usually tells me two things - either that you wrote convoluted code (that’s probably the most common); 2) that you have a set of conventions that you’re just still waiting to extract, and then once you extract these, you can remove all this repetitional commenting… And these are the driving motivations for why we want to get rid of them. Why? There are code smells.

In some rare cases, what you want to do is actually just counter-intuitive, and sometimes it’s basically around browser bugs, or something else. You’re doing something that does not make sense if you just read it, and you need a code comment to explain, “You know what? That’s because i.e. Edge does something stupid here, and this is why we need to do this monkey dance to make it happen.”

Right.

But even in that case, I often find that you can still encapsulate that monkey dance in a method that succinctly explains that it’s because Edge or Safari or Chrome or whatever is doing something that you need special consideration for.

Alright, David, you said what you saw was a lot of regressions, and maybe even side-steps in certain cases… One of the things I remember back in the day when Backbone.js first hit the scene - because this was one of the very first front-end frameworks that specifically said “Get the state out of the DOM. Get it into JSON, get your state out of the HTML.” And that was a step, and everybody started doing that in different ways, so it began completely detaching the front-end and the back-end. And like you said, Stimulus offers really a different opinion or a different way of going about building applications than a lot of the other front-end frameworks out there, and one of the things that it says on the home page is “State is stored in the HTML, so that controllers can be discarded between page changes, but still reinitialize as they were when the cache HTML appears again.” So that’s definitely a big difference from other things out there.

Can you tell us about how that state is stored, how you deal with change? I know you mentioned it a little bit during the last segment, but let’s go a little bit deeper into how that all works.

Sure. So we store state in much the same ways that we declare the targets and the controllers and so on, through data attributes. We basically just set these data attributes on usually the root element of that controller, and that stores the usually minimum amount of state. I think one of the reasons Backbone and other frameworks have argues for extracting state into something else is because maybe they had a ton of it.

We try not to have a ton of it. We try to have very little state within the DOM itself, but just enough so you can reinitialize a controller, and it can come back to the form that it was. For example, a state could be – we have a collapse controller that allows you to click a certain element to open or close another element, right? Like a Show More, or See Less, or whatever. And the state of whether that is open or closed is something you can store in the DOM. In that particular case, often times the state is actually just an application of the classes. Classes provide usually most of the state that we need.

For example, for the collapse example we will apply a hidden class when the element is closed, and we will remove that hidden class when the element is open. That right there will store the state in itself. We try not to duplicate or create a shadow state of what the DOM already has, because we’re trying to enable you to get HTML from anywhere. Most of the time this HTML is coming in the form of the initial render, and the controller simply has to take that initial render and instantiate themselves based off that.

Then if you have other updates, it could be websocket updates; you have a websocket channel that inserts new HTML into the DOM. When that new HTML is inserted, it needs to include its own state. We don’t have something else to also pass that state along with. As I said, we use HTML predominantly as the transport layer, not JSON. So the transport layer has to include its own state, and that’s where we found this is actually a nicer way of doing it.

Then just the fact that we go back and forth between pages - we’re using this together with Turbolinks, and Turbolinks stores a nice cache of the pages we’ve been switching between. So if you have controllers that have been mutating the DOM, usually just by adding CSS classes or whatever, Turbolinks will remember the state of that when you go from one page to another. So we get to sort of keep that state alive through the Turbolinks cache, and that’s adequate and actually a useful constraint in most cases to not go hogwild with a whole bunch of intricate state.

[54:24] I suppose that works just fine without Turbolinks, you just don’t get the advantage of the fast refreshing at a full page’s reload.

Yeah, and you’ve just gotta be a little careful with those full-page reloads and what the browser ended up caching… Whether it ended up caching the final version of what the DOM looked like, or it’ll reinstantiate it from scratch. Turbolinks helps a little bit there in terms of making it more fluid and making it easier to keep that state cached.

So were there specific changes that went into Turbolinks to support this? I know you called a one two-pack punch, or something like this… I was wondering if that just happened – they just paired well nicely, or if Turbolinks requires some specific upgrades or enhancements to actually support Stimulus natively?

It didn’t, because we were already basically writing Stimulus before we were writing Stimulus. This approach of using progressive enhancement and storing state in HTML, and using HTML as the transport layer - that has been our pattern and our paradigm for a long time, and we built Turbolinks originally with that paradigm in mind. So Stimulus is basically just an encapsulation of that paradigm, and packaging it up in a nice way. That was really the missing second punch to Turbolinks. We would pitch Turbolinks to someone, like “Hey, this is this wonderful thing that can actually cut out 80% of all this dynamic behavior you’re doing, because it’ll speed up the page changes to such a degree that you don’t need anything else.” There’s a lot of dynamic behavior we’re doing for performance reasons, because it feels too slow to do a full-page change, that you no longer need when you’re using Turbolinks because the page changes are really fast. So you can just send the whole page again, even though you’re making a relatively small change, in a lot of cases.

But then there was still the last 20% where you didn’t wanna do that, right? You had some small change, like a collapse show thing, as we’ve just talked about… It’s a little excessive if you’re showing or hiding, which is basically just applying or not applying a CSS class to an element required a whole roundtrip to the server, and sending down all the HTML for a whole new page, just to apply a single additional class. That doesn’t make sense. That’s not proportionate, and it’s not gonna be fast enough to feel really good.

So that was this missing grey land, the last 20% of behavior where we kind of just waved our hands and said “Um, have you looked at MutationObservers?” and then left it as an exercise for the reader, which was actually a fairly large task for someone to do. So I could see how not having a clear answer for that last 20% held back Turbolinks in some ways. So I’m really happy that we now have Stimulus to provide 100% of the answer for applications like Basecamp, and if you wanna write them in this way, you now have all our tools; there’s nothing hidden under the carpet here. Everything that we use to write Basecamp the way it is today is open source, packaged up as an easy-to-use library or framework, and the story is now complete… And I think that that’s really important - if someone is looking at their application and they can’t visualize how they’re going to solve this specific part of it or this specific feature, it’s hard to gain adoption.

[57:35] I think Turbolinks suffered for some time with that… It also suffered for other reasons, for example this notion that apparently it was hard to understand the concept of a persistent process, and that you couldn’t just drop in any jQuery plugin that was written with the idea of every page change blowing the process and the instantiated application away, even though that was exactly the model that people had been following with single-page applications in the heavy frameworks, right? They all are run in a persistent process where every click of a link is not a full reload… But for whatever reason I think a lot of people just saw like “Hey, I should be able to just use Turbolinks in any odd jQuery plugin that I can find on the internet” and those things need to just magically work together out of the box… And they sometimes just didn’t. You had to do special considerations to deal with the fact that Turbolinks does not change the full page, right? It gets a whole HTML document, but then it does the updating of that document in process, and it keeps the state of the JavaScript around, and it keeps the interpreted CSS around, and that’s where it gets its super speed, its turbo speed.

Turbo, yeah. [laughter]

Exactly. At Basecamp we just didn’t use a bunch of JavaScript or a bunch of jQuery plugins. We wrote most of our code that we needed ourselves. And sometimes we used some jQuery plugins, and we just altered them, so they were compatible with Turbolinks. That never seemed like such a big deal to us, because the benefits that we got back were so monumental, and what it allowed us to do especially in terms of the majestic monolith and applying a simple application that was automatically updated across five different platforms was just such a huge win that it was really hard for us to imagine that anyone would look at the small changes or concessions that you had to make to get those wins and go like “Oh yeah, that’s not worth it.”

Anyway, I mean, Turbolinks and Stimulus - they’re very much for writing for ourselves. I’m doing these things because we need them in Basecamp, and when someday that day is going to come when I write Basecamp 4, I want them to be available and packaged in clean form so I can just use these frameworks off the shelf and get on my merry way. And if someone else ends up using it, that’s great. And if not a lot of people end up using it, that’s also great; I really will continue to do it. But it’s sort of the same approach I have with Rails…

When I originally wrote Rails, I wrote all of it. I needed a way to talk to the database - well, I wrote Active Record. I needed a way to render templates - well, I wrote Action View. So at Basecamp we have a tradition of writing our own tooling, and then we share our tooling more out of gratitude to the rest of the community for the tools that we do use, and just because that’s a nice thing to do, and sometimes it ends up taking off, as in the case of Ruby on Rails, and we get some benefits from that. And if it doesn’t take off, that’s also fine and we get to use them.

I think that’s really a sort of ambivalent or distant relation to the open source process, that we open source because we can and because we like it, not because “Oh, it has to gain adoption.”

Right. Just hypothesizing a little bit about Turbolinks, because I’ve been around I think for the length of its run, and I’ve seen the community reaction to it over time, and as you know, we were fans of Turbolinks, we opted into Turbolinks with a non-Rails app, which I think is somewhat unique… But I think some of it had to do with, like you said, that problem with existing jQuery plugins and the fact that Turbolinks was such a plug and play aspect of a Rails application; you could literally just comment it in or out, and it would or would not do everything for you. In terms of Turbolinks itself, it was just so easy to flip it off - pun not intended, but perhaps it should have been - so easy for people to just use Turbolinks as the scapegoat.

If it’d be on, this JavaScript issue doesn’t happen; oh, it must be Turbolinks. So it just got a bad name right off the bat, just because of the existing ecosystem it came into. And then you know, it was kind of like the Siri situation - it gets this reputation upfront, and then over time people kind of just naysay it or don’t give it a second look. But maybe through Stimulus people will give Turbolinks a second look, or maybe not. But like you said, you’re ambivalent to it, so…

[01:02:11.09] Well, I think also a part of it is you have to understand what it’s doing for you, and I think perhaps that was the drawback of having it be so easy to turn on and be included by default… If you don’t fully understand the benefit that you’re getting, you don’t understand the tradeoff. And if you don’t understand what benefits you get, any cost is too high. If I perceive the benefit as nil or nothing, then if I just have to pay just a modicum of work myself to get it, I’m gonna say “That’s not worth it for me.” And I think we’re perhaps getting to the point where more people are realizing that using these heavy-duty client-side frameworks, “Oh, wait, they also have costs”, and they also have in many cases towering complexity, and I think we’re getting some veterans that are coming out of that process that go like “You know what, if I’m gonna build another thing, I’m not gonna do it like this again. That was just painful.” I think that pain is exactly what we try to address. And until you’ve suffered that pain, I don’t think you can fully appreciate the salve that we are offering, the band-aids that we are offering… Which in many ways was the same way that Ruby on Rails came to be. There was so much pain that a lot of people had experienced using Java frameworks or PHP without any frameworks that they were very in tune with the pleasure that Ruby on Rails could bring them because they knew the pain. I think until you know the pain, you don’t have space in your brain to appreciate or even properly evaluate the solutions to it.

I think everyone should go and build a full application in the heaviest duty of JavaScript frameworks, regardless of whether the application warrants it, just so they can suffer through it on their own skin and come out scarred and battered on the other end and go like “You know what, maybe there’s a better way.”

That’s the bitter and sweet, right? You can never really understand the sweet goodness of the chocolate bar unless you’ve had that nasty piece of candy after dinner, or whatever. You’ve gotta have the bitter to enjoy the sweet.

That’s right.

Yes, and I think that that goes for all sorts of learning. I think that’s why lists of best practices for example, divorced from the pain from where they arose, often don’t make a whole lot of sense or they don’t stick. People aren’t ready to internalize lessons until they’ve encountered situations that really demanded those lessons in flight. I think that’s just part of the learning process; you can’t appreciate everything up front. I think plenty of people end up going off to college and they end up having all sorts of courses in philosophy or whatever and they can’t apply them to their experiences of where they are in life, and they go like “Oh, this is worthless. What’s existentialism about? I can’t use that for anything”, and then a decade later they go, “Oh, wait a minute… Let me hear what Camus has to say about the meaning of life”, because they’re at a different station in life.

I think there’s a lot of technology that works like that - it doesn’t really reveal itself until someone has suffered through the long road.

Something that goes to teaching and documentation, and one of the things that is very difficult to do through readme’s and docs, and even blog posts - blog posts are a little better, but they’re so hard to find over time - is like “What were the circumstances in which this solution came to be and why does it exist?”, which is some of the gaps that we try to fill with the Changelog and shows; conversations with the people to give that historical context… Because there are no panaceas, there’s no silver bullet, and all of these have trade-offs and all of these have reasons why they were created.

[01:06:03.10] So if you lack that historical context as somebody who’s coming to a Stimulus or coming to a React, and just picking the tool off the shelf based on the readme, and you don’t understand the historical context in which those tools were developed and why they exist, then you’re basically doing a coin flip, and you don’t know how you can apply it to your given circumstances.

So it’s tough… Like you said, sometimes you just have to live and learn, you have to just go through it and realize it, but I think we can work together to give these historical contexts to people, so they’re more equipped to make those decisions.

I think that’s spot on. I think there’s so much technology that’s presented just as the how, not as the why, and it’s that why that gives us the context to evaluate whether this is a good fit for us. Can we see ourselves (the person who developed this solution) and their troubles in our troubles? That was one of the reasons why when we introduced Stimulus we did it with a document called “The Origin of Stimulus”, which basically walks through “Why did we extract this? Why did we make this?”, and tells the story of the Basecamp code and our journey of making a Majestic monolith, and how we use Turbolinks together with Stimulus, why the concepts make sense, which – one of the problems, as we’ve talked about here, right? Like the problem of using CSS classes for targeting - it’s brittle, it’s all these other things… It’s not just like “Oh, here’s how to do that”, right? And I think that that’s often missing, and I think sometimes people sort of evaluate things from the wrong perspective.

One thing I’ve heard a lot of times is “Oh, I wanna use React because Facebook is using React, so it’s gotta be good enough for me.” I’ve heard that with a lot of other pieces of technology. “Big company is using X, thus it must be good enough for me.” I actually think it’s often the exact opposite. A lot of the patterns and even outright technologies that large companies use are the worst thing you could pick when you’re just starting out, or if you’re a single developer or a small team, because these things are designed to work with much larger teams, in much larger companies with all sorts of different considerations and specializations, and a stomach for a different level of complexity. When you’re trying to serve a billion and a half people, you just have different problems than what we’re trying to solve at Basecamp.

We’re trying to solve servicing a few million people at the most, right? It’s different orders of magnitude, and the correct and applicable patterns and practices that are relevant for someone trying to solve for a solutions base of a few million people less is just very different from the kind of people like Facebook, who have tens of thousands of people working on the product and are trying to solve for a billion and a half. And I think sometimes people just get enamored with this “Oh, I wish I was Facebook. So if I just start using their toolset and their methodology maybe I’ll become Facebook.” No. If you looked at any history of actual Facebook, do you know what their code originally looked like when Zuckerberg wrote it? I don’t think there’s any of those practices left anymore at the company, because they evolved and they turned into something else. But if Facebook had started out with the heavy duty patterns and practices and methodologies that they’re using now, if Zuckerberg would have had to do all those things as just developer one Facebook would never have happened.

That’s right. I have a counter to that, though. Twitter used Rails, and many people use Rails.

I think Twitter is this great example, a great scarecrow, a great reminder that there’s so much more than technology to whether someone fails, even technology-wise or not. Twitter in the early days of the Fail Whale blamed Ruby on Rails for its trouble, because it was much easier to blame an external vector like Rails than their crappy architecture for why the site kept falling over.

[01:10:15.26] And then just recently, a few days ago Twitter - or some former executive from Twitter - through an article in Vanity Fair, blamed Ruby on Rails for the fact that Twitter ten years into its existence has still not dealt with harassment and abuse in a proper way, on Ruby on Rails… Which is just wonderful. It’s a wonderful anecdote of how humans are so desperate to diverge and deflect blame and accept responsibility for their own actions, and they were just trying to find any scapegoat.

It’s all your fault, man.

Exactly, right?

If you hadn’t released Ruby on Rails, they would have never had this problem with harassment.

And actually, that is true, right? That is actually true, in some sense, because maybe Twitter would never have existed, or it would never have taken off, or it would never have gotten done in time, or they would have run out of money, or something else. So in a way, I think Ruby on Rails is implicated in the harassment problem at Twitter, because Ruby on Rails helped Twitter get started and helped Twitter get off the ground.

The fact that maybe it’s been 10+ years and they’ve still done so poorly at addressing the fundamental problems of harassment and abuse on the platform - maybe that blame falls elsewhere, but anyway.

Let’s move into something I guess a bit more promising. So you’ve had your hand in obviously writing frameworks, we know that… You do some great writing on Signal v. Noise, you’ve got a podcast, you’ve written books, you race cars, you’ve got kids… You’re like everybody else, you’ve got all these cool things, but next up is YouTube for you. You’ve got this cool new channel maybe not everybody has heard about yet, but I’ve been enjoying it, and one thing I think is pretty interesting is that you get this chance to essentially sit down with you and look through the Basecamp codebase, and you’re just sharing all the reasons why you’ve done what you’ve done. Can you take a moment and just kind of share what your plans are with that channel?

Sure. The channel is called “On writing software well”, and it’s just on my YouTube. It’s basically just me opening up an editor and taking a topic that could be testing, or callbacks or whatever, and showing how we use that in Basecamp, and showing how we use Rails and Ruby to solve the problems.

What I wanted it to feel like was if I sat down with another programmer and we just looked through some code together - I always love doing that, because I find that many programmers when they’re talking in the abstract about code and patterns and so forth, they have these fierce arguments: “No, this is the wrong way of doing it! This is the right way of doing it!” And then if you sit down with them and you look at the actual code, you end up agreeing way more often than not, because the pressures and the concerns of a specific piece of code guides most reasonable people in a similar direction, at least when they have somewhat of a shared background and experience. There may be functional programmers who are like “Oh, anything object-oriented or side effect-laden is wrong” and whatever, you’re not gonna find common ground with them perhaps, but for anyone who exists in the same paradigm and somewhat have shared beliefs, if you look at concrete code, we end up liking the same things a lot of the times, a lot more often than if we just argued about it in the abstract. And this is one of those lessons that I’ve learned time and time again. Ruby or Rails back in (I think) 2009 merged with another Ruby Framework called Merb. And Merb was born for a lot of different reasons, and some of the reasons were that the people behind Merb cared about different things than what I cared about; not that I actively didn’t care about them, they just weren’t top of mind. There were some extensibility concerns that they had and some performance concerns that they had, and we thought we had these fundamental, underlying philosophical differences about how to write a framework in Ruby.

[01:14:08.05] So I sat down with Yehuda Katz in particular, who was one of the guys involved with Merb at the time, and we had these fierce debates when we were just chatting in Campfire, and then we sat down, looked at the same piece of code and went “Oh yeah, we believe the same thing.” And we’re like, “Wait, what?!” We were just arguing our heads off in opposite directions, and then we looked at a piece of code together and we came to the same conclusions. I’ve done that so many times now that I believe that it’s really the primary way you should be arguing code patterns and principles - by looking at actual real production code, and doing A/B’s. “Let’s write it your way, then let’s write it my way, then let’s see if there’s one or the other ways that’s the best, or more likely that there’s a combination of the two ways that turned out the best that we both like.” I find that that happens just all the time.

So I wanted the YouTube channel to have that feel as though we were sitting down at the keyboard together and looking at code together and coming to similar conclusions. That doesn’t mean – I mean, everyone is not gonna sit down and watch these videos and go like “Oh yeah, I would have written it exactly like David would have written it”, but at least if you hear the why, why we wrote it that way, how we weighed the tradeoffs and came to certain conclusions, you’ll understand why we did it the way that we did, and I think that that understanding is often sorely missing when people are talking programming and talking shop.

That’s why the arguments get so heated. We start having these violent disagreements that in many ways are completely unnecessary, completely unjustified by actual code… And it can’t be solved or it can’t be addressed just by looking at example code. It can’t be addressed by the stylized, idealized version of what programming looks like because until you have all the real constraints and pressures of production code, you’re not taking all the complexity into consideration, and often times that’s exactly what tips the scale as to whether to go one way or the other way - to look at the real code.

For example, I did an episode on globals. Rails 5.2, which is just about to be released, has a new encapsulation of globals for dealing with things like current used and current account within the lifecycle of a single request. And if you just sat down and had an abstract conversation about globals with a programmer, most programmers would say “Well I learned/heard/know that globals are considered harmful. Why are you using globals? Globals are terrible? Don’t use globals. Are you a bad programmer? Are you terrible? What’s going on here?” [laughter] And then you sit down and like “Yeah, all those things have strengths of truth in the aspect that globals are dangerous and you do need to be careful, but hey, let’s look at this code.

Let’s look at how much simpler and easier it actually becomes to understand once we use a global. It’s not a thing you should use all the time and in all circumstances, and you can certainly get carried away with it, but in this particular instance, using the feature on this particular aspect of the code, it gets better. This A is better than that B.”

I think that those are the concrete tradeoffs, as I said, that are really just fascinating, and I think it opens people’s minds much more than just a blog post that says “Global is considered harmful.”

That’s one thing I’ve really appreciated about this series with you, is that you explain your preferences, and then you look at the code and you say “This is why I’m breaking my own rule. This is why I’m putting these methods out of order in the private method, rather than as a table of contents, like you normally would. This is why, because it reads better, so I’m willing to trade off and go against my typical grain for these reasons.” You kind of get a chance to step into your mind, watch how you program an actual application called Basecamp, you’re looking at actual Basecamp code. I think that’s a pretty interesting concept and I’m glad you’re doing it.

[01:18:09.02] That is really I think the pivotal thing, the nugget, the justification of it. The fact that there are all these principles and patterns and best practices of how you should write software, and in isolation, they all make sense. But when you write a real application, usually all of them disagree. There will be one pattern that tells you to do things this way, and then there’ll be a best practice that tells you to do things in the other way, and you have to weigh these things and consider “Which one is more important in this particular instant, and which one will I put more weight on?” and that’s where the wisdom is hidden.

I find that a lot of programmers apparently seem content to just learn the recipes, to just learn like “Oh, I can recite all the patterns. I can recite Solid, I can recite the Law of Demeter”, and then they don’t really know what to do when these principles are in conflict with each other, and that is really where actual code is written. It’s written in conflict, it’s written under one pattern and one consideration pulling in one direction and another pulling in another direction, and you have to carefully weigh - and sometimes subtly so - which you’re going to put more emphasis on.

I agree with everything Adam said about the saw. A quick suggestion or a feature request for upcoming – and this might be a little harder for you to organize, but it’d be cool if you would sit down with Sam or with Javan or with somebody who has a different opinion about even Basecamp code, and you guys could talk through a refactoring, or talk through things where it actually is a dialogue. It might be an interesting alternate style format for this show.

Pair programming.

Yeah.

Yeah, I’d love to do that. I think that is basically what I’m trying to do. I’m having a dialogue somewhat with an imaginary programmer sitting next to me, and the two minds -my own and my head, right?

Right.

But part of it too with this – so I’ve produced five episodes, two hours of content or so, in like a week. The reason I can do that and the reason I can churn these things out is because I do one take, and I just flip on my browser, I load up a couple of tabs of code, and then I’ll just freewheel it. That is crucial to why this is happening.

If I had to put in diligence and preparation work that perhaps it would take to include multiple participants, I just couldn’t do it at this velocity and perhaps it’d be harder for me to maintain the stamina to do it at all. I think that’s why it’s kept me back from doing this before, because I thought “Oh, it’s gonna be big production, it’s gonna be a big thing.” No. I mean, most of these episodes have like literally 5-10 minutes of prep work where I’m picking out the files I wanna talk about, and then I hit record, and then I push Stop when I’m done, and that’s it.

Yeah…

It’s a tradeoff.

In a good way. It’s a pretty simple production process for you.

I think ‘production process’ is even a very fancy word for it. I mean, I’m hitting Record and then I’m hitting Stop, and that’s it. There’s not a lot of post-processing or second takes.

Gotcha. Was there anything else to share about the future of Stimulus, maybe the next version of Rails before we close out? Any sort of planned convergence? I know that Rails does a lot of generation in terms of scaffolding; any plans of Stimulus being baked in and HTML going down the pipe with data attributes? Anything left unplugged?

What is coming is greater integration with Webpack. We already have wonderful integration with Webpack through a gem called Webpacker, which gives you a way of using Webpack in a Rails application in an easy way… And Stimulus is made for that. Stimulus is made for use with Webpack, and sort of – you can use it outside of that; there is a compiled version, you can do that, but most people most of the time will use it with Webpack or something similar. That’s the direction we’re moving towards. We’re moving away from generating or compiling JavaScript through the asset pipeline…

[01:22:13.23] The asset pipeline goes all the way back to I think 2008 or something, where we had to build our own tooling for compiling JavaScript, because there just wasn’t a good ecosystem around JavaScript itself to do it. Well, that’s totally different now. There’s actually multiple competing ways of compiling JavaScript, and many of them are very fine, and we’ve decided to pick Webpack as the default… And we should take full advantage of that, because that’s where most of the development is happening.

So the asset pipeline can kind of step back now that the job of compiling JavaScript is so eminently handled by the JavaScript community itself, and say “You know what, I’m just gonna hand this responsibility over.” So Rails 6, as we’re coming up upon after the release of Rails 5.2, is going to figure out a way to make that blend beautifully, and have an emphasis and a focus on Webpack out of the box… Which then also means a focus on making it super duper easy to use Stimulus out of the box, because you’ll basically just be able to say “Rails, New, My App –Webpack = Stimulus” and we’ll set you up with a default scaffold for using Stimulus… Even though there’s not that much scaffolding to set up, but I find often times what keeps people back and stops them from using something is just getting Hello World, right? Just getting all the things wired up… I mean, the joke, particularly in the JavaScript community, is that you have to spend two days setting up your compilation pipeline before you even get to Hello World.

So Rails certainly wants to sort of assist you with that, such that you can run one command and you can see Hello World is already there, and then you can start filling in with your own application code. So that’s one area that we’re focused on… And just mending relations perhaps in some ways with the JavaScript community - I think I’ve certainly had a contentious relationship with JavaScript for many years, because I thought ES3 was kind of a crappy language, to be quite frank, for this kind of style (object-oriented focused) that I wanted to write, and that the ecosystem around interfacing with the DOM, and the differences between browsers just made it a miserable experience. Well, lo and behold, ten years makes such a difference, and things are different now, and I’ve come to really enjoy JavaScript in many of its forms, and Stimulus is certainly one of those forms, so I’m just really excited to share that with the world and share a different paradigm, as we’ve talked about… From the “JSON over the wire is the only way to go” idea, to say “Hey, do you know what? HTML is actually a wonderful wire format and we’ve used it successfully for 30 years now, and it still has likes in it, and then some, and it has clear advantages on productivity and other things.

So let me promote that as an alternative path; I have no illusions that just because we promote this, it’ll turn into the path and that everyone will switch to it. I think there’s tons of momentum - and deservedly so - around a solution like React, and elsewhere. And we don’t have to win total domination.

[01:25:16.01] I think development communities sometimes have this mistaken notion that it’s all about market share, and it’s all about picking a winner. To me, there’s just such wonderful beauty in the diversity of the web. The fact that the web was this first platform that allowed us to pick any kind of implementation language that we wanted on the back-end, that you could write it in Perl, Ruby or AppleScript, and as long as it output HTML, the user is none the wiser… Which just gave a rise to just such a wonderful diversity, that we could have people of all different inclinations and brain shapes find exactly the language and the environment that spoke to them, and I hope that everyone has the opportunity to find a language and an environment that speaks to them as much as Ruby and Rails has spoken to me. I think we can carry that into the JavaScript land to a large extent now, because of the advances with transpilers and so fort… And so should we with the proliferation of different paradigms.

Well, David, thank you so much for your time today, man. I appreciate your willingness to experiment, to dive into Basecamp and extract patterns and come out with what is now Stimulus. And then your passion for open source, to release it because you can, and because you have gratitude back towards the tools that you’ve used. We appreciate that about you, so thank you very much.

Thank you. I can honestly say that it’s entirely my pleasure.