Rollbar – Our error monitoring partner. Rollbar provides real-time error monitoring, alerting, and analytics to help us resolve production errors in minutes. To start deploying with confidence - head to rollbar.com/changelog
Linode – Our cloud server of choice. Get one of the fastest, most efficient native SSD cloud servers for only $5/mo. Use the code
changelog2018 to get 4 months free!
Fastly – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com.
Play the audio to listen along while you enjoy the transcript. 🎧
Progressive enhancement was more an architectural technique of how to build great applications, which is actually similar to how I came to REST, and the REST principles for building HTTP applications. It wasn’t so much that I had this pure affection for representative state transfer and the thesis behind that. It was simply because it was a wonderful architectural pattern for guiding development and guiding how I should put an application together… And so it is with progressive enhancement - using HTML and having the server side generate that, and then sort of sprinkling little bits of dynamic dust all over the application. “Oh, this little button, instead of being a full page change, we’re just gonna have it dynamically update a tiny bit of the page.”
[04:06] It was really appealing to me, because the kind type of applications we build at Basecamp, including Basecamp itself, lends itself very well to that style. And I think that’s perhaps where some of the differences sometimes come in.
Basecamp and applications like it, including GitHub, that use a very similar style even if we don’t use the same frameworks, they have something called PJAX, that was a predecessor to Turbolinks, and that idea (which we can talk about later, too) - they actually go well together, Turbolinks and Stimulus; so that’s part of a two-pack punch. But in any case, applications like that, where it’s not about presenting a let’s say desktop-like, super high-fidelity, super-connected UI, where you’re sort of changing a little thing over here and then there’s five different other points that need to update at the same time… That was the world we were in, right? And some people are in a different world.
First of all, it didn’t fit obviously with progressive enhancement, but it also didn’t fit with what we were trying to do or made anything better. Over the years, we’ve had our dabbles in using frameworks like that, and it never ended up that the code was better afterwards. One of the techniques that I’m very fond of is doing VAB. You take a piece of real production code, a real feature, and you write it five different ways. That just tells the truth incredibly quickly. And the truth that I derive from that –
So did you try–
Sorry, go ahead.
In the process of time, did you try out like an Angular or a React and dabble with the Basecamp source code, and do these different things?
Yeah, I tried a bunch of different frameworks. Actually, just before going head into Stimulus, I did another broad search, essentially, evaluating all the frameworks that were already out there and saying “You know what, I don’t necessarily want to create another framework.” We have frameworks coming out the wazoo at Basecamp, right? I have my hands plenty busy with just Rails and Turbolinks and whatever. Is there a way that we can just adopt one of the existing frameworks, as we actually did with jQuery.
In any case - where was my train of thought here? Did I try these things, was that what you were asking?
I was down this rabbit hole…
[08:13] Well, you’re the kind of person who has your own thoughts, has your own viewpoints on elegance, especially when it comes to code, and it would make sense that maybe you wouldn’t try it, but it seems you did.
Oh, absolutely, because I get inspired by it, too. Oh, that was where we were coming from, whether there was another existing framework out there that could serve our needs… I mean, I’m sure there would be; it’s not like the most terrible thing in the world would be if we used React for some things, for example, right? I don’t have any fundamental opposition against that. In fact, I think React in particular had an incredibly correct core inside, that instead of maintaining this complicated sense of state, you would just blow away the world and rerender it every time. I think that’s a great inside and actually quite similar to our views on how to do things. It sort of then went downhill from there, in my opinion. That was the sparkling moment where I thought “Oh, there’s something really interesting there”, and then I started seeing the mismatch of smashing all the considerations and concerns around components into one big file. We’ve spent I don’t know how long trying to separate things out such that we can be on different rates of change and all these other good things, and now here we go, React, HTML smashed in with code, smashed in with state management, smashed in with everything, and I thought “Meh, I don’t think that’s actually progress.”
[11:59] They’ve made both Trix, and they’ve made Turbolinks, and it’s beautiful, wonderful code, and it’s really pure and wonderful… And then we had specific support sprinkles for a feature here, a feature there, and we had like four different styles of doing that, four different ways of attaching event handlers, and so forth… And someone coming in new to the Basecamp would go like “Which path should I follow again?” and then usually they’d just open some file that vaguely resembled what they were trying to do, and kind of follow that… Which wasn’t great, right?
You probably had heaps of CoffeeScript code at this point too as well, didn’t you?
We’re basically looking at things in sort of new horizons. When we make something new, we will make it to the best of our abilities, to the best concepts and whatever that we have, which today is Stimulus. So when we write new features, we write Stimulus controllers and that’s how we go. If we go back and revisit existing features and we have to update them in some way, we will sort of weigh whether the change is large enough to warrant a full conversion to Stimulus, or we should just leave it in place. Because the thing is you don’t really need to do this mass migration. The wonderful thing about having Babel and these transpilers is the fact that we can mix and match as we please. In fact, we do, even with modern code; Stimulus itself is written in Typescript, not in just vanilla ES5 or ES6. That’s because Sam and Javan enjoy using Typescript when they’re writing framework code… And I’m not a huge fan of sort of statically-typed or explicitly-typed (I should say) code.
So that’s not what I really wanna write and it’s not what we need to write when we write features, but they enjoy doing it for the framework itself, and since they’re doing the work, I’m like, “Of course you guys get to pick what you wanna do.” It doesn’t matter, it all just compiles down to the same thing in the end, so what do I care whether the parts of Stimulus that I didn’t have to write is written in Typescript, or CoffeeScript, or whatever?
There’s something to be said for having a general style in your application code perhaps, but these aren’t drastically different languages, right? They’re like dialects. I’m from Denmark, and in Copenhagen we speak one dialect, and I mostly still understand people from Jutland and the dialect that they speak. It’s actually not a big deal. I think people have a tendency to make it a much bigger deal than it really is.
That makes a lot of sense. I know as somebody who has dug into the Turbolinks source code, and I’ve written a fair bit of CoffeeScript back in the day, but hadn’t for maybe a year, 18 months… As a casual contributor or potential contributor, I maybe just opened a bug report or something, but looking at the Turbolinks code, which was CoffeeScript, I know that there was definitely some cognitive overhead, some catch-up I had to do in order to read that dialect.
I think on the other hand Typescript for libraries actually makes it easier to contribute because of the documentation available and the types being so obvious and stuff like that… Whereas CoffeeScript may have been a detractor for third-party contributors. I think Typescript might actually make it even easier for people to get involved.
[15:54] Maybe… [laughter] I’m not too convinced that that’s the main barrier to entry. In fact, I think that technologies have the tendency to overstate the differences in the technology choices versus the actual cognitive barrier there is to understand systems.
To contribute to, say, Turbolinks, there’s a fair conceptual model that you have to understand before you can contribute…
Yeah, that’s true.
…and I think climbing that hill - it’s a taller one than the subtle dialect differences between CoffeeScript and either ES5 or ES6 or Typescript. But to each their own… I think what’s just great is the fact that we don’t have to go through these violent transitions, where in order to have new features, for example of our application written in Stimulus, that uses Typescript, written in ES5 or ES6, we don’t have to rewrite everything that we did before that, because I think those types of transitions are incredibly painful, and I think in many ways it’s actually a blessing that we’re using transpilers all over the place, and it all just boils down to whatever you’re compiling to, whether that’s ES3 or ES5 or whatever your final output target is… It just matters less and less these days. I think that’s a real progress.
My role - I sort of just shipped it from “Okay, I built a prototype, and I have a very clear idea of where I want the final API to go and what the client code should look like, so let me provide that guidance and let me help sort through some of the conceptual issues that we then faced on how to design the API, especially around targets and so forth”, and here we are.
Alright, so let’s talk about the architecture or the concepts of Stimulus, how you designed it, how it works and how you use it.
So one of the things I really wanted with Stimulus was I wanted to solve a couple of specific problems or bad patterns that I was seeing in our sprinkles code at Basecamp. One of the first things I wanted to address was the notion of how do you find the elements that you want to mutate or work with? We had a bunch of different styles.
[24:07] Sometimes we were using a hierarchical approach where you’d say “Oh, give me the parent of the parent of the parent here. I know that the structure of my DOM tree is like this, and I know I want the third parent up.” That’s a pretty brittle way of targeting elements, right? Someone reorganizes things or puts them in a different way and all of a sudden you’re getting the wrong element. So that wasn’t a good pattern.
Another pattern that we’ve used was targeting elements by finding them through CSS classes. So we’d say “Give me the elements that match this CSS class”, which on the one hand seems okay, because a lot of times the CSS classes are explanatory. They say “Oh, this element is about the title of a person, or something”, so when you query for that in the code, it kind of explains what you’re trying to get, but it’s also pretty brittle.
So I didn’t like the ugliness of the code, and I also didn’t like the brittleness of it. But BEM often times – like, you add another –down or a –pad, or whatever it is that you add to a BEM class to ever so slightly tweak it; well, the designers were doing that. Well, again, it broke the code. So that sense of brittleness I wanted to sort of get away from, and that’s where the concept of targets came up.
Basically, Stimulus implores you to only find DOM elements you want to work with through the concept of targets, which is basically just an explicit name that says “This element is going to go by this logical name that belongs to this controller.” And then we can move that name around and it’s not tied to this specific type of element; this could be an input element, it could be a button element, it could be spam element, and as long as it has a data-target that’s of a certain thing, you can always find it… Which not only gives you this sense of clarity around what elements when you read the HTML code are actually used by the dynamic behavior, and in the code itself, it’s very clear when you’re referencing a specific target what that target is and what the purpose is.
[27:58] And I wanted to get a bit more generic around that, such that we could build up a library of generic behavior that we could apply to any sort of structure, regardless of how the designer wanted the page to look. So I wanted that distance between those two things.
That’s one of the other reasons why I’m not really a big fan of tying the specific DOM layout to the code, as React does and what a lot of component-style development does, because it doesn’t afford you this sense of reuse that you can develop generic concerns and generic aspects of your dynamic behavior that you can tie to any DOM tree.
That was one of the considerations, so that’s the target concept, that’s a prime motivator, and then the other thing – there’s really only two other things. There’s controllers and there’s actions. And actions are quite similar to targets. They are the triggers. A lot of code we had in Basecamp was using explicit event handling, where we would tie an event handler to usually somewhere up the tree, to some parent, and then we would sort of interrogate that event as it bottled up, to see if it was relevant for the behavior we were trying to match, and then do some code.
Usually, what we were doing - we were using the attribute called data behavior, and then we would scan these data behavior attributes as the events bottled up, and then if the attribute was a match, the data behavior was a match, we would trigger the behavior. Well, that does not provide very readable code always, I’d say. Sometimes we would provide a data behavior on a parent element, and then there would be specific DOM elements inside of that parent that would trigger behavior and we would catch these events and we would do something.
[31:52] The second part of that was that just like data targets, the actions were generic and they could be moved around. So if we currently have an action on, let’s say, a button, and we move that action to a link, the controller could remain unchanged, because the controller didn’t actually care or know what type of element invoked its functions, which again is quite different from when you marry the HTML structure with your component and with your dynamic behavior. Then you’re kind of locked in step and you can’t reuse these things and you can’t move them around. So that seemed like a big advantage.
There’s three basic concepts, and it doesn’t take a long time to learn it. Even without learning it, you can read the code, you can read the HTML structure and you can understand what’s going on, and that’s really all we needed. In fact, I was kind of shocked when I first started extracting this stuff; I was kind of thinking “Oh man, there’s gonna be so much stuff here” and it was the real epiphany of seeing that those three basic concepts - the controllers, the targets and the actions - were enough to extract such a wide body of behavior from Basecamp.
Going back to the targets for a second - I guess I have a comment and then a question as well… I guess the comment would be that you seem to have formalized – honestly, David, this is what I love about what you do in the open source community, because you conventionize things that often times people are doing informally… Because I’ve been doing a very similar thing for years in kind of a half-hacked way, which is trying to balance between selectors, classNames, dealing with the design side, dealing with the functionality side… So what I’ve been doing for a long time is just using like a js-prefix on a className, and saying js-, and then you’re basically doing a target. That is then a signal to the designer that “Okay, this class does not have to do with the look and feel. This is a JS-specific thing”, so therefore they won’t get changed based on someone trying to change the way something looks.
I love this question, because this is literally the fourth concept that we’ll be introducing shortly and that I’ve been working on for some time. It’s exactly as you say, we do almost all of the mutation of existing elements through classes, whether we want to hide something - well, that’s just adding in a class that hides it; or we wanna show something, or we wanna play an animation, we use CSS animations… Classes are really the way to mutate the DOM the vast majority of the time. And those classes are explicit, right? They are BEM-ed, or whatever they are…
We’re gonna be working on that next. I’m hoping that Stimulus 1.1 will include the abstraction of classes into this structure, such that we can use logical classNames that make sense for the code, rather than concrete classNames that make sense for the designer, that adhere to BEM, that do all these things.
Does that mean that the className will get set through a data attribute?
Exactly, yes. So there will be, for example, somewhere on the controller, so they can be changed by the designer in the HTML. So the designer does not need to monkey around with the controllers, at least as long as it’s a stable set of logical classNames. If they add additional classNames or whatever, you might still have to have some involvement, but in many cases, especially with BEM - BEM has this idea that you can mutate and you can combine a single presentation of an element through a mutation of one className.
That is really something we want to abstract and get away from. There should not be BEM classNames inside of a controller. They should be logically referred to.
Well, you only have to do it for the CSS classes that you need to dynamically apply, remember that. Most of the CSS classes you don’t need to do this work with. It’s only for the ones you need to dynamically apply to something, like if you have a specific class for hiding an element or something else like that.
I don’t actually think that in most cases you can go generic with them, especially if you follow BEM. A specific presentation of one feature might have a logical className for hiding things, but the concrete implementation in BEM for hiding something might be slightly different than it is somewhere else, because it adds or removes padding or margin in some ways.
We’re still sort of feeling that out for now. I’m pretty confident that a huge step forward would simply be to go abstract with the CSS classes you need to dynamically apply, and then declare either the same element that holds the controller name, or on the target or somewhere else where it makes sense. Just disconnecting these two things so we get on two different trains of changing them.
It’s interesting that, Jerod, your version and David’s version is essentially not the same, but it’s very explicit; you were prefixing JS, and David’s solution is reusing the concept of data attributes, and again, being explicit and saying “This is different, this is not your normal class. It’s for a special purpose.”
Yeah, and I think I was trying to get back to something that we lost when we went from writing your click handlers right there in the HTML, which is decidedly too low-level to be in your markup… Moving away from that, when we switched to that style of markup to “Now we’re going to put everything into jQuery click handlers” or what have you; we lost that connection. Now there’s a lot of people rightly saying “Hey, this is actually a step backwards, because now you have random things happening that you don’t know about. There’s no connection in the code, so this kind of bridges that gap and puts a nice happy medium in place.
Right, the good old days…
…because a lot of it is tied up into – exactly, first of all, good old days. The good old days were often not that good. And second of all, just that we have different applications and they work in different ways. If you’re trying to make this very intricate UI with tons of related things and blah-blah-blah, then maybe these heavy frameworks do make sense. If you’re trying to make an application like Basecamp or GitHub, they don’t make sense and they are overly complex for what we’re trying to do.
Stimulus provides a completely different paradigm, a paradigm where the server continues to create the entire HTML document, and then we sprinkle this remaining behavior that we need onto this through a progressively-enhanced approach. But even when we do dynamic stuff - so we have Stimulus controllers, for example, that will trigger a behavior on the server side; what the server side will return is a fragment of HTML. We use HTML as the transport protocol. Very rarely - although sometimes - we will use JSON to do it.
[44:22] So when we want to update a part of the page, we ask the server-side, “Hey, can you give me this fragment of it?” Which allows wonderful things like fragment reuse, which you’re familiar with. In Rails we call those partials, and the fact that you can use the partials, the fragments of the HTML, both to render the initial version of the page and the subsequent updates, is a huge step forward.
That’s interesting. I’m curious about this. I know that in Ruby code, David, you like to remove comments and rewrite code, or replace comments with more readable code. And in your process of evaluating Basecamp and looking through things of this transition to Stimulus, I’m curious how much of that happened in your code, like how many comments were joyously removed in replacing with this prototype version that you created?
A fair amount of it, and I think that that is exactly the attraction to adorning the HTML with this very explicit tie-in with how things are called… Because when you have a button that says – it has a date or action where if you click this button we’re gonna call the greet action on the hello controller, that’s incredibly self-documenting. You do not need a code comment to explain what’s coming on. And on the controller side, the same thing, right? When you have a declaration of action methods upfront, you don’t need to comment on how those are hooked up. You’re relying on the conventions that Stimulus affords you, and those are already documented in Stimulus, so we can really cut down on the amount of needless documentation that we need in our application code… Which for me, whenever you have copious amounts of code commenting, it usually tells me two things - either that you wrote convoluted code (that’s probably the most common); 2) that you have a set of conventions that you’re just still waiting to extract, and then once you extract these, you can remove all this repetitional commenting… And these are the driving motivations for why we want to get rid of them. Why? There are code smells.
In some rare cases, what you want to do is actually just counter-intuitive, and sometimes it’s basically around browser bugs, or something else. You’re doing something that does not make sense if you just read it, and you need a code comment to explain, “You know what? That’s because i.e. Edge does something stupid here, and this is why we need to do this monkey dance to make it happen.”
But even in that case, I often find that you can still encapsulate that monkey dance in a method that succinctly explains that it’s because Edge or Safari or Chrome or whatever is doing something that you need special consideration for.
Alright, David, you said what you saw was a lot of regressions, and maybe even side-steps in certain cases… One of the things I remember back in the day when Backbone.js first hit the scene - because this was one of the very first front-end frameworks that specifically said “Get the state out of the DOM. Get it into JSON, get your state out of the HTML.” And that was a step, and everybody started doing that in different ways, so it began completely detaching the front-end and the back-end. And like you said, Stimulus offers really a different opinion or a different way of going about building applications than a lot of the other front-end frameworks out there, and one of the things that it says on the home page is “State is stored in the HTML, so that controllers can be discarded between page changes, but still reinitialize as they were when the cache HTML appears again.” So that’s definitely a big difference from other things out there.
Can you tell us about how that state is stored, how you deal with change? I know you mentioned it a little bit during the last segment, but let’s go a little bit deeper into how that all works.
Sure. So we store state in much the same ways that we declare the targets and the controllers and so on, through data attributes. We basically just set these data attributes on usually the root element of that controller, and that stores the usually minimum amount of state. I think one of the reasons Backbone and other frameworks have argues for extracting state into something else is because maybe they had a ton of it.
We try not to have a ton of it. We try to have very little state within the DOM itself, but just enough so you can reinitialize a controller, and it can come back to the form that it was. For example, a state could be – we have a collapse controller that allows you to click a certain element to open or close another element, right? Like a Show More, or See Less, or whatever. And the state of whether that is open or closed is something you can store in the DOM. In that particular case, often times the state is actually just an application of the classes. Classes provide usually most of the state that we need.
For example, for the collapse example we will apply a hidden class when the element is closed, and we will remove that hidden class when the element is open. That right there will store the state in itself. We try not to duplicate or create a shadow state of what the DOM already has, because we’re trying to enable you to get HTML from anywhere. Most of the time this HTML is coming in the form of the initial render, and the controller simply has to take that initial render and instantiate themselves based off that.
Then if you have other updates, it could be websocket updates; you have a websocket channel that inserts new HTML into the DOM. When that new HTML is inserted, it needs to include its own state. We don’t have something else to also pass that state along with. As I said, we use HTML predominantly as the transport layer, not JSON. So the transport layer has to include its own state, and that’s where we found this is actually a nicer way of doing it.
Then just the fact that we go back and forth between pages - we’re using this together with Turbolinks, and Turbolinks stores a nice cache of the pages we’ve been switching between. So if you have controllers that have been mutating the DOM, usually just by adding CSS classes or whatever, Turbolinks will remember the state of that when you go from one page to another. So we get to sort of keep that state alive through the Turbolinks cache, and that’s adequate and actually a useful constraint in most cases to not go hogwild with a whole bunch of intricate state.
[54:24] I suppose that works just fine without Turbolinks, you just don’t get the advantage of the fast refreshing at a full page’s reload.
Yeah, and you’ve just gotta be a little careful with those full-page reloads and what the browser ended up caching… Whether it ended up caching the final version of what the DOM looked like, or it’ll reinstantiate it from scratch. Turbolinks helps a little bit there in terms of making it more fluid and making it easier to keep that state cached.
So were there specific changes that went into Turbolinks to support this? I know you called a one two-pack punch, or something like this… I was wondering if that just happened – they just paired well nicely, or if Turbolinks requires some specific upgrades or enhancements to actually support Stimulus natively?
It didn’t, because we were already basically writing Stimulus before we were writing Stimulus. This approach of using progressive enhancement and storing state in HTML, and using HTML as the transport layer - that has been our pattern and our paradigm for a long time, and we built Turbolinks originally with that paradigm in mind. So Stimulus is basically just an encapsulation of that paradigm, and packaging it up in a nice way. That was really the missing second punch to Turbolinks. We would pitch Turbolinks to someone, like “Hey, this is this wonderful thing that can actually cut out 80% of all this dynamic behavior you’re doing, because it’ll speed up the page changes to such a degree that you don’t need anything else.” There’s a lot of dynamic behavior we’re doing for performance reasons, because it feels too slow to do a full-page change, that you no longer need when you’re using Turbolinks because the page changes are really fast. So you can just send the whole page again, even though you’re making a relatively small change, in a lot of cases.
But then there was still the last 20% where you didn’t wanna do that, right? You had some small change, like a collapse show thing, as we’ve just talked about… It’s a little excessive if you’re showing or hiding, which is basically just applying or not applying a CSS class to an element required a whole roundtrip to the server, and sending down all the HTML for a whole new page, just to apply a single additional class. That doesn’t make sense. That’s not proportionate, and it’s not gonna be fast enough to feel really good.
So that was this missing grey land, the last 20% of behavior where we kind of just waved our hands and said “Um, have you looked at MutationObservers?” and then left it as an exercise for the reader, which was actually a fairly large task for someone to do. So I could see how not having a clear answer for that last 20% held back Turbolinks in some ways. So I’m really happy that we now have Stimulus to provide 100% of the answer for applications like Basecamp, and if you wanna write them in this way, you now have all our tools; there’s nothing hidden under the carpet here. Everything that we use to write Basecamp the way it is today is open source, packaged up as an easy-to-use library or framework, and the story is now complete… And I think that that’s really important - if someone is looking at their application and they can’t visualize how they’re going to solve this specific part of it or this specific feature, it’s hard to gain adoption.
Turbo, yeah. [laughter]
Anyway, I mean, Turbolinks and Stimulus - they’re very much for writing for ourselves. I’m doing these things because we need them in Basecamp, and when someday that day is going to come when I write Basecamp 4, I want them to be available and packaged in clean form so I can just use these frameworks off the shelf and get on my merry way. And if someone else ends up using it, that’s great. And if not a lot of people end up using it, that’s also great; I really will continue to do it. But it’s sort of the same approach I have with Rails…
When I originally wrote Rails, I wrote all of it. I needed a way to talk to the database - well, I wrote Active Record. I needed a way to render templates - well, I wrote Action View. So at Basecamp we have a tradition of writing our own tooling, and then we share our tooling more out of gratitude to the rest of the community for the tools that we do use, and just because that’s a nice thing to do, and sometimes it ends up taking off, as in the case of Ruby on Rails, and we get some benefits from that. And if it doesn’t take off, that’s also fine and we get to use them.
I think that’s really a sort of ambivalent or distant relation to the open source process, that we open source because we can and because we like it, not because “Oh, it has to gain adoption.”
Right. Just hypothesizing a little bit about Turbolinks, because I’ve been around I think for the length of its run, and I’ve seen the community reaction to it over time, and as you know, we were fans of Turbolinks, we opted into Turbolinks with a non-Rails app, which I think is somewhat unique… But I think some of it had to do with, like you said, that problem with existing jQuery plugins and the fact that Turbolinks was such a plug and play aspect of a Rails application; you could literally just comment it in or out, and it would or would not do everything for you. In terms of Turbolinks itself, it was just so easy to flip it off - pun not intended, but perhaps it should have been - so easy for people to just use Turbolinks as the scapegoat.
[01:02:11.09] Well, I think also a part of it is you have to understand what it’s doing for you, and I think perhaps that was the drawback of having it be so easy to turn on and be included by default… If you don’t fully understand the benefit that you’re getting, you don’t understand the tradeoff. And if you don’t understand what benefits you get, any cost is too high. If I perceive the benefit as nil or nothing, then if I just have to pay just a modicum of work myself to get it, I’m gonna say “That’s not worth it for me.” And I think we’re perhaps getting to the point where more people are realizing that using these heavy-duty client-side frameworks, “Oh, wait, they also have costs”, and they also have in many cases towering complexity, and I think we’re getting some veterans that are coming out of that process that go like “You know what, if I’m gonna build another thing, I’m not gonna do it like this again. That was just painful.” I think that pain is exactly what we try to address. And until you’ve suffered that pain, I don’t think you can fully appreciate the salve that we are offering, the band-aids that we are offering… Which in many ways was the same way that Ruby on Rails came to be. There was so much pain that a lot of people had experienced using Java frameworks or PHP without any frameworks that they were very in tune with the pleasure that Ruby on Rails could bring them because they knew the pain. I think until you know the pain, you don’t have space in your brain to appreciate or even properly evaluate the solutions to it.
That’s the bitter and sweet, right? You can never really understand the sweet goodness of the chocolate bar unless you’ve had that nasty piece of candy after dinner, or whatever. You’ve gotta have the bitter to enjoy the sweet.
Yes, and I think that that goes for all sorts of learning. I think that’s why lists of best practices for example, divorced from the pain from where they arose, often don’t make a whole lot of sense or they don’t stick. People aren’t ready to internalize lessons until they’ve encountered situations that really demanded those lessons in flight. I think that’s just part of the learning process; you can’t appreciate everything up front. I think plenty of people end up going off to college and they end up having all sorts of courses in philosophy or whatever and they can’t apply them to their experiences of where they are in life, and they go like “Oh, this is worthless. What’s existentialism about? I can’t use that for anything”, and then a decade later they go, “Oh, wait a minute… Let me hear what Camus has to say about the meaning of life”, because they’re at a different station in life.
I think there’s a lot of technology that works like that - it doesn’t really reveal itself until someone has suffered through the long road.
Something that goes to teaching and documentation, and one of the things that is very difficult to do through readme’s and docs, and even blog posts - blog posts are a little better, but they’re so hard to find over time - is like “What were the circumstances in which this solution came to be and why does it exist?”, which is some of the gaps that we try to fill with the Changelog and shows; conversations with the people to give that historical context… Because there are no panaceas, there’s no silver bullet, and all of these have trade-offs and all of these have reasons why they were created.
[01:06:03.10] So if you lack that historical context as somebody who’s coming to a Stimulus or coming to a React, and just picking the tool off the shelf based on the readme, and you don’t understand the historical context in which those tools were developed and why they exist, then you’re basically doing a coin flip, and you don’t know how you can apply it to your given circumstances.
So it’s tough… Like you said, sometimes you just have to live and learn, you have to just go through it and realize it, but I think we can work together to give these historical contexts to people, so they’re more equipped to make those decisions.
I think that’s spot on. I think there’s so much technology that’s presented just as the how, not as the why, and it’s that why that gives us the context to evaluate whether this is a good fit for us. Can we see ourselves (the person who developed this solution) and their troubles in our troubles? That was one of the reasons why when we introduced Stimulus we did it with a document called “The Origin of Stimulus”, which basically walks through “Why did we extract this? Why did we make this?”, and tells the story of the Basecamp code and our journey of making a Majestic monolith, and how we use Turbolinks together with Stimulus, why the concepts make sense, which – one of the problems, as we’ve talked about here, right? Like the problem of using CSS classes for targeting - it’s brittle, it’s all these other things… It’s not just like “Oh, here’s how to do that”, right? And I think that that’s often missing, and I think sometimes people sort of evaluate things from the wrong perspective.
One thing I’ve heard a lot of times is “Oh, I wanna use React because Facebook is using React, so it’s gotta be good enough for me.” I’ve heard that with a lot of other pieces of technology. “Big company is using X, thus it must be good enough for me.” I actually think it’s often the exact opposite. A lot of the patterns and even outright technologies that large companies use are the worst thing you could pick when you’re just starting out, or if you’re a single developer or a small team, because these things are designed to work with much larger teams, in much larger companies with all sorts of different considerations and specializations, and a stomach for a different level of complexity. When you’re trying to serve a billion and a half people, you just have different problems than what we’re trying to solve at Basecamp.
We’re trying to solve servicing a few million people at the most, right? It’s different orders of magnitude, and the correct and applicable patterns and practices that are relevant for someone trying to solve for a solutions base of a few million people less is just very different from the kind of people like Facebook, who have tens of thousands of people working on the product and are trying to solve for a billion and a half. And I think sometimes people just get enamored with this “Oh, I wish I was Facebook. So if I just start using their toolset and their methodology maybe I’ll become Facebook.” No. If you looked at any history of actual Facebook, do you know what their code originally looked like when Zuckerberg wrote it? I don’t think there’s any of those practices left anymore at the company, because they evolved and they turned into something else. But if Facebook had started out with the heavy duty patterns and practices and methodologies that they’re using now, if Zuckerberg would have had to do all those things as just developer one Facebook would never have happened.
That’s right. I have a counter to that, though. Twitter used Rails, and many people use Rails.
I think Twitter is this great example, a great scarecrow, a great reminder that there’s so much more than technology to whether someone fails, even technology-wise or not. Twitter in the early days of the Fail Whale blamed Ruby on Rails for its trouble, because it was much easier to blame an external vector like Rails than their crappy architecture for why the site kept falling over.
[01:10:15.26] And then just recently, a few days ago Twitter - or some former executive from Twitter - through an article in Vanity Fair, blamed Ruby on Rails for the fact that Twitter ten years into its existence has still not dealt with harassment and abuse in a proper way, on Ruby on Rails… Which is just wonderful. It’s a wonderful anecdote of how humans are so desperate to diverge and deflect blame and accept responsibility for their own actions, and they were just trying to find any scapegoat.
It’s all your fault, man.
If you hadn’t released Ruby on Rails, they would have never had this problem with harassment.
And actually, that is true, right? That is actually true, in some sense, because maybe Twitter would never have existed, or it would never have taken off, or it would never have gotten done in time, or they would have run out of money, or something else. So in a way, I think Ruby on Rails is implicated in the harassment problem at Twitter, because Ruby on Rails helped Twitter get started and helped Twitter get off the ground.
The fact that maybe it’s been 10+ years and they’ve still done so poorly at addressing the fundamental problems of harassment and abuse on the platform - maybe that blame falls elsewhere, but anyway.
Let’s move into something I guess a bit more promising. So you’ve had your hand in obviously writing frameworks, we know that… You do some great writing on Signal v. Noise, you’ve got a podcast, you’ve written books, you race cars, you’ve got kids… You’re like everybody else, you’ve got all these cool things, but next up is YouTube for you. You’ve got this cool new channel maybe not everybody has heard about yet, but I’ve been enjoying it, and one thing I think is pretty interesting is that you get this chance to essentially sit down with you and look through the Basecamp codebase, and you’re just sharing all the reasons why you’ve done what you’ve done. Can you take a moment and just kind of share what your plans are with that channel?
Sure. The channel is called “On writing software well”, and it’s just on my YouTube. It’s basically just me opening up an editor and taking a topic that could be testing, or callbacks or whatever, and showing how we use that in Basecamp, and showing how we use Rails and Ruby to solve the problems.
What I wanted it to feel like was if I sat down with another programmer and we just looked through some code together - I always love doing that, because I find that many programmers when they’re talking in the abstract about code and patterns and so forth, they have these fierce arguments: “No, this is the wrong way of doing it! This is the right way of doing it!” And then if you sit down with them and you look at the actual code, you end up agreeing way more often than not, because the pressures and the concerns of a specific piece of code guides most reasonable people in a similar direction, at least when they have somewhat of a shared background and experience. There may be functional programmers who are like “Oh, anything object-oriented or side effect-laden is wrong” and whatever, you’re not gonna find common ground with them perhaps, but for anyone who exists in the same paradigm and somewhat have shared beliefs, if you look at concrete code, we end up liking the same things a lot of the times, a lot more often than if we just argued about it in the abstract. And this is one of those lessons that I’ve learned time and time again. Ruby or Rails back in (I think) 2009 merged with another Ruby Framework called Merb. And Merb was born for a lot of different reasons, and some of the reasons were that the people behind Merb cared about different things than what I cared about; not that I actively didn’t care about them, they just weren’t top of mind. There were some extensibility concerns that they had and some performance concerns that they had, and we thought we had these fundamental, underlying philosophical differences about how to write a framework in Ruby.
[01:14:08.05] So I sat down with Yehuda Katz in particular, who was one of the guys involved with Merb at the time, and we had these fierce debates when we were just chatting in Campfire, and then we sat down, looked at the same piece of code and went “Oh yeah, we believe the same thing.” And we’re like, “Wait, what?!” We were just arguing our heads off in opposite directions, and then we looked at a piece of code together and we came to the same conclusions. I’ve done that so many times now that I believe that it’s really the primary way you should be arguing code patterns and principles - by looking at actual real production code, and doing A/B’s. “Let’s write it your way, then let’s write it my way, then let’s see if there’s one or the other ways that’s the best, or more likely that there’s a combination of the two ways that turned out the best that we both like.” I find that that happens just all the time.
So I wanted the YouTube channel to have that feel as though we were sitting down at the keyboard together and looking at code together and coming to similar conclusions. That doesn’t mean – I mean, everyone is not gonna sit down and watch these videos and go like “Oh yeah, I would have written it exactly like David would have written it”, but at least if you hear the why, why we wrote it that way, how we weighed the tradeoffs and came to certain conclusions, you’ll understand why we did it the way that we did, and I think that that understanding is often sorely missing when people are talking programming and talking shop.
That’s why the arguments get so heated. We start having these violent disagreements that in many ways are completely unnecessary, completely unjustified by actual code… And it can’t be solved or it can’t be addressed just by looking at example code. It can’t be addressed by the stylized, idealized version of what programming looks like because until you have all the real constraints and pressures of production code, you’re not taking all the complexity into consideration, and often times that’s exactly what tips the scale as to whether to go one way or the other way - to look at the real code.
For example, I did an episode on globals. Rails 5.2, which is just about to be released, has a new encapsulation of globals for dealing with things like current used and current account within the lifecycle of a single request. And if you just sat down and had an abstract conversation about globals with a programmer, most programmers would say “Well I learned/heard/know that globals are considered harmful. Why are you using globals? Globals are terrible? Don’t use globals. Are you a bad programmer? Are you terrible? What’s going on here?” [laughter] And then you sit down and like “Yeah, all those things have strengths of truth in the aspect that globals are dangerous and you do need to be careful, but hey, let’s look at this code.
Let’s look at how much simpler and easier it actually becomes to understand once we use a global. It’s not a thing you should use all the time and in all circumstances, and you can certainly get carried away with it, but in this particular instance, using the feature on this particular aspect of the code, it gets better. This A is better than that B.”
I think that those are the concrete tradeoffs, as I said, that are really just fascinating, and I think it opens people’s minds much more than just a blog post that says “Global is considered harmful.”
That’s one thing I’ve really appreciated about this series with you, is that you explain your preferences, and then you look at the code and you say “This is why I’m breaking my own rule. This is why I’m putting these methods out of order in the private method, rather than as a table of contents, like you normally would. This is why, because it reads better, so I’m willing to trade off and go against my typical grain for these reasons.” You kind of get a chance to step into your mind, watch how you program an actual application called Basecamp, you’re looking at actual Basecamp code. I think that’s a pretty interesting concept and I’m glad you’re doing it.
[01:18:09.02] That is really I think the pivotal thing, the nugget, the justification of it. The fact that there are all these principles and patterns and best practices of how you should write software, and in isolation, they all make sense. But when you write a real application, usually all of them disagree. There will be one pattern that tells you to do things this way, and then there’ll be a best practice that tells you to do things in the other way, and you have to weigh these things and consider “Which one is more important in this particular instant, and which one will I put more weight on?” and that’s where the wisdom is hidden.
I find that a lot of programmers apparently seem content to just learn the recipes, to just learn like “Oh, I can recite all the patterns. I can recite Solid, I can recite the Law of Demeter”, and then they don’t really know what to do when these principles are in conflict with each other, and that is really where actual code is written. It’s written in conflict, it’s written under one pattern and one consideration pulling in one direction and another pulling in another direction, and you have to carefully weigh - and sometimes subtly so - which you’re going to put more emphasis on.
I agree with everything Adam said about the saw. A quick suggestion or a feature request for upcoming – and this might be a little harder for you to organize, but it’d be cool if you would sit down with Sam or with Javan or with somebody who has a different opinion about even Basecamp code, and you guys could talk through a refactoring, or talk through things where it actually is a dialogue. It might be an interesting alternate style format for this show.
Yeah, I’d love to do that. I think that is basically what I’m trying to do. I’m having a dialogue somewhat with an imaginary programmer sitting next to me, and the two minds -my own and my head, right?
But part of it too with this – so I’ve produced five episodes, two hours of content or so, in like a week. The reason I can do that and the reason I can churn these things out is because I do one take, and I just flip on my browser, I load up a couple of tabs of code, and then I’ll just freewheel it. That is crucial to why this is happening.
If I had to put in diligence and preparation work that perhaps it would take to include multiple participants, I just couldn’t do it at this velocity and perhaps it’d be harder for me to maintain the stamina to do it at all. I think that’s why it’s kept me back from doing this before, because I thought “Oh, it’s gonna be big production, it’s gonna be a big thing.” No. I mean, most of these episodes have like literally 5-10 minutes of prep work where I’m picking out the files I wanna talk about, and then I hit record, and then I push Stop when I’m done, and that’s it.
It’s a tradeoff.
In a good way. It’s a pretty simple production process for you.
I think ‘production process’ is even a very fancy word for it. I mean, I’m hitting Record and then I’m hitting Stop, and that’s it. There’s not a lot of post-processing or second takes.
Gotcha. Was there anything else to share about the future of Stimulus, maybe the next version of Rails before we close out? Any sort of planned convergence? I know that Rails does a lot of generation in terms of scaffolding; any plans of Stimulus being baked in and HTML going down the pipe with data attributes? Anything left unplugged?
So let me promote that as an alternative path; I have no illusions that just because we promote this, it’ll turn into the path and that everyone will switch to it. I think there’s tons of momentum - and deservedly so - around a solution like React, and elsewhere. And we don’t have to win total domination.
Well, David, thank you so much for your time today, man. I appreciate your willingness to experiment, to dive into Basecamp and extract patterns and come out with what is now Stimulus. And then your passion for open source, to release it because you can, and because you have gratitude back towards the tools that you’ve used. We appreciate that about you, so thank you very much.
Thank you. I can honestly say that it’s entirely my pleasure.
Our transcripts are open source on GitHub. Improvements are welcome. 💚