Prisma founder Johannes Schickling has been using the Effect library for the last couple years. Today he joins Jerod & Nick to tell us all about this very interesting tool for building robust apps in TypeScript.
Fly.io – The home of Changelog.com — Deploy your apps and databases close to your users. In minutes you can run your Ruby, Go, Node, Deno, Python, or Elixir app (and databases!) all over the world. No ops required. Learn more at fly.io/changelog and check out the speedrun in their docs.
|Chapter Number||Chapter Start Time||Chapter Title|
|1||00:00||It's party time, y'all|
|5||08:31||What scale of apps need this?|
|6||10:46||Error (non) handling|
|7||16:54||The right thing should be easy|
|9||20:57||A trivial example|
|10||25:34||On incremental adoption|
|11||28:12||The observability story|
|12||30:34||Trying to scale console.logs|
|14||36:49||Nick thinks adoption out loud|
|15||40:37||Tool adoption practices|
|16||43:11||On learning curves|
|17||44:58||On state management|
|18||47:05||Effect Days 2024 in Vienna|
|21||50:23||Next up on the pod|
Play the audio to listen along while you enjoy the transcript. 🎧
Oh yes, you know what the sound of those Breakmaster Cylinder beats means… It means it’s time once again for JS Party. I’m Jerod, your internet friend, and I’m joined by my friend Nick Nisi. Ahoy-hoy, Nick. What’s up, man?
Ahoy-hoy. How’s it going, Jerod? It’s been a while.
It’s going very well. Yeah, where have you been, man? I’ve been on the pod nonstop.
I went to Disney World.
That’s right, you did. You also went to Changelog & Friends.
I did, yeah.
For those who missed it, we did a special Changelog & Friends with Nick, where we talked about what we want from a web browser. So if you’re only a JS Party listener and not a listener of the Changelog, what’s wrong with you? Hop on over there, check out the Changelog feed, hear Nick talk about web browsers for something two hours. We went deep on that episode. That was fun.
We’re here today not to talk about web browsers. Well, sort of, in a sense, insofar as they are the way that we access the web… We’re here to talk about Effect, a really interesting project, used by Johannes Schickling, the founder of Prisma, a user and a community member… He’ll tell us more about his involvement with Effect. But Johannes, welcome to JS Party.
Hey, thanks so much for having me. It’s a pleasure to be here.
Pleasure to have you. And this is a cool new project… I mean, it’s new to me. I’m not sure exactly if it’s new. I think you said you’ve been working with it for something three years, so I guess new is not the correct adjective. Is it only new to me, or is it new to a lot of people?
So the sign on the window says “The best way to build robust apps in TypeScript.” So if you’re wondering why I had to have Nick here is because this is a TypeScript-focused project, and Nick is a TypeScript-focused person… And so what exactly is Effect doing for people building apps with TypeScript?
So this is a question we still don’t have figured out a precise pitch…
Okay, five years in; we’re getting there.
[00:08:30.24] At what phase in the lifecycle of an application, or maybe even in the size or scale of an application - I’m talking about lines of code, not scale of user base - would somebody typically want to reach for something this? Like, square one, grab Effect and get going, or write your own stuff for a while, and then bring it in once you start to feel some pain? Where does it fit?
That’s a great question. So I think if you’re already familiar with Effect - let’s say you’ve used it for your previous project, etc. I think you’d probably be reaching for it on day one. I think there’s a similar kind of getting used to it journey and dynamics similar to - you would have probably not used TypeScript from day one if you’ve never used it before… But you’ve started reaching for TypeScript once you’ve experienced some painpoints, and it was worth it for you to get started. But then in your next project you use it from day one. So it’s similar about Effect.
I would say you’ll probably start looking for something Effect when you’re dealing with really hairy problems building your app. Let’s say there’s something that’s just too much – like, you know it’s slowing you down so much… So let’s say your app really could use some better error handling, but Try Catch is not the answer. You can’t wrap every little function, every little block in your app with Try Catch. You know your users are suffering, because your app doesn’t have proper error handling, and then you start looking for “Hm… What are better ways to do error handling?” Then you might be reaching for something Effect.
So I think depending on what is your biggest pain point, that becomes your entrypoint for Effect. And once you’ve picked it up, then you see how many other things it can offer, and then I think you just use it by default for every new thing.
So kind of looking at the docs, that was the main thing that kind of stuck out for me in terms of what Effect was. And I’m specifically talking about the effect object or type that you get from the library. Right in the documentation for that it talks about a typical function, even in TypeScript, you can define the function, and you can define the arguments that get passed to it, so you know exactly what that is, and you can define exactly what gets returned from it. But there’s no indication of whether that function will throw, or could throw, and what it might throw. And so is that what Effect is really trying to do, is to set you up to properly type and account for all of the known errors or failures that might occur within that function?
[00:12:27.16] So let’s say you have a function that can do something, maybe it needs to act with an HTTP endpoint, or maybe it needs to interact with a database, or it needs to interact with something… Anything that can go wrong and that you want to recover from in some way or deal with in a way that is not completely useless to your users, then you want to kind of lift that up, and expose it and deal with it somewhere in your program. So lifting that up into the type system is incredibly powerful, and then goes a step further. The third parameter is very similar to what React context is, except it’s type-safe. So when anything in your program - let’s use that React analogy… If any of your components have somewhere in the components use contexts, and then it uses something from the context, in your React program let’s say you use that component somewhere - you have no clue whether that component is used or not. Maybe you need to use a default context, or its throws in with effect; just by using something further down, it kind of bubbles up in that context type, and you see “Oh, somewhere deep down in my program I’m using a database, or I need a filesystem” etc. So it kind of nicely composes what can go wrong, what does my program need, and then still what does it return.
Nick, how do you deal with these things in your hairy TypeScript programs? Just hope the errors don’t throw, or put your head in the sand? Or what are you up to?
Why were you laughing internally?
Because I’m so guilty of that all the time… [laughter] I’m making a fetch call right here, and I know that it could fail, but I don’t have to do anything. Something up above me will catch it if it really cares about that… The idea of from the outset setting up a way to not only describe what could go wrong, but give me a pattern that I can put into place for actually handling that… Because that’s the thing that I always run into, is “Oh, should I catch here? If I catch, what am I going to do? Console.log and rethrow it, or something?”
Yeah. So having more of a pattern, and – the idea of just putting you into that mindset of “I’m going to define all of this everywhere” really makes it so that you are thinking of that, rather than just kind of spraying it out there and then praying that somebody up higher in the chain is going to catch and do something with it.
Unfortunately, a lot of times who gets left with a tab is your user, right? [laughs]
If somebody doesn’t handle it, the user has to handle it.
You know what - they’re not there in the PR to complain about it, so…
And I think the worst part is basically, like, if you go about error handling a little – if you just invest a little bit of time, and you just distinguish between “Hey, something has gone wrong here. We can do something useful about it”, and then just propagating that to the user and giving them a Retry button, as opposed to just not doing anything about it and just giving them a 500, there’s some really low-hanging fruits if we could go a little bit further. And you mentioning about that fetch call… Even worse, if you do that fetch call in a non-async function and you don’t await it, since now if it fails, you might even have an unhandled promise rejection… And so there’s so many ways how you can really hurt yourself, and you don’t think about that while you’re in happy programming mode, when you first build out your app very blissfully.
[00:16:13.04] I feel he’s really describing the way I code over here, Nick, and you’re nodding along as if he very much knows our happy path programming styles… [laughter] I just prefer not to think about other things but the happy path. So - I mean, I think this is great. I think that a lot of times, you said, Nick, I guess the hesitancy, or the… You kind of throw up your hands and move on… It’s just not having a firm “What’s my practice in this circumstance?” Almost like underpinnings or undergirdings for you. So something Effect, which says “Here’s how we handle these circumstances”, and then you’re just enabled to make the decision, versus not just having to make the decision of what do I do, but how do I do it? So that’s cool. What other things does Effect bring to the table?
And so yeah, besides error handling, I think a form or an aspect of error handling is retrying something, which is super-cumbersome to do it in a non-principled way… And Effect makes that literally a one-line thing that you just add underneath your function or underneath your Effect.
Does Effect have browser and server-side effects? Is it a frontend thing? Are there limitations to where and how you can use it? And then if you’re shipping things to the frontend, what does this add to your packages?
[00:20:56.03] There’s an example on the website that I was curious, maybe we could walk through it and talk about what it’s doing or what it’s adding… And it’s specifically in the Quickstart on the React and Vite side… And it’s just taking a use state hook call, where it’s setting a count… But then it creates a task using useMemo. And inside of that, it just calls effect.sync, which will hit said count, and then an increment call which is just a callback that’s running something called runsync. I’m trying to understand what this is adding.
Right. So this example rather shows you – this is kind of a Hello World. A Hello World isn’t very useful. It doesn’t do much for you. It rather shows you “Oh, this is how I do Hello World in that esoteric programming language.” So arguably, the examples that we have on the website and in the docs right now don’t do a great job yet of selling the value of Effect, but we’re actually working on a new website that will show a lot of before and after, “This is how you do something with promises” or “This is how you do something with RxJS, and this is how you do it with Effect directly.” So this is rather trying to show you “Okay, this is a minimal set, this is a minimal usage, how you can use it within React, or how you can use it with Bun” etc. But this doesn’t show yet how it helps you deal with complexity.
Okay. Yeah, that’s what I was curious about… Because I wasn’t sure what it was adding to the normal use state call.
This is what they call self-documenting.
[00:24:08.12] Yeah… But it can be a really nice boundary. I think in the React community the useEffect call is a bit notorious; some people like it, some people hate it… And typically things within – whatever you’re going to put into the useEffect call is kind of like “Okay, now you’re on your own.” So this is where you should be very principled; you should return something that cancels the thing, whatever you’re triggering in there, but on there, you’re left to yourself. And this is where you do the fetch call without the error handling etc. And so this is actually the place where you can run some Effects and do things in a more principled way if your app demands it.
So I would say probably most people would rather reach for Effect on TypeScript backends. This is where it kind of matters, where you have more side effecty things, where you do more things that can go wrong, and the complexity on the frontend is typically more on the side of how you build, how you compose your UI is etc. But I think if you’re building an app on the order of Slack or Discord, then your frontend is actually very capable, and you need to think a lot about concurrency, and retries, and so on. And this is where something like Effect is super-critical.
What would incremental adoption look for somebody who has an existing app and would like to use Effect? …oh, goodness. Namespace conflict… [laughter] Yeah, what would that look like? How would I adopt it not whole hog, but just a little bit at a time? I assume that’s possible.
Yeah, totally. So I think you should go with whatever pain point is most critical to you right now. I think an example on the backend could be a bit easier to talk about. Let’s say you build a GraphQL endpoint, or maybe you’re using something like gRPC etc. Let’s just pick out one whatever dimension you want to choose. Maybe you want to pick out a specific feature, a specific request handler… So if you think about your program in terms of like a tree, you could just pick any kind of leaf, and that leaf could be a promise right now, and you say “Hey, I want to improve something here. I want to add better error handling here.” Or “I want to add some observability here.” Then you can just take that thing, that thing could be a promise, that thing could be an async function, or a regular sync function… And you just rewrite that to be an effect, and an effect similar to a React component. You can think about – there’s some analogies there in terms of mental model in that regard, that in a component you can compose other components. Or an async function is the same way; you have an async function, you call another async function in it… Very similar about Effect there.
And so you just say “Hey, this is how much time I have right now to refactor”, and you just refactor your async function to become an effect. And then you have that boundary, and then you just call that effect from wherever that async function was called before, and you run that as a promise, and then you have your boundary there.
And from there, you could say “Okay, I don’t want to go any further. I just want to let it sit there and focus on another part”, or you could from there kind of let Effect take over more and more of your program… And you get a lot of benefits through it, since this is where - yeah, that error handling or observability… I think observability is also a super-underrated aspects in most web development, and it provides so much value. This is where you get so many things in just for free.
So you can think of a trace that’s basically like a visual print of like “Okay this is what my program has done.” A trace consists of many small spans. One way how you can for example design this is that most significant functions, you say, “Hey, I executed that function. That should become a span.” And then you get this nice tree diagram of your program execution, and you see “Oh, this function has taken 12 milliseconds. Oh, what has it done for 12 milliseconds?”, and then you can break it down to “Oh, it has called these other functions, and here it has written a file, and writing that file took 10 milliseconds.” And then you can dig in, “Oh, why 10 milliseconds?” and you can look into performance, you can understand where bottlenecks are, and you also see when something goes wrong. This is another aspect of the happy path programming - you’re just flying blindly. You have no idea what your program is doing, as long as it behaves the way how you want. But if something goes wrong, then you start adding console logs. And console logs don’t scale, and this is where observability comes in.
Nick, are you familiar with OpenTelemetry? Are you using it on any of your apps at work, or was this all new to you?
I am one of the people that is trying to scale console logs in a very futile way… [laughter]
Oh, this is great. I like how we just call out Nick just periodically throughout this episode… It’s alright. Well, I think you’re representative. I think you’re representative.
I think so too, exactly. Yeah.
Nick, is this something that you would consider adopting at work? And if so, how would you go about that?
That’s a great question, because I’m thinking that exact thing right now. [laughter]
Well, think out loud for us.
Well, I don’t fully understand the whole picture. I really like the baseline thing that Effect would give me, which is just building in and setting a standard for how we’re going to handle all of these different things that can happen within the application. So it just brings a greater visibility to that, and if that’s something that we can codify within the codebase with this, but also just in the mindset of the developers on the team, that this is something that we have to handle and it’s just a baseline thing that we have to do, rather than assuming the happy path - that’s a net win. But I’m trying to understand a bit more of what it can do. And kind of looking at the website for it again, it looks – we already talked about concurrency with it, but also it looks… Is there composability with these things? I’m specifically looking at the pipe, I think. The pipelines.
Right. So composability is not a feature as such. I think it’s rather an attribute that your overall system has or doesn’t have. And I would say composability is kind of the ultimate goal that we’re all chasing about. As developers, we kind of dream about “Oh, there’s this better abstraction, and everything will be better with it.” And I think what we’re aiming to solve there with better abstractions is composability. And I think how we got closest to composability in the systems we’re building I think is with React. Just think about the magnitude of improvement that React has unleashed on the world. Before, there was jQuery, and jQuery plugins etc. Sure, there was an ecosystem, but they didn’t really fit nicely together. And with React, we have React Components, so you can just install one from Npm, and you can add it to your app, and you’ve got a fair bit of functionality and UI, and you made your app composable. We don’t really have that for kind of the complement to Vue programming. So we don’t really have that, let’s say, in the backend. Sure, we have packages from Npm etc. but the more sophisticated they get, the more they try to do, the less they won’t just work in your contexts. So typically, the more sophisticated your requirements are, the more you’ve gotta handle everything by yourself.
And so I think this is just a testament that we don’t really have composable primitives. We don’t really have a foundation that we could say “Hey, that thing does what I want, and I just add it to my app.” There’s all of those different styles, how something is written in, and then it just doesn’t work inside of your app. And yeah, bringing composability into an ecosystem I think is kind of an ultimate goal, and this is what has got me really interested about looking into Effect, and what is so mind-blowing to me, because I haven’t seen it anywhere else.
I definitely want to dig deeper, I think, and look at it more… Because like I said, I think it just helps to codify some things that we overlook, and I like that a lot.
So regardless of Effect specifically… Let’s go a little bit meta, because I’m interested in adoption practices, and I’m curious inside of your work if you’ve decided this was a good direction for us, or for me, or for your team, or whatever, what would you do? How would you go about it? Are you in a position where you could just pull it into the pkg.json and go? Or would you have to convince somebody that this is a good idea? How does it work at your work?
I’m at a bit of the perfect place to try something this, I guess, because I am – we’re pretty deep into a a completely greenfield rewrite of a legacy Angular.js app… And we’re rewriting it in React, with TypeScript, and it’s very much – we’re getting towards the end of it, but there’s still only two devs kind of going full steam on it right now… And the thing that we’re really looking at right now is we kind of – we’re at 90% completion, and one of the big focuses for me is on that hardening piece. Like, “Okay, we’re gonna have to bring in other developers into this. Let’s go back and look at the practices that we’ve put into place, because three months ago when we started this, these were what we thought were the best practices. Did they actually play out properly? And are we actually running into problems when it comes?”
When it comes to error handling and things like that specifically, one thing that we really went full-fledged on for error handling and things like that are Suspense, and Suspense boundaries within React. But it’s still pretty much like mostly happy path, like “Here’s the fetch call, here’s a use query call”, no error handling associated with that… It’ll get caught by a Suspense boundary at some point, and then show some generic error, but it’s not very specific to “This specific thing failed”, and we don’t really account for things like making several requests, and only one of them fails, what do we do there? Right now, we just show the Suspense boundary and don’t really retry things.
So I think that it’s interesting, and that is the approach that I would take for trying to adopt this into that codebase, is “How is this going to make that specific piece better?” Because yeah, it’s really handwavy right now. And what is that going to mean when we bring in more developers onto the project? …when it’s more than just myself and one other person. Are we setting up the best practices and patterns to have that carried on?
That makes a lot of sense. And I think there’s also an interesting double-edged sword there. Learning Effect, has a good learning curve. It took me a certain period of time to pick it up, and I learned that when there weren’t any docs, I had the luxury of learning directly from the author, and I used that also to contribute back some ideas how to possibly simplify things… But there is still a learning curve; it’s already getting a lot better, and there are some really great resources already, the docs and also some YouTube videos where the community is just creating a lot of those resources… But learning it is an investment, but I think it’s paying off big time. But you’ve gotta bring your team also on board, to be willing and say “Yes, this is how we’re going to do it.” And I think that’s going to take a little bit of time; maybe you have just hired a new engineer, and the likelihood that that engineer says “Oh, of course I’ve used Effect at my previous company…” That’s gonna take a little bit of time until that is a given, similar to how that’s a given with React right now.
Now, one thing that as you were saying that, talking about the complexities and how it’s a big investment into learning all of this, one foray into something like that that we’ve done in the past to kind of handle state management - and my question is, I’m curious, how this relates to Effect or not? Or could they live together, or do they serve separate purposes? And that is we really took to adopting state machines and XState for some complex state management within one of our apps, and I’m curious, is that solving some of the similar problems in kind of a different way by giving you those finite states to have to handle? Or could those two projects complement each other?
Right. So we’re actually in close contact with David, who’s the creator of XState, and we want to figure out a good integration story between Effect and XState. And in fact, we’re going to have David over at the first Effect conference in Vienna next February, where we hopefully get to announce exactly that.
So I hope that we have a good answer for that soon. The way how I think about the responsibilities of how they could work together is XState is all about the state machine part; you know that “Hey, I’m currently in this state”, and you get a visual representation for that, you get types inferred for that etc. All of that - this is where XState shines, where XState, and similar to React kind of leaves things up as an exercise to the developer, is the state transition. Like, in React where you say useEffect, or in XState when you perform the effect… Now, this is still up to you. You might still do the fetch, and this is where you’re still on your own with the error handling, with the retrying, with doing things concurrently, or getting some observability insights into what’s happening. And this is where they can play really nicely together.
[00:47:04.26] Vienna in February, Nick… It sounds like a decent place for a remote JS Party, perhaps. [laughter]
Tell us more about this conf.
Yeah, we have a really interesting set of speakers already confirmed. So I’ve mentioned we have David from XSate over, we have Guillermo Rauch from Vercel giving a talk as well, we have a lot of folks from the Effect community who are going to present the various projects they’re working on with Effect… We have some other well-known people from the TypeScript community, who are taking their first steps with Effect and share the experience. And what’s exciting to me is all of those folks have in common – those are some of the most experienced engineers I’ve ever met in the web ecosystem, and they all seem to be kind of interested in Effect. We’re just where great ideas come together.
Very cool. Well, the website is Effect.website. There’s a new website that I assume will live at the same website soon. Johannes gave us a little sneak peek. They are working on updating the docs, updating the examples, making it more useful, more approachable. I think that’s what this project needs, because it looks very powerful, and as you said, there is a learning curve, and yet probably worth that learning curve for folks who need this. So the more people we can get over the learning curve, into using it and making their apps more robust, more best-practiced, more observable, more error handled, Nick, the better, right? So anything else, Johannes, about the project that you want to make sure we cover before we call it a show?
No. I’m super-happy that I got to share some of my experience and thoughts on Effect. As you mentioned initially, I’ve been using it for the last two and a half years, and the projects and products I’ve built over that time - I could not have built them without Effect. I think that’s kind of the highest praise I can give to a project, if it really provides a lot of value… And now it’s just up to making it more accessible, easier to use, and help people get to use it. So if you’re into type safety and if you feel like there’s too much complexity in your app and you need some better ways to tame it, feel free to check out the website, join our Discord… We have a really lively and helpful Discord community where you typically get an answer in about 30 minutes, and we are going to help you adopt Effect successfully.
Yeah. It sounds like you have to great effect.
I had to, I’m sorry.
I was planning on dropping a Wreckx-n-Effect joke at some point, but I forgot… So here it is, Wreckx-n-Effect. Look it up, kids. On behalf of Nick Nisi, I’m Jerod, this is JS Party, and we will talk to you all on the next one.
Our transcripts are open source on GitHub. Improvements are welcome. 💚