Kaizen! The best pipeline ever™
Gerhard is back! Today we continue our Kaizen tradition by getting together (for the 10th time) with one of our oldest friends to talk all about the continuous improvements we’re making to Changelog’s platform and podcasts.
Discussion
Sign in or Join to comment or subscribe
Nabeel Sulieman
2023-06-03T22:44:05Z ago
At the very end of this episode Gerhard talks about the challenge of horizontally scaling The Changelog. Specifically, he mentions that only one “main” server should handle emails and things like that. I disagree :-)
Ideally in situations like that, you should have a queue that all instances can check for “jobs” to be done. Any instance can handle the job, but the queue will ensure that only one will get that job. It will also return the job to the queue if one of the instances fails to complete the job.
Right?
Jerod Santo
Omaha, Nebraska
Jerod co-hosts The Changelog, crashes JS Party, and takes out the trash (his old code) once in awhile.
2023-06-04T19:49:25Z ago
I agree in principle. That’s how our bigger mailers work (using Oban which stores the queue in Postgres), but our one-off transactional emails (sign in, confirmation, etc) just send straight away in a background thread.
I believe Gerhard was thinking more in terms of having experimental instances that he can run without them accidentally doing something “real” (like popping an email job off the queue and sending it or adding new ones to the queue), but we didn’t talk in enough detail for me to know for sure…
Nabeel Sulieman
2023-06-04T22:24:38Z ago
For test/canary instances that should be solvable with different configurations (environment variables) that can disable that functionality. But it sounded to me like he was talking about multiple regular production instance.