Earn a Sidekiq blackbelt by breaking a few boards

Wynn posted about Sidekiq last February, briefly introducing a new way of handling background workers. For those of us who took on the challenge of switching from Resque to Sidekiq, you can probably agree with me that it brought a new set of challenges to tackle. The upside of that, though, is that tackling those quirks is well worth the payoff.

To save some trouble for those of you who want to switch but haven’t yet, here’s what to look out for on the conversion.

#1: Sidekiq is too darn fast

It sounds ridiculous, but Sidekiq is so fast that it can run your worker before your model even finishes saving. There’s one easy solution for this: don’t use after_create or after_save, but instead use after_commit to make sure that model is done with the database.

after_commit :run_expensive_hello_world, :on => :create

This will solve 99% of your use-cases, but if you use something like state_machine, you won’t be able to use any of state_machine’s transitions to push out a worker. Instead, we solved it by creating an instance variable that the after_commit callback uses.

attr_accessor :just_moved

after_commit :tell_neighbors_hi, :if => :just_moved

state_machine :status, :initial => :sitting do
  # ...
  after_transition any => :moved do |i|
    i.just_moved = true
  end
end

#2: You can't see failed jobs. Well, sort of.

Sidekiq is different from Resque in that it doesn’t keep a permanent record of your failed jobs, so you won’t always be able to go back later and run it again after it’s fixed.

The reasoning behind this design decision is that Sidekiq, unlike Resque, will retry your job at increasing intervals automatically up until about 20 days, along with giving you the ability to manually retry it – plenty of time to see the error, assign the bug and fix it. After that, you can say sayonara to that record. Sidekiq has some nice docs of how to handle errors, though, including saying one last goodbye to the error, so it’s not too hard to set up some safeguards against losing data.

If you don’t want to take the time to set all that up, there’s always sidekiq-failures.

#3: Sidekiq's perform is not Resque's self.perform

Mike Perham has done a good job of making the terminology fairly similar when switching from Resque to Sidekiq. However, make note that Sidekiq’s perform method is an instance-level method, whereas Resque’s is class-level.

The main takeaway from this is that in Resque, it’s not uncommon to throw the perform method on whatever object you want to use for the worker. In Sidekiq, you’ve got to watch out for initializing that new object. You might be able to get away with it if your initialize method has no arguments, but you’re better off creating a new class just for the worker.

After getting accustomed to some of the differences between Resque and Sidekiq, it’s pretty easy to see that Sidekiq can shave time and money off your server and is definitely worth the switch.