Wynn Netherland changelog.com/posts

Octobot: Fast, reliable Java/Scala task queue for RabbitMQ, Beanstalk, Redis and more

octobot logo

Background tasks are crucial for any non-trivial web application so it’s no wonder that the landscape of queuing technologies is rapidly evolving.

The latest entry is Octobot, a Java-based task queue worker from C. Scott Andreas designed to be reliable, easy to use, and powerful.


Octobot uses best-of-breed tools for queuing, supporting AMQP/RabbitMQ, Beanstalk, and Redis (Pub/Sub) as backends out of the box. The architecture is extensible so additional backends can be added in the future.

Easy to use

Octobot tasks are simply classes with a static run method which accept a JSON object.

package com.example.tasks;
import org.apache.log4j.Logger; 
import org.json.simple.JSONObject;

public class TacoTask { 
  private static Logger logger = Logger.getLogger("TacoTask");

  public static void run(JSONObject task) {
    String payload = (String) task.get("payload");
    logger.info("OMG, GOT A TACO: " + payload");

… or in Scala:

package com.example.tasks
import org.apache.log4j.Logger
import org.json.simple.JSONObject

object TacoTask {
  val log = Logger.getLogger("TacoTask");

  def run(task: JSONObject) { 
    val payload = task.get(“payload”)
    log.info(“OMG, GOT A SCALA TACO: ” + payload)

Octobot also has a simple YAML-based config file format:

    - { name: tacotruck,
        protocol: AMQP,
        host: localhost,
        port: 5672,
        vhost: /,
        priority: 5,
        workers: 1,
        username: cilantro,
        password: burrito

  metrics_port: 1228

  email_enabled: false
  email_from: ohai@example.com
  email_to: ohno@itsbroke.com
  email_hostname: localhost
  email_server: smtp.gmail.com
  email_port: 465
  email_ssl: true
  email_auth: true
  email_username: username
  email_password: password


Octobot is designed for high throughput, heavy workloads, and ultra-low latency. Early benchmarks using AMQP and MongoDB lookups demonstrate, task execution actually improves as the JIT optimizes execution paths.

[Source on GitHub] [Homepage] [Download Docs]


Sign in or Join to comment or subscribe

Player art
  0:00 / 0:00