Thoonk: Persistent, Redis-powered push feeds and queues for Python and JavaScript

As more and more applications become publishers and subscribers of data
in external systems, open source projects continue to innovate in the background jobs space. Redis, featured in Episode 0.4.5, seems to be a recurring theme with these types of projects. This week, Nathan Fritz and Lance Stout at &yet have released Thoonk, a project that takes a unified approach to feeds, jobs, and queues, all with a Redis backend.

Thoonk defines its different item types this way:

A Feed is a subject that you can publish items to (string, binary, json, xml, whatever), each with a unique id (assigned or generated).

Queues are stored and interacted with in similar ways to feeds, except instead of publishes being broadcast, clients may do a “blocking get” to claim an item, ensuring that they’re the only one to get it. When an item is delivered, it is deleted from the queue.

Jobs are like Queues in that one client claims an item, but that client is also required to report that the item is finished or cancel execution.

Thoonk comes in two flavors Python and JavaScript for Node.js. Compare examples for subscribing to a feed in Python:

thoonk.register_handler("create_notice", create_handler_pointer)
//args: feedname

thoonk.register_handler("delete_notice", delete_handler_pointer)
//args: feedname

thoonk.register_handler("publish_notice", publish_handler_pointer)
//args: feedname, item, id

thoonk.register_handler("retract_notice", retract_handler_pointer)
//args: feedname, id

… and the obviously Node JavaScript example.

thoonk.subscribe(function(id, item) {
    //publish event
},
function(id, item) {
    //edit event
},
function(id) {
    //retract event
},
function(id, position) {
    //position event for sorted feed
    //position is begin:, :end, :X, X: where X is the relative id
},
function() {
    //subscription ready
});

Be sure and check out the READMEs for each project for advanced usage and tips on contributing.

[Thoonk.py Source on GitHub] [Thoonk.js Source on GitHub] [&yet Blog post]