Mahdi Yusuf is back with another set of Architecture Notes, this time covering everyone’s favorite “data structure server”, Redis:

Rather than iterating over, sorting, and ordering rows, what if the data was in data structures you wanted from the ground up? Early on, it was used much like Memcached, but as Redis improved, it became viable for many other use cases, including publish-subscribe mechanisms, streaming, and queues.

Dragonfly is a modern in-memory datastore, fully compatible with Redis and Memcached APIs. Dragonfly implements novel algorithms and data structures on top of a multi-threaded, shared-nothing architecture. As a result, Dragonfly reaches x25 performance compared to Redis and supports millions of QPS on a single instance.

Redis is awesome, but Redis commands are not always fun. Pottery is a Pythonic way to access Redis. If you know how to use Python dicts, then you already know how to use Pottery. Pottery is useful for accessing Redis more easily, and also for implementing microservice resilience patterns; and it has been battle tested in production at scale.

Patrick DeVivo:

This is a project in Go that compiles to a SQLite runtime loadable extension, which brings Redis commands into a SQL context. This allows you to write SQL queries against data in a Redis instance, using Redis commands like LRANGE as SQL functions.

Experimental for now. But why? Patrick says:

In general, Redis is fairly accessible from many programming languages, and any query using reqlite could probably be implemented in a language of your choice using a Redis client. However, sometimes declarative SQL can be a better choice to express what you’re looking for, and that’s where this project may be most useful.

Kredis (Keyed Redis) comes from the Rails team and lets you interact with Redis-backed keys as coherent Ruby objects rather than isolated procedural commands. Here’s what that might look like when dealing with strings, integers, and JSON:

string = Kredis.string "mystring"
string.value = "hello world!"  # => SET mystring "hello world"
"hello world!" == string.value # => GET mystring

integer = Kredis.integer "myinteger"
integer.value = 5  # => SET myinteger "5"
5 == integer.value # => GET myinteger

json = Kredis.json "myjson"
json.value = { "one" => 1, "two" => "2" }  # => SET myjson "{\"one\":1,\"two\":\"2\"}"
{ "one" => 1, "two" => "2" } == json.value  # => GET myjson

(There are also Redis-backed data structures for counters, enums, flags, lists, unique lists, sets, and slots.)

I love the idea of being able to think at the layer of objects instead of database record manipulation. Dark is doing something similar with its persistence engine, which we discuss on an upcoming episode of The Changelog.

JuiceFS is an open-source POSIX file system built on top of Redis and object storage (e.g. Amazon S3), designed and optimized for cloud native environment. By using the widely adopted Redis and S3 as the persistent storage, JuiceFS serves as a stateless middleware to enable many applications to share data easily.

A solid, brief Q&A with a couple of Redis’ core team members about what’s next for the project now that its BDFL is no more.

Salvatore Sanfilippo:

So, dear Redis community, today I’m stepping back as the Redis maintainer. My new position will be, on one side, an “ideas” person at Redis Labs, in order to provide inputs for new Redis possibilities: I’ll continue to be part of the Redis Labs advisory board. On the other hand however my hands will be free, and I’ll do something else, that could be writing code or not, who knows, I don’t want to make plans for now. However I’m very skeptical about me not writing more code in the future. It’s just too much fun :D

Thank you, Salvatore, for your many years of work on one of my favorite pieces of software.

Lj Miranda explains their architecture decisions with a metaphor I’ve never seen applied to software systems…

In this blogpost, I’ll explain why we need Flask, Celery, and Redis by sharing my adventures in buying McNuggets from Mcdonalds. Using these three (or technologies similar to them) is integral to web backend development so that we can scale our applications.

I love these “why we did X” style posts where folks share their real-world decision making processes and how they played out over time.

Mount RedisWebManager to a path in your Rails router and you’re off to the races. You can view memory and CPU usage, review and edit your keys, and check up on connected clients. There’s also some low-hanging fruit for contributions.

We’ve been tracking the community’s concerns and feedback about Commons Clause fairly well. In this post, Stephen O’Grady basically writes a book on the subject and the impact of this controversial software license.

…the Commons Clause turns open source software into non-open source software, according to the industry’s accepted definition of that term. Specifically it says that the terms of the original open source license notwithstanding, you may not sell software “whose value derives, entirely or substantially, from the functionality of the Software.”

…there are several logical questions to explore regarding the Commons Clause. What are the drivers behind it? What does it mean for the companies that employ it and the wider industry? And lastly, is it a good idea?

Set aside 20 minutes and read this if you care about how this license is becoming popular among those (Redis as of recent) who are protecting their right to generate revenue from their open source code, while removing that ability for everyone else.

In response to the post from Paul Dix on the misunderstandings going on around Redis and the Common Clause license — Matt Klein tweeted:

Won’t defend Redis Labs, this is a dead end move, but there needs to be more recognition that the economics of OSS are fundamentally broken.

In his post he starts by saying…

I want to provide a long form discussion of my two Twitter threads as this topic is nuanced and quite interesting. Note: this post is heavy on opinion and light on facts/references backing up those opinions. Thus, preface everything that follows with “IMO.”

Matt goes on to share some history of open source software and his opinions on modern expectations of software being free and open, startups and open source, and who pays…

There is a big debate underway over Commons Clause and its recent application to certain Redis enterprise add-ons. The Commons Clause license is open source and was drafted by Heather Meeker — whom you might remember from Request for Commits #9.

This language from the license forbids the ability to sell the software (similar to the the Elastic License discussed on The Changelog #292).

…the grant of rights under the License will not include, and the License does not grant to you, the right to Sell the Software.

Steven J. Vaughan-Nichols writes for ZDNet:

Redis Labs has been unsuccessful in monetizing Redis, or at least not as successful as they’d like. Their executives were discovering, like the far more well-known Docker, that having a great open-source technology did not mean you’d be making millions. Redis’ solution was to embrace Commons Clause.

This license forbids you from selling the software. It also states you may not host or offer consulting or support services as “a product or service whose value derives, entirely or substantially, from the functionality of the software”.

I’m really curious to see how this tread plays out as more and more organizations see service providers (cloud hosting, SaaS, etc.) and consultants (support contracts, etc.) “getting rich” off of the projects they work so hard to maintain as open source, while they struggle to find a sustainable model for funding the efforts to keep the open source ship afloat.

Paul Dix shared his thoughts on the subject of Redis and the misunderstandings going on around Redis Common Clause Licensing. Paul writes on the InfluxData blog:

The accusation that RedisLabs did a bait and switch is entirely unfair. They’ve been funding open source Redis development for years and that work is now and will be in the future under the liberal BSD license. It’s not like they tricked a bunch of people into using Redis and pulled the rug out from under them. I’m sure that more than 99.99% of the Redis users are completely unaffected by this. And for those others, it’s not like the code that’s already out there is unusable. To my knowledge they can’t retroactively apply the license. So we’re really only talking about forward development to specific modules (not Redis core).

Paul also shares how he favors open core, and the issues he has with other models to sustain the development of open source at scale.

Open core is a fairly honest way to go about developing open source software. As long as you’re clear about what is open and what is closed.

Bradley Kuhn, Executive Director and President of Software Freedom Conservancy, also shared some thoughts on “Commons Clause” style licenses.

Update 2018/08/24 @ 15:09 — this Twitter thread is a nice read too.

The rumors of Redis taking on a new Creative Common license ARE NOT true.

Antirez (Salvatore Sanfilippo) writes on his personal blog:

Redis is, and will remain, BSD licensed. However in the era of uncontrollable spreading of information, my attempts to provide the correct information failed, and I’m still seeing everywhere “Redis is no longer open source”. The reality is that Redis remains BSD, and actually Redis Labs did the right thing supporting my effort to keep Redis core open.

Here’s what IS happening…

What is happening instead is that certain Redis modules, developed inside Redis Labs, are now released under the Common Clause (using Apache license as a base license). This means that basically certain enterprise add-ons, instead of being completely closed source as they could be, will be available with a more permissive license.

Here’s how Redis is licensed.

This web crawler uses Headless Chrome (via Puppeteer) to crawl dynamically generated websites in addition to typical static HTML crawling. It can also be run distributed across multiple machines for speed’s sake, but that requires Redis for shared cache storage.

A sharded First-in First-out queue using Redis and Resque.

This gem unables you to guarantee in-order job processing based on a shard key. Useful for business requirements that are race-condition prone or needs something processed in a streaming manner (jobs that require preservation of chronological order).

Wynn caught up with Salvatore Sanfilippo to talk about Redis, the super hot key value store.