Antirez has returned to Redis! Yes, Salvatore Sanfilippo (aka Antirez), the creator of Redis has returned to Redis and he joined us to share the backstory on Redis, what’s going on with the tech and the company, the possible (likely) move back to open source via the AGPL license, the new possibilities of AI and vector embeddings in Redis, and some good ’ol LLM inference discussions.
Salvatore Sanfilippo: Actually, thank you for using it, because if you think at Redis, even if it compiles certain ideas in new ways, they are all old ideas. So I believe that software is mainly a platform for people to communicate concepts. And without a community – Redis was so resilient; it lasted so much time because the community created a mental framework about fixing problems with such tools that are inside Redis. So basically, the community is Redis, in some way.
And the way it started was because – my first days as a programmer were all about security. I worked in security for a couple of years. Then what happened in security is what is happening right now to software. It started to be cool, it started to be business, a lot of business. And security was no longer research, it was a product. And I grown immediately tired of this industry, so I started to write embedded systems.
[00:07:52.07] And then I started to get a bigger project, because I was in a very bad situation in my life. I had struggles with my relationships, I had a son that was like three years ago, and I was - I could say depressed, but it’s impossible for me to get depressed physiologically, because the kind of attitude I have in general. But I was low in my life, and I needed a project to focus my energies.
So the first weeks I just watched Star Trek Voyager nonstop, and South Park… And then I started to work to an interpreter for the Tcl programming language, for embedded systems. And much of this code was the foundation to make Redis one year later.
So basically, with this interpreter, which was not just a toy interpreter, it was compatible with all the Tcl programming language set, extended it in many ways… It added continuations and other stuff, and was in some way, in some things even faster, and so forth. Finally, it clicked something in my mind about how to design bigger systems. But then I forgot about this, and since I had no money, I remember that in this period in my life, I often looked under in my car to see if maybe one, two euros was around. And then I asked even for money to the girlfriend I had at the time. So I said, “Okay, that’s not possible. I need to do something.” And so a friend of mine [unintelligible 00:09:47.11] went and said “Do you want to collaborate? Let’s do something together.” And this was the start of Web 2.0 in the United States. But in Italy, there was still – what was Alta Vista, basically, in the U.S. So these portals, static portals, no user generated content. So we said, “Probably if we just copy what they are doing, we will have an edge.”
And we created two services. One was exactly like Delicious, and another one was an improved Digg, much more like Reddit now, basically. So there was complex moderation, there was a Tcl program that did the advanced filtering in order to avoid friends voting themselves. And this became popular in Italy, and so Telecom Italia, which is the biggest ISP here in Italy, a telephone company, called us. They didn’t know in any way us. They called and said, “Why you don’t join the Telecom Italia network with your services?” [unintelligible 00:11:01.01]
I remember when we went in Milan I was dressed so badly that they didn’t want to let me at the hotel in. They’re like “Are you sure that you have a reservation in this hotel?” I said, “Call this number.” They said “Okay, you can enter.” Because we were underground people. In Palermo we had Linux systems with cables in order to pay, and we had illegal greens to connect to the telephone to internet… We did this kind of thing, so this was our mood. They were business people, so there was a cultural gap, in some way.
And so we entered the network, and from one day to the other, we had to sustain the traffic of a lot of users, all the Italy connected. And our systems were not ready. So the first thing that I did was in PHP - the system was written in PHP - to write a memoization function that basically whatever you could call inside this function, the return value was stored, serialized into some kind of BTree that I created on the disk.
[00:12:22.15] And this way we had some kind of cache in order to avoid stressing MySQL too much. And this worked the first weeks, but then we created a new service that was a real-time log analyzer, and the use case was not static caching. You know, till this point I didn’t know even that Memcached existed. So I could not use it, just because of the lack of knowledge of existence.
But even if I had Memcached, that was not enough for me, because I needed a cache with data structure where I could add items and remove all the items. So I took Tcl and wrote a prototype, and they tested it, and it worked so well. And they said, “Okay, that’s good, but it’s not a scripting language thing. I want an actual daemon in the ground, with a low memory footprint.” And especially, I realized that I wanted a fork in order to persist on disk with the other child.
And since I in the past had background in security and low-level programming for cryptography, I studied the very in-depth the Stevens books, all the system call of Unix, the implementation of small Unix systems like Minix, provided a few patches to the TCP/IP stack of Linux… So I was a very low-level already, and I realized that with fork - this is an interesting story, because fork, after 15 years, still is the best way to do the Redis thing from the point of view of persistence; this magical thing of copy and write of the kernel pages. So this fundamental idea back then survived to these days.
And so I started to write this Redis thing, and it took me two weeks to have the basic core in order to apply it in our system. And since I had a long story of releasing open source software, [unintelligible 00:14:34.05] device drivers for the Canon cameras, web log analyzers, many projects that were in Debian and in other Linux distributions, I said “You know what? If I release Redis, this will not impact in any way our business”, because the other application was closed source. So I released it on Hacker News, because Hacker News was very new when I released Redis in Hacker News… There were like 10 upvotes, a few people commented, but basically nobody cared at all. But Ezra Mobius…