Alex Kessinger changelog.com/posts

Speed up AWS S3 by 2000x with this transparent proxy

Amazon S3 works pretty well, is cheap, and is not too slow. It is employed as a blob store by so many companies that it’s practically the de facto solution. So, if you could speed up S3 I am sure it would have a pretty big impact. That is exactly what MimicDB is trying to do.

By maintaining a transactional record of every API call to S3, MimicDB provides a local, isometric key-value store of data on S3. MimicDB stores everything except the contents of objects locally. Tasks like listing, searching and calculating storage usage on massive amounts of data are now fast and free.

The readme says that on average tasks like those are 2000x faster using MimicDB. It also reduced the number of API calls to S3 thus reducing the price. If you use S3 heavily, MimicDB looks like it could be an interesting addition to your stack.


Discussion

Sign in or Join to comment or subscribe

Player art
  0:00 / 0:00