Databases Icon

Databases

Databases, structured data, data stores, etc.
122 Stories
All Topics

JS Party JS Party #276

The ORMazing show

Nick & KBall sit down with the brilliant Stephen Haberman to discuss all things ORMs! šŸ’»šŸ”

From the advantages and disadvantages of ORMs in general, to delving into the intricacies of his innovative project Joist, which brings a fresh, idiomatic, ActiveRecord-esque approach to TypeScript. šŸš€

So sit back, relax, and let’s dive deep into the world of ORMs with the experts!

SQLite unixsheikh.com

SQLite the only database you will ever need in most cases

Anybody following our news and/or podcast feeds for a bit already know of SQLite and have likely used it and continue to use it, but for those just joining us, here’s yet another dev with effusive praise for the little database engine that could:

SQLite is one of those projects that I wish I had known about long before I did. I had heard about it, but for many years I never thought about taking a serious look at it because I was under the false impression that is was a tiny database only useful for personal address books and small embedded devices.

I highly recommend you take a look at SQLite!

Tim github.com

Transform Gist into your personal key/value data store

Sometimes all a project needs is the ability to read/write small amounts of JSON data and have it saved in some persistent storage. Imagine a simple data-model which receives infrequent updates and could be represented as JSON object. It doesn’t demand a full-blown database, but it would be neat to have a way to interact with this data and have it persist across sessions.

This is where gist-database comes in handy, by leveraging the power of the gist api you can easily create a key/value data-store for your project.

This is a perfect solution for low write / high read scenarios when serving static site content with Next.js and using Incremental Static Regeneration to keep your cached content fresh.

Databases brandur.org

An alternative to `deleted_at` for soft record deletion

You’ve probably done the whole deleted_at thing to mark records as deleted without actually deleting them from the database. That strategy is fraught for a few reasons.

In this post, Brandur Leach proposes an alternative strategy with the following endorsement:

Speaking from 30,000 feet, programming is all about tradeoffs. However, this is one of those rare places where as far as I can tell the cost/benefit skew is so disproportionate that the common platitude falls flat.

Anytime your fellow developer can make that statement in earnest, whatever they’re talking about is worth a try…

Saul Pwanson visidata.org

Connect VisiData to SQL Databases with vdsql

vdsql is a new plugin that allows VisiData to connect to databases and query them directly, using the database’s own query engine. It uses Ibis to generate SQL for many popular backends, including Postgres, DuckDB, Clickhouse, and more.

vdsql v0.2, released this past week, is already quite useful, and development continues to improve both vdsql and VisiData for bigger data!

Changelog Interviews Changelog Interviews #510

Taking Postgres serverless

This week we’re talking about serverless Postgres! We’re joined by Nikita Shamgunov, co-founder and CEO of Neon. With Neon, truly serverless PostgreSQL is finally here. Neon isn’t Postgres compatible…it actually is Postgres! Neon is also open source under the Apache License 2.0.

We talk about what a cloud native serverless Postgres looks like, why developers want Postgres and why of the top 5 databases only Postgres is growing (according to DB-Engines Ranking), we talk about how they separated storage and compute to offer autoscaling, branching, and bottomless storage, we also talk about their focus on DX — where they’re getting it right and where they need to improve. Neon is invite only as of the recording and release of this episode, but near the end of the show Nikita shares a few ways to get an invite and early access.

JS Party JS Party #245

Launching Platformatic DB šŸš€

Patformatic co-founders Matteo Collina & Luca Maraschi join Amal & Chris to discuss their just-announced (and we mean just announced) open source database tool: Platformatic DB!

It’s a daemon that can turn any PostgreSQL, MySQL, MariaDB, or SQLite database into a REST and GraphQL endpoint. What makes it special is that it allows massive customization thanks to the flexibility of Fastify plugins.

GitLab shekhargulati.com

My notes on GitLab’s Postgres schema design

Reading other people’s code is a sure-fire way to improve as a developer. But what about as a database designer? The same process applies!

This post by Shekhar Gulati is him sharing what he learned by applying that principle to GitLab’s schema.

I learnt a lot from the GitLab schema. They don’t blindly apply the same practices to all the table designs. Each table makes the best decision based on its purpose, the kind of data it stores, and its rate of growth.

More posts like this, please!

Databases brandur.org

Soft deletion probably isn't worth it

This article confirms my biases because I’ve always despised every soft delete implementation I’ve come up with. Most of them have looked something like what the author describes:

the technique has some major downsides. The first is that soft deletion logic bleeds out into all parts of your code. All our selects look something like this:

SELECT *
FROM customer
WHERE id = @id
    AND deleted_at IS NULL;

And forgetting that extra predicate on deleted_at can have dangerous consequences as it accidentally returns data that’s no longer meant to be seen.

ORMs help with this, but not enough. You set it as a default scope and then there’s that one time where you also want the deleted records so you come up with a custom query or dig into your ORM and try to find how to bypass the rule. Yuck!

He goes on to describe other problems as well. Maybe it’s all a big case of YAGNI?

Once again, soft deletion is theoretically a hedge against accidental data loss. As a last argument against it, I’d ask you to consider, realistically, whether undeletion is something that’s ever actually done.

When I worked at Heroku, we used soft deletion.

When I worked at Stripe, we used soft deletion.

At my job right now, we use soft deletion.

As far as I’m aware, never once, in ten plus years, did anyone at any of these places ever actually use soft deletion to undelete something.

Ship It! Ship It! #59

Postgres vs SQLite with Litestream

Ben Johnson, the creator of Litestream, joined Fly.io a few weeks after we migrated changelog.com - episode 50 has all the details. That was pure coincidence. What was not a coincidence, is Gerhard jumping at the opportunity to talk to Ben about Postgres vs SQLite with Litestream.

The prospect of running a cluster of our app instances spread across all regions, with local SQLite & Litestream replication, is mind boggling. Let’s find out from Ben what will it take to get there. Thanks Kürt for kicking off this dream.

Tooling prql-lang.org

PRQL is a modern language for transforming data

The P in PRQL (pronounced ā€œPrequelā€) stands for Pipelined, which I’m convinced is a great way of writing and reasoning about queries:

A PRQL query is a linear pipeline of transformations

Each line of the query is a transformation of the previous line’s result. This makes it easy to read, and simple to write.

It compiles to SQL, which means it’s compatible with most databases already and there are currently bindings for Python, JS & Rust, which is the compiler itself.

Try it out in their web-based playground. (Thanks, Wasm!)

Python github.com

Efficiently diff rows across two different databases

  • ⇄ Verifies across many different databases
  • šŸ” Outputs diffs of rows in detail
  • 🚨 Simple CLI/API to create monitoring and alerts
  • šŸ” Bridges column types of different formats and levels of precision
  • šŸ”„ Verify 25M+ rows in <10s, and 1B+ rows in ~5min
  • ā™¾ļø Works for tables with 10s of billions of rows

Databases github.com

Dragonfly – a cost-effective, high-performing, and easy-to-use Redis replacement

Dragonfly is a modern in-memory datastore, fully compatible with Redis and Memcached APIs. Dragonfly implements novel algorithms and data structures on top of a multi-threaded, shared-nothing architecture. As a result, Dragonfly reaches x25 performance compared to Redis and supports millions of QPS on a single instance.

Dragonfly – a cost-effective, high-performing, and easy-to-use Redis replacement

Databases github.com

Grist is a lot like Airtable, but open source and more customizable

In their own words:

Grist is a modern relational spreadsheet. It combines the flexibility of a spreadsheet with the robustness of a database to organize your data and make you more productive.

Since so many people make the Airtable comparison that I did in the headline, the team behind Grist has written up a comparison of the two offerings.

CSS leemeichin.com

Yes, I can connect to a DB in CSS

Just wow. This is an impressively hacky hack. You’re probably wondering how? As many such things do, it all starts with a new (Chrome-only) API:

A new set of APIs affectionately known as Houdini give your browser the power to control CSS via its own Object Model in Javascript. In English, this means that you can make custom CSS styles, add custom properties, and so on…

And it ends with something that looks like this:

main {
  // ...
  --sql-query: SELECT name FROM test;
}
Player art
  0:00 / 0:00