Duplicates taking up tons of space on your home NAS? fclones quickly identifies duplicates, even when there’s 10s of thousands that you’re scanning over the network:
fclones treats your data seriously. You can inspect and modify the list of duplicate files before removing them. There is also a –dry-run option that can tell you exactly what changes on the file system would be made.
Also check out the algorithm used to detect duplicates.
Spacedrive helps you organize your files across many devices in one place. Here’s the motivation:
Many of us have multiple cloud accounts, drives that aren’t backed up and data at risk of loss. We depend on cloud services like Google Photos and iCloud, but are locked in with limited capacity and almost zero interoperability between services and operating systems. Photo albums shouldn’t be stuck in a device ecosystem, or harvested for advertising data. They should be OS agnostic, permanent and personally owned. Data we create is our legacy, that will long outlive us—open source technology is the only way to ensure we retain absolute control over the data that defines our lives, at unlimited scale.
Today we’re talking with Zach Lloyd, founder of Warp — the terminal being re-imagined for the 21st century and beyond. Warp is a blazingly fast, rust-based terminal that’s being designed from the ground up to work like a modern app. We get into all the details — why now is the right time to re-invent the terminal, where they got started, the business they aim to build around Warp, what it’s going to take to gain adoption and grow, but more importantly — what’s Warp like today to get developers excited and give it a try.
Still in beta, but Fleet has a lot of promise. It boasts compile times up to 5x faster than cargo. Here’s how:
Fleet works by optimizing your builds using existing tooling available in the Rust ecosystem, including seamlessly integrating sccache, lld, zld, ramdisks (for those using WSL or HDD’s) et al.
Adam Chalmers:
Over the last few weeks I built my own DNS client. Mostly because I thought dig was kinda clunky. Partly because I wanted to learn more about DNS. So here’s how I built it, and how you can build your own too. It’s a great weekend project, and I learned a lot from finishing it.
DNS is simple enough that you can grok it quickly, but still complex enough that Adam learned a ton along the way: reading RFCs, network sockets, Bitvec, and more.
Automerge is a Conflict-Free Replicated Data Type (CRDT), which allows concurrent changes on different devices to be merged automatically without requiring any central server.
I could see this library being super useful in many applications.
So Argc is a little Rust program that your bash scripts can use to make handling command-line parameters easy. You describe the options, parameters, and subcommands in comments like this:
# @describe A demo cli
# @cmd Upload a file
# @arg target! File to upload
upload() {
echo "cmd upload"
echo "arg: target $argc_target"
}
Then you trust Argc to parse that and do the heavy lifting:
eval "$(argc -e $0 "$@")"
The only drawback (aside from potential parsing bugs) to this approach is your script now depends on having argc in the machine’s executable path. i.e. – it’s not portable.
What’d be even cooler, IMHO, would be to instead use argc as a pre-processor of sorts that spits out a pure-bash implementation from your input script. Regardless, a cool idea!
Jujutsu (jj at your CLI) combines features from popular distributed version controls systems (Git, Mercurial, Darcs) with features not found in any of them, such as:
rsync/Dropbox/etcAnd since it’s Git-compatible, adoption should be a breeze.
Jujutsu has two backends. One of them is a Git backend (the other is a native one). This lets you use Jujutsu as an alternative interface to Git. The commits you create will look like regular Git commits. You can always switch back to Git.
I’ve often wondered what will come next after Git. I’m not saying this is that thing, but it’s cool to see people working on the problem.
Regular expressions aren’t easy to write, but they’re especially hard to read. This project is yet another attempt to make them more readable and maintainable. The aim is to create a new language that compiles to regexen so we don’t have to write them directly.
PgBouncer is a popular connection pooler for Postgres. PgCat is that same functionality, only in Rust. This is alpha software and there is no worse place to run alpha software (IMHO) than in places that have direct connections to your database. Still, this is worth linking to because it’s yet another instance of open source infrastructure code getting Rusty.
PRQL (pronounced “Prequel”) aims to be “a simpler and more powerful SQL”
Like SQL, it’s readable, explicit and declarative. Unlike SQL, it forms a logical pipeline of transformations, and supports abstractions such as variables and functions. It can be used with any database that uses SQL, since it transpiles to SQL.
To get an idea on PRQL’s design, they provide this SQL statement as an example:
SELECT TOP 20
title,
country,
AVG(salary) AS average_salary,
SUM(salary) AS sum_salary,
AVG(salary + payroll_tax) AS average_gross_salary,
SUM(salary + payroll_tax) AS sum_gross_salary,
AVG(salary + payroll_tax + benefits_cost) AS average_gross_cost,
SUM(salary + payroll_tax + benefits_cost) AS sum_gross_cost,
COUNT(*) as count
FROM employees
WHERE salary + payroll_tax + benefits_cost > 0 AND country = 'USA'
GROUP BY title, country
ORDER BY sum_gross_cost
HAVING count > 200
And then translate it to PRQL, which looks like:
from employees
filter country = "USA" # Each line transforms the previous result.
let gross_salary = salary + payroll_tax # This _adds_ a column / variable.
let gross_cost = gross_salary + benefits_cost # Variables can use other variables.
filter gross_cost > 0
aggregate by:[title, country] [ # `by` are the columns to group by.
average salary, # These are the calcs to run on the groups.
sum salary,
average gross_salary,
sum gross_salary,
average gross_cost,
sum gross_cost,
count,
]
sort sum_gross_cost # Uses the auto-generated column name.
filter count > 200
take 20
System76’s COSMIC desktop environment is still in early development, but here’s an early preview that might pique your interest. Love that launcher. 🤩
Lee Robinson from Vercel/Next.js:
Rust is now replacing parts of the JavaScript web ecosystem like minification (Terser), transpilation (Babel), formatting (Prettier), bundling (webpack), linting (ESLint) and more. Let’s take a deep dive into why this trend is gaining popularity and wider adoption.
We just discussed this topic (and some apprehension about it) on JS Party last week. Definitely a trned worth brushing up on, if you haven’t already.
Reshape… automatically handles complex migrations that would normally require downtime or manual multi-step changes. During a migration, Reshape ensures both the old and new schema are available at the same time, allowing you to gradually roll out your application.
Designed for Postgres 12+ and experimental for now. Do not use in production until it shapes up.
If you ever get into an “oh snap!” situation where you think you just lost days of work, checkout a
durabranch and recover.Without
dura, you use Ctrl-Z in your editor to get back to a good state. That’s so 2021. Computers crash and Ctrl-Z only works on files independently. Dura snapshots changes across the entire repository as-you-go, so you can revert to “4 hours ago” instead of “hit Ctrl-Z like 40 times or whatever”. Finally, some sanity.
This sounds a lot like how Fossil works out of the box, except maybe even more paranoid because Fossil will propagate every commit whereas Dura propagates all changes even before they’re committed.
Check out the original RFC to find the genesis of this effort. Here’s the opening pitch:
We know there are huge costs and risks in introducing a new main language in the kernel. We risk dividing efforts and we increase the knowledge required to contribute to some parts of the kernel.
Most importantly, any new language introduced means any module written in that language will be way harder to replace later on if the support for the new language gets dropped.
Nevertheless, we believe that, even today, the advantages of using Rust outweighs the cost. We will explain why in the following sections.
Godspeed to everyone collaborating on what will surely be a massive, years-long undertaking.
Bryan Cantrill, announcing Hubris and Humility:
… we found ourselves increasingly forcing existing systems out of the comfort of their design centers, we wondered: was our assumption of using an existing system wrong? Should we in fact be exploring our own de novo operating system? In particular, our colleague Cliff Biffle, who had a ton of experience with both Rust and embedded systems, had a vision for what such a system might look like (namely, the system that he had always wanted for himself!). Cliff dove into a sketch of his ideas, giving the nascent system a name that felt perfectly apt: Hubris.
There’s some serious innovation going on at Oxide and I can’t wait to see what they do next.
Blueboat aims to be a developer-friendly, multi-tenant platform for serverless web applications. A monolithic approach is followed: we try to implement features of commonly used libraries (in the web application context) natively in Rust to replace native Node addons, improve performance and reduce duplicated code. Blueboat’s architecture ensures the security of the platform, prevents code duplication and keeps the overhead low.
elfshaker is a CLI tool written in the Rust programming language. It stores snapshots of directories into highly-compressed pack files and provides fast on-demand access to the stored files. It is particularly good for storing lots of similar files, for example object files from an incremental build.
They’ve had phenomenal results when using this with clang builds.
This is a fun post about the inner workings of terminal emulators. In it you’ll find:
Have you heard? The just-released 12th version of Next.js uses a Rust-based compiler (swc) to achieve ~3x faster local refreshes and ~5x faster production builds.
Compilation using Rust is 17x faster than Babel and enabled by default using Next.js 12, replacing transforming JavaScript and TypeScript files. This meant we had to port the Babel transformations in Next.js to Rust, including a brand new CSS parser in Rust used to implement the
styled-jsxtransform.
With the increased popularity of Rust and Go-based JS tooling, could there be a future where we don’t use any JavaScript to build our JavaScript apps? 🤔
Alan Smithee on why Rust’s high performance, reliability, and productivity make it a good fit for embedded systems.
Seed is a Rust framework that uses an Elm-like architecture to help you build fast and reliable web apps that run on WebAssembly. Here’s why you might want to use it:
Seed allows you to develop the front-end with all the benefits of Rust, meaning speed, safety, and too many more things to count.
The Seed templating system uses a macro syntax that makes Rustaceans feel right at home. This means linting, formatting, and commenting will work, and it’s all in Rust. This is opposed to a JSX-like syntax that relies on IDE extensions to improve the developer experience.
Seed has a batteries-included approach. This means less time writing boilerplate and less time installing dependencies.
And a few reasons why you might not:
- It’s newer. It’s harder to find support outside of Discord.
- WebAssembly is newer with less support. Browser compatibility is at 92.9%.
- Pre-built components are rare. You will likely have to roll your own components such as date pickers.
- No server-side rendering yet
- You may prefer other Rust frameworks like MoonZoon or Yew.
git-cliff can generate changelog files from the Git history by utilizing conventional commits as well as regex-powered custom parsers. The changelog template can be customized with a configuration file to match the desired format.
htmlq uses CSS selectors to extract bits of content from HTML files. Mozilla’s MDN has a good reference for CSS selector syntax.
This looks super handy. Examples!
// Find part of a page by ID
curl --silent https://www.rust-lang.org/ | htmlq '#get-help'
// Find all links in a page
curl --silent https://www.rust-lang.org/ | htmlq --attribute href a
// Get the text content of a post
curl --silent https://nixos.org/nixos/about.html | htmlq --text .main