Ars Technica gave some of our favorite command-line tools the deep-dive they deserve:

Instead of giving you encyclopedic listings of every possible argument and use case for each of these ubiquitous commands, we’re going to teach you how to think about them—and how to easily, productively incorporate them in your own daily command-line use.

This is a solid primer on the usefulness of jq (a lightweight, command-line JSON processor.)

In this article, I’m going to go over the basics building blocks of jq in enough depth that you will be able to understand how jq works. Of course, you still might occasionally need to head to google to find a function name or check your syntax, but at least you’ll have a firm grounding in the basics.

This one is for Jerod since he’s such a Nick Miller fan 🤣

Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON

This comment on HN does a great job summarizing Bashly.

…think of this as an argparse equivalent for Bash. You provide a YAML file listing commands, subcommands, arguments, and flags, and it automatically generates a Bash script that can parse and validate them, provide help messages, and run your code for each command.

It also lets you keep the actual code for each command and subcommand in separate files, which are merged together into one distributable Bash script at generation time. It’s basically a templating system to auto-generate argument parsing so you don’t have to solve that again or deal with things like optparse.

Brett Cannon:

… over 3 years ago I set out to re-implement the Python Launcher for Unix in Rust. On July 24, 2021, I launched 1.0.0 of the Python Launcher for Unix… This gives you a py command on Unix which will always use the newest version of Python.

He goes on to describe some workflow niceties that a built in and also what this project is not about:

The Launcher is purely a convenience and not meant to be The Launcher For All Things; this should never end up in a Docker container.

Nick told us about this on our modern Unix tooling episode, but I thought I’d link up his excellent writeup/video on the subject for those who had a hard time following with audio only.

Let’s say you have an nginx or Kubernetes config file which doesn’t support templating out of the box and you want to dynamically create config files based on 1 or more environment variables. This is what envsubst lets you do.

This week we’re talking with Nick Janetakis about modern unix tools, and the various commands, tooling, and ways we use the commmand line. Do you Bash or Zsh? Do you use cat or bat? What about man vs tldr? Today’s show is a deep dive into unix tools you know and love, or should know and maybe love.

Angle-grinder allows you to parse, aggregate, sum, average, min/max, percentile, and sort your data. You can see it, live-updating, in your terminal. Angle grinder is designed for when, for whatever reason, you don’t have your data in graphite/honeycomb/kibana/sumologic/splunk/etc. but still want to be able to do sophisticated analytics.

Angle grinder can process well above 1M rows per second (simple pipelines as high as 5M), so it’s usable for fairly meaty aggregation. The results will live update in your terminal as data is processed. Angle grinder is a bare bones functional programming language coupled with a pretty terminal UI.

I’m not gonna lie, they had me with the name on this one.

Difftastic is an experimental structured diff tool that compares files based on their syntax. It is very much unfinished. It works reasonably on very parenthesised data (lisps, JSON), it works sometimes on other languages with sufficient parentheses (Rust, JS), and falls back to a line-oriented diff otherwise.

A bad CLI can easily discourage users from interacting with it. Building successful CLIs requires attention to detail and empathy for the user in order to create a good user experience. It is very easy to get wrong.

In this guide I have compiled a list of best practices across areas of focus which aim to optimize for an ideal user experience when interacting with a CLI application.

You’ve probably seen many of these commands hit Changelog News over the years, but now you can see them all again in one hand-curated place. Who knows, maybe one or two will be new to you. I hadn’t heard of curlie previously, which looks like a nice merging of cur and httpie.

This isn’t merely a speech-to-text thing. It also provides intent recognition, which makes it great for doing voice commands. For example, when trained with this template, the following command:

$ voice2json transcribe-wav \
      < turn-on-the-light.wav | \
      voice2json recognize-intent | \
      jq .

Produces this JSON event:

{
    "text": "turn on the light",
    "intent": {
        "name": "LightState"
    },
    "slots": {
        "state": "on"
    }
}

And it can be retrained quickly enough to do it at runtime. Cool stuff!

rpg-cli is a bare-bones JRPG-inspired terminal game written in Rust. It can work as an alternative to cd where you randomly encounter enemies as you change directories.

You’ll want to practice a bit first, then once you get good at it go ahead and override the builtin cd by adding this function to your bash profile.

cd () {
   rpg-cli "$@"
   builtin cd "$(rpg-cli --pwd)"
}

Bash is great, but when it comes to writing scripts, people usually choose a more convenient programming language. JavaScript is a perfect choice, but standard Node.js library requires additional hassle before using. zx package provides useful wrappers around child_process, escapes arguments and gives sensible defaults.

I wouldn’t say JavaScript is a perfect choice for this kind of scripting, but it’s definitely a suitable one (especially if it’s the language you already know well). Here’s what scripting looks like with zx:

#!/usr/bin/env zx

await $`cat package.json | grep name`

let branch = await $`git branch --show-current`
await $`dep deploy --branch=${branch}`

await Promise.all([
  $`sleep 1; echo 1`,
  $`sleep 2; echo 2`,
  $`sleep 3; echo 3`,
])

let name = 'foo bar'
await $`mkdir /tmp/${name}`

Top-level await sure makes things nicer. (Deno supports this out of the box, btw.)

Thanks to Muhammad Raza for sharing some of his favorite command line one liners to make his workflow productive and more efficient.

As a regular Linux user, I have been using commandline extensively to perform daily tasks such as creating files, navigating through directories, moving files and editing files using vim.

You can think of Julia Evans’ new dnspeep tool as similar to tcpdump but specifically for watching your machine’s DNS queries.

One thing I like about this tool is that it gives me a sense for what programs on my computer are using the Internet! For example, I found out that something on my computer is making requests to ping.manjaro.org from time to time for some reason, probably to check I’m connected to the internet.

A friend of mine actually discovered using this tool that he had some corporate monitoring software installed on his computer from an old job that he’d forgotten to uninstall, so you might even find something you want to remove.

It also probably comes in handy when debugging those pesky “could it be DNS?” issues, but this might be a limitation on that front:

One thing this program doesn’t do is tell you which process made the DNS query, there’s a tool called dnssnoop I found that does that. It uses eBPF and it looks cool but I haven’t tried it.

This doesn’t aim to entirely replace find and ls, but if you already know SQL (like many of us do), why not be able to leverage that knowledge for your more advanced file-finding needs? Here’s a couple of examples so you get the idea:

Find temporary or config files (full path and size):

fselect size, path from /home/user where name = '*.cfg' or name = '*.tmp'

Use aggregate functions:

fselect "MIN(size), MAX(size), AVG(size), SUM(size), COUNT(*) from /home/user/Downloads"

Find by date and time intervals:

fselect path from /home/user where modified gte 2017-05-01

grex is a library as well as a command-line utility that is meant to simplify the often complicated and tedious task of creating regular expressions. It does so by automatically generating a single regular expression from user-provided test cases. The resulting expression is guaranteed to match the test cases which it was generated from.

Unfortunately, the tool’s authors still think you need learn how to write regexes, even when grex works flawlessly (and I tend to agree with them).

monsoon is a so-called command-line HTTP enumerator: A tool that iterates over a list of values, for example a word list or a range of integers, and sends one HTTP request per item towards a given server.

The team behind monsoon enumerated some common examples in their introductory blog post.

jq is a hugely useful tool for anyone dealing with JSON of varying shapes and sizes. If that’s you, but you haven’t given jq a serious try, this is a great little primer on its use and use cases.

Matthew Monahan:

dug is designed to help you check the global status of your DNS records. You can use the built in servers, update them from remote or local sources, specify servers, whatever. It also supports templated output in CSV or JSON for use in monitoring applications or just piping the results around.

Crush is an attempt to make a traditional command line shell that is also a modern programming language. It has the features one would expect from a modern programming language like a type system, closures and lexical scoping, but with a syntax geared toward both batch and interactive shell usage.

Check out the overview of features right here.

From the foreword:

Most people today don’t know what the command line is, much less why they would want to bother with it. As computing pioneer Alan Kay said in a 2017 interview, “Because people don’t understand what computing is about, they think they have it in the iPhone, and that illusion is as bad as the illusion that ‘Guitar Hero’ is the same as a real guitar.”

Off to a good start…

Inspired by traditional UNIX philosophy, driven by an interest in encouraging a more delightful and accessible CLI environment, and guided by our experiences as programmers, we decided it was time to revisit the best practices and design principles for building command-line programs.

Used to be written in Python, but is now 100% Rust.

It was designed to help remind *nix system administrators of options for commands that they use frequently, but not frequently enough to remember.

Let’s imagine a completely hypothetical world where it’s the umpteenth time you’ve used it, but you still can’t remember which flags to send to tar… so you run:

cheat tar

You’ll be greeted by:

# To extract an uncompressed archive:
tar -xvf '/path/to/foo.tar'

# To extract a .gz archive:
tar -xzvf '/path/to/foo.tgz'

# To create a .gz archive:
tar -czvf '/path/to/foo.tgz' '/path/to/foo/'

# To extract a .bz2 archive:
tar -xjvf '/path/to/foo.tgz'

# To create a .bz2 archive:
tar -cjvf '/path/to/foo.tgz' '/path/to/foo/'

The cheatsheets themselves are community-sourced.