Into the Nix ecosystem
This week weāre talking about Nix with Domen Kožar. The Nix ecosystem is a DevOps toolkit that takes a unique approach to package management and system configuration. Nix helps you make reproducible, declarative, and reliable systems. Domen is writing the Nix ecosystem guide at nix.dev and today he takes us on a deep dive on all things Nix.
Matched from the episode's transcript š
Domen Kožar: Alright. Well, thereās quite a lot going on behind the scenes, so letās go through each step. If you say ānix install firefoxā, Nix will first of all try to see where youāre trying to install Firefox from⦠And by default, it will use Nix packages, which is the official, one of the sources it can install from. But it can be anything, so weāll skip that part for now; by default, it will use Nix Packages.
Then there is a top-level file in Nix Packages called all-packages. You can imagine this ā you know Nix language is kind of like JSON with functions⦠So in there youāll see a key Firefox, and it will point then to a file, which will import that file.
Inside the Firefox file (firefox.nix, wherever it is), there is a description of how to build Firefox. In the Nix language there is a primitive called derivation, which is kind of like the core of the whole concept. In the Firefox case, it will say ā you know, thereās a bunch of dependencies, you have to run make with these flags, and a bunch of other things. And this derivation function is really the core of it⦠And what it will do is it will first go through all of the dependencies and build those, of course⦠You know, all to the bottom of it, which is something we call the bootstrap, where we build the minimum possible environment. Then it will build the minimum possible environment. Then it will build all those dependencies up to Firefox. And all of those dependencies go through this derivation function.
What happens in there is the derivation function gets a bunch of inputs, which you can imagine as like key-value pairs, essentially⦠And it passes that to a builder, which is some kind of an executable. By default in Nix all the builders are done in Bash, but you can have an executable as a builder, essentially, and pass all the inputs to it.
This builder, once it will be executed, is run in a sandbox environment; you can imagine this as something like a Docker environment, where it will not have access to the internet, it will be completely isolated from the file system, and so on.
The idea of this sandboxing is, of course, for the build to be reproducible, and only dependent on these inputs. That is one of the core design decisions. And as Iāve previously mentioned, all these inputs are then calculated ā there is a hash calculate out of these inputs, and this uniquely identifies how this package was built, and what is the source of this package. So not just the source, the binary of the thing, but also all the instructions for it.
So the builder will take care of the building part, and this is where I previously talked about evaluation and building separation kicks in. When you will install Firefox, it will find a Firefox file, it will evaluate this first⦠So it will evaluate the Firefox derivation, and then everything up to the bootstrapping bit. And then once thatās done, it will start to build.
[24:14] And the building phase is not that interesting. Thereās essentially two parts to it. One is that it will use these derivation files to call this builder, as Iāve mentioned⦠But before it does that, it will also check with this hash if there exists a binary package for it, and it will substitute it if there is one. And if not, and if not, then it will go and build it.
How that works is that when the package is built, as Iāve mentioned, Nix will put it in the /nix/store/, and then hash and the name of the package, and then everything goes in there. And the same for all the dependencies.
So letās assume now that Nix downloaded some binaries as a dependency of Firefox, and then it builds the Firefox. Now it just has a bunch of folders in /nix/store, and now it will link those into something we would call file system hierarchy, which is the standard youāre used to in Debian, for example, so /user and /opt, and so on. And it will just layer these things essentially together into something Nix calls profile. And this profile is really just one snapshot of when you installed a package, or a group of packages completely linked together.
So thatās how Nix goes from this global store into an actual file system hierarchy that weāre used to. Itās one big assembling farm, the way to imagine it⦠This is then how you go through ā if you install something, it will install all the packages you had before, and then this package on top of it. So itās kind of like immutable; you build up these things. Thatās how I like to imagine this.
And the same when you uninstall, it will not remove the Firefox from the /nix/store directory, but it will just create a new profile version without Firefox linked in. And this is very typical in memory management, where you essentially just allocate, and then you garbage-collect when you want to. Nix works in a similar way. So then actually deleting packages would be an explicit garbage collect operation, where you go through these profile versions and you can say āOh, keep just the last oneā, for example.
So thatās the garbage collection bit⦠But letās go back to installing. Now we have this profile where Firefox was installed in, and Nix will activate it, which means that the specific snapshot of the profile is now the activated one. And these profiles can be stacked one upon another as well, and there is like the user profile, so each user can have a profile; each user can install their own set of packages, and then on NixOS, the distribution, there is also a system profile, which is the actual OS profile, that then represents the environment that you access, and then Nix exposes in a typical package manager Firefox in the path variable, for example.