Introducing tori

2024-06-30T00:00:00+00:00

tori</a> is a tool to track your personal systems’ configurations and replicate them.

For the past 5 months, I’ve been simultaneously using and writing it to manage my main machine’s configuration. By “manage the configuration” what I mean is keeping track of installed packages, configuration files, symlinks, and other settings that can be queried and set through command line interfaces.

After installing a given system, I wanted to get it to the same configuration state I was used to, or to a certain configuration specific to its purpose. Just copying backups would certainly be a very manual task, namely because:

Not all settings live in /etc</code></li>
Some settings must be set using a specific CLI utility</li>
Backups usually carry an overwhelming amount of redundant default configuration you never even touched</li>
It does not track what is changing as you are still using the system</li>
I actually wanted to know what I was tracking</li> </ul> Configuring a system can become a very vague process as you start to lose track of where the changes are being made and what is the specific configuration needed for something to work. Every time you change some configuration file, every time you create a symlink somewhere, that’s all having effects on the system that you may expect to be there in the future, but you may not remember how to accomplish that. This drift between what you have and what you are able to replicate only grows as you keep using your system. To get a better idea, see the code snippet below. It’s from the main file that I use to manage all function calls:xbps_get_many packages check_service dbus check_group audio check_group video copy dhcpcd.conf /etc/dhcpcd.conf place kernel-cmd-line.conf /etc/dracut.conf.d check_link /etc/localtime /usr/share/zoneinfo/America/Sao_Paulo get_nix tailspin tspin get_cargo taplo-cli taplo --locked get_bun bash-language-server get_bin 'https://raw.githubusercontent.com/hackerb9/lsix/master/lsix' lsix check_gsettings gtk-theme 'Plata-Noir' check_gsettings font-name 'Mononoki Nerd Font Regular'</code></pre> What is happening here:

A file named packages</code> containing package names is parsed and all packages are queried and installed by the xbps</code> package manager, if not already installed</li>
Service dbus</code> is enabled if not already enabled</li>
The user is added to groups audio</code> and video</code>, unless already in them</li>
File dhcpcd.conf</code> from the configuration directory’s base</code> directory is checked against the one in the passed path and overwrites it if the user chooses to do so</li>
File kernel-cmd-line.conf</code> from the configuration directory’s base</code> directory is copied into the passed path. If the file already exists or differs, tori will present an error</li>
A symlink is checked to be on /etc/localtime</code> pointing to the passed path. If it doesn’t, it is created or fixed</li>
If not installed, a few packages are installed using different package managers: tailspin</code>, taplo</code> and bash-language-server</code></li>
If absent, an executable for lsix</code> is downloaded from a URL and placed at ~/.local/bin</code></li>
Some gsettings</code> values are read and set if they differ: gtk-theme</code> and font-name</code></li> </ol> Notice how everything is conditioned to the system not already presenting that state? tori aims to be idempotent. Running it twice should do nothing the second time it runs so you can run it multiple times while making changes without any doubled effects. I mentioned a base</code> directory. This is what a sample tori directory would look like in its present state:
. ├── base │ ├── dhcpcd.conf │ ├── kernel-cmd-line.conf │ ├── packages │ └── vars.sh ├── .bkp │ ├── canonical │ │ ├── etc │ │ └── opt │ └── ephemeral │ └── etc ├── src │ ├── checks.sh │ ├── copy.sh │ └── get.sh └── strap</code></pre> What you are seeing in this sample of the directory are the following files and directories:

base</code>: Where you place the configuration files that functions like copy</code> and place</code> will look for and copy into the desired locations</li>
.bkp/canonical</code>: Where tori will look for initial backups and create them if none exists</li>
.bkp/ephemeral</code>: Where tori will place timestamped backups every time a file is modified or overwritten</li>
src</code>: Where the source files live, mostly containing function definitions</li>
strap</code>: The main file used to call the functions</li> </ul> Because I developed tori for my own purposes initially, I didn’t really care to separate the actual source files from the context-sensitive data. While a mistake from a higher level, it allowed me to just keep developing the whole system configuration and the code that tracked it from a single, version-controlled location, amounting to very little complexity. I can’t deny to have enjoyed it so far, but going forward that is going to change. Currently, tori is able to install several package managers and their packages, including xbps, apt, nix, opam, stack, cargo, go, sdkmanager, npm, flatpak and pipx. It can also perform several other tasks:setup programming runtimes for OCaml, Scala (via Coursier), Go, JavaScript (node and bun), Rust (via rustup)</li> generate GPG certificates</li> query and set options with update-alternatives</code> and gsettings</code></li> change the user shell</li> check and enable services (systemd and runit)</li> download pre-built binaries from tarballs and (g)zip files, unpacking and making them executable</li> get files through the network using rsync</li> several other things likely not worth mentioning</li> </ul> The application slowly grew to accommodate many of my needs, but I also made it very hard to share with the world in the process, since I never really meant to go public with it. Portability issues #</a></h2> Despite it being very useful to me in its current state and still being something I actively use every day, a lot of it is hard-coded for my very personal use. It was not written with portability in mind and therefore requires a lot of source-code editing to use in a different system. For instance, when I switched from Debian to Void Linux, most of it broke. I certainly would not expect the package list to be compatible between them, but I realized at that point how tightly it was coupled to Debian. When I started delving deeper into FreeBSD and setting up the system, I kept reaching out to something like tori. But it wasn’t there, and it wouldn’t work even if it were. Something that certainly influenced my desire to write tori was my experience using NixOS, which was full of mixed feelings, but undeniably had good feelings that stuck. I really liked being able to manage the system configuration and packages from a single file. But, at the same time, I felt it was overkill. It was limiting because most of the time you were forced to configure things through its interfaces. It was basically incompatible with what every other Unix system expects, and therefore what people who write software for these systems also expect. I appreciated bringing the system configuration to a centralized file, but I certainly did not want to manage all my ~/.config</code> configuration files from that same place. After writing tori, I can choose what to place under its tracking and what not to. Third-party software still works as both me and its creators expect it to, instead of my system breaking things and needing them to work the way it expects. Glad to reinvent the wheel #</a></h2> While I understand there are very mature and powerful tools to manage a system’s state and reproduce it, I am aiming here for a much simpler use case. I have no intention to see it used in enterprise or distributed systems. It is all about managing how your personal computing is set up and having a backup of how you did it. What I need is not a tool that can orchestrate a fleet of containers running a given configuration. What I need is a tool that can run in a bare system that just got installed and get it to a state that feels useful to me. I do not want it to run instructions over a range of IPs, I just want to be able to check at any time if the system state has diverged from the configuration I am using to track it. I wanted a tool that would help me develop a different habit when I need to make system-level changes. And finally, I suppose I just really wanted to build this. I really enjoy the process of configuring operating systems and learning how they work and differ. And I really wanted to learn more about portable, POSIX-compatible shell scripting. So I decided to rewrite it with portability in mind. I am doing this rewrite in FreeBSD, to put the portability to the test. Once some basic functionality is done, the next step will be bringing it to Void Linux, Debian and NetBSD. tori is a bird that has just hatched, so everything is still very, very crude. At this stage, the docs often show intentions rather than implemented functionality. Still, because it is something I’ve come to depend on, it has this rewarding sense of usefulness behind it. If it sounds interesting to you, take a look. You can follow development at the main repository at Codeberg</a> or through its mirrors on GitHub</a> and elsewhere</a>. Going forward, I will also probably be talking a lot about it on my Mastodon profile</a>.

Meeting the BSD family

2024-05-20T00:00:00+00:00

During this year I have been delving deeper and deeper in the BSD realm. Switching my home server to FreeBSD, trying NetBSD and OpenBSD on my backup machine, getting a cheap SSD to see how they’d all run on my main one, all beaming with the joy of tinkering and learning.

As a nerd who delights in reading documentation, manuals and handbooks, I feel like I have found a gigantic library to lose myself in. And to me the delight of such reading is in that it’s never a passive learning experience, but something you can act on and bring to fruition yourself.

While Linux-based operating systems, with all the popularity they have gained, have developed into a complex and extremely active ecosystem, the BSD operating systems feel less bloated and more focused on whatever their specialty is.

You can’t really complain about software availability, given the amount of pre-packaged binaries you will find. When trying FreeBSD, I could not miss anything I needed. More recently, on NetBSD, I also found most of the tools I reached for.

Though I have a mostly text-driven workflow, doing almost all things with a browser and a terminal alone – which certainly helps in making your stack more portable – I do rely on some GUI applications for the domains where they excel.

What you might experience is a slower pace of change for major things, such as on Wayland adoption, which like it or not is coming for all of us with X deprecation looming.

Running BSD is an incredible opportunity to really learn about UNIX-like systems and operating systems in general.

Recently, I’ve been learning more about NetBSD after spending some time with FreeBSD. And this inner diversity of fully-independent operating systems with their own kernels and perks keeps multiplying the learning opportunities.

If you already learned a lot about whatever OS you currently use, I’d say particularly if that OS is Linux-based, when you start to play with a BSD system you are able to realize what is similar and what is not.

Whatever is different is likely teaching you the more portable, UNIX way of doing things. Even if it isn’t, it’s teaching you how a different OS is designed and behaves.

Things that are the same, which are not few, also offer learning opportunities. You get to see what parts of a Linux-based OS perhaps didn’t really originate there, or aren’t in any way an exclusive feature of it.

Now, to lay any zealousness aside and not make this a saccharine one-sided tale, I’d also like to mention a certain social phenomenon that this endeavour reminded me of.

This is certainly not something specific to BSD, but because it has such an engaged and savvy community, you definitely get to notice it sometimes. I’m talking about the tendency to identify with and then indiscriminately defend the software you use.

One common meme you’ll find is people complaining about lack of hardware support, especially wifi. In response, I’ve seen people stating with little nuance that any difficulty to getting your hardware to work on <insert a BSD OS here></code> is to be explained by poor skills or lack of dedication in reading the documentation.

I see that as denial. When everyone around is just defending something to no end, no critiques allowed, it starts to feel… awkward, to say the least.

Conversely, when I see people openly pointing out weaknesses in something I value and that I can tell they also care for, I feel relief and admiration for that person and that community at large. And thankfully I have also found a lot of this among the BSD folks.

Because running a given operating system on a machine you rely on is such a big commitment, it intensifies this phenomenon where users start to identify with the software they use and defend it beyond reason.

It happens with frameworks, desktop environments and window managers, but operating systems require you to commit even more because you can’t just swap them as easily, so my guess is we identify to compensate this sense of being tied to it. And from this identification comes an urge to deny any defect.

If you are cognizant of the perils, identifying with something is not necessarily a bad thing, though. To some extent, it is inevitable, and being really into something, caring about it, nurturing immense curiosity and a desire to discuss it, are all sources of pleasure I do not excuse myself from.

Software wars aside, getting to know this family of operating systems better has been a joy. It opened up whole new avenues and perspectives to understanding operating systems as a whole, and how beyond Linux-based OSs there are numerous other free and open source operating systems that strengthen the diversity in this field.

jutty.dev - operating-systems

Introducing tori

Meeting the BSD family