Half an year on Alpine: just musl aside

2025-08-31T20:05:26-03:00

Warning: Contains tongue-in-cheek language that might feel provocative if you have invested part of your identity into your software choices.^{[1]</a>}

For roughly six months now, I’ve been running Alpine</a> on my laptop as a daily driver.

Before that, I was running Void</a>. I like how blazing fast Void is to start up and shut down, has an excellent package management system and its init system, runit, is beautifully integrated. Alpine also fits this description.

When I first learned about Void, I read it described itself as “rolling but stable”, meaning you get new package versions often, but not to the point of breakage. While I never had an upgrade break a system, it still bothered me to upgrade and suddenly have sound or Bluetooth not work anymore, or have the system Python version change frequently, leading to all sorts of complaints from this or that application I didn’t even know was a Python program.

I wanted to go back to a more stable distribution, but I also wanted a simpler, faster init system than systemd. Alpine, which has fixed releases every six months, eventually rose as an alternative.

Expand for an overly detailed rundown of how I narrowed down my options </summary>
How many options, really?</h2>
Preferably not rolling. Preferably without systemd. At first glance it may seem like the choices of operating systems and distributions are endless, but turns out these two requirements are enough to nearly wipe them out.
My main reason to not like systemd is not that I believe Red Hat is involved in a deep state conspiracy to turn frogs gay. In fact, I’d take systemd over SysV-style init on a heartbeat. What I prefer are the sweet spots between them, not the regression into the previous mess.
My reasoning is actually mostly around speed. Call me impatient, and I know it can be tweaked to improve this, but I don’t like it when a machine is coming up or going down and you get a timer from some random process saying it will wait X minutes to exit. I’d rather have a simpler tool tackling initialization and shutdown, and have learned to compose the operating system with various programs that fill in for systemd’s vast feature set.
Admittedly, I also have an insatiable curiosity for getting to know and experiment with how different operating systems work. Before this experience, I knew nothing about OpenRC and several other peculiarities Alpine brings to the table. So no regrets there.
I also did not want a distribution that is too new or too niche, as I don’t appreciate being stuck in dead ends too often. Just sometimes. I am curious about fringe operating systems, but I am looking for something to daily drive and work on, not experiment with.
The systemd realm</h3>
There’s a plethora of Linux distributions. But the vast majority are based on some other one. I don’t mean that this isn’t meaningful for whatever use case they are suited for — some derivatives do go the extra mile and increment their bases in more advanced ways — but when you are looking for a distribution that you want to set up your way, it makes less sense to go for one whose selling point is how it’s already set up for something.
If we drop the derivatives and focus on some arbitrary measure of adoption, we end up with Debian, Fedora, Arch, OpenSUSE and Nix.
I never seriously tried OpenSUSE or Fedora. Among these three, I think I prefer Debian simply because its governance model seems (feels?) healthier and more independent/resilient.
NixOS is its own kind of hell. It truly sets it apart. If you are grasping my preference for compatibility and ease of running experimental programs, you can probably guess how I felt about not even having a standard filesystem hierarchy.
What I am saying about stability and preferring fixed over rolling release should also explain why Arch is not an option either, although I’d say its cultural issues far outweigh anything else for me.
The non-systemd realm</h3>
Then, for all the other ways to summon an operating system from its slumber, we have the BSDs, Gentoo, Slackware, Guix, Void and Alpine. And because changing the init system is significantly more effort than just providing some configuration on top of another system, I think it’s fair to include derivatives here, such as Devuan and antiX.
I run FreeBSD on a separate, desktop machine. I use it a lot over SSH and have also set up a graphical session there. It is a delight to use and learn about, particularly because of its Unix heritage, first-class ZFS support and how different its ecosystem is in comparison. But compatibility-wise (hardware and software), it doesn’t fit the bill for this laptop either. Something (roughly) similar can be said of the other BSDs.
I have tried Gentoo, with binary packages enabled and all, but who has time for compiling a browser? Certainly not me, with my old refurbished computers. There is Guix, but my experience with Nix was a portability nightmare that, while not as wasteful of my time as Gentoo, gets close. And while definitely a control freak, I am not at the Slackware level of control freakiness. As you can see, usability and practicality have grown on me with all this messing around, so now I’d rather not have “just install everything” proposed as the solution to dependency resolution.
You see, it’d be much easier not to be such a nerd. When I set up a computer for someone else, it’s Windows or LMDE or Fedora. It’s such a no-brainer. The relationship with a computer can be simple: like/dislike, works/doesn’t work, can fix/can’t fix. One can actually accept that something can’t be fixed or changed! Really admirable. But that’s not me. I actually care what is the DHCP daemon, who’s resolving the DNS queries and how aggressive the OOM killer is going to be.
And because Void is a rolling release distribution, the only survivor of all this begging and choosing is Alpine. </details>
Choosing a path forward</h2>
So, prompted by a charger dying in the beginning of this year, which put a machine out of work, I decided to take the opportunity to try another system with a slower release cycle and started weighing my options.
What remained in the end were Devuan, antiX and Alpine. And I briefly tried all three during this experiment before settling on Alpine.
Devuan and antiX do offer options besides SysV-style init, but they don’t really integrate them very well in their systems. While with Void you will really use runit to manage any services installed through the distribution’s package manager, because when a package has a relevant service to ship it will ship a runit service, Devuan and antiX have other init systems as options rather than the distribution’s init system.
What you really get then is a mixture of runit/OpenRC and SysV-style init, leaning a lot more towards SysV-style and leveraging the runit/OpenRC ability to also run SysV-style init scripts.[2]</a> If you like managing SysV-style init, that should not be a problem, although in that case you might as well get the SysV-style version then! Alpine, on the other hand, just like Void, has OpenRC as the way to run services.
Alpine it is</h2>
For starters, I wanted to learn a little bit more about this unknown-to-me init system, OpenRC, since I’d be spending some time setting up services with it.
OpenRC can be used alongside or in place of SysV-style init. I will not get into the different roles played by so-called init systems and process supervisors, but OpenRC provides openrc-init</code>, which can replace /sbin/init</code> for a fully OpenRC-powered system.
I could not find explicit information on whether this is the case with Alpine, but an actual inspection of the running system shows it is not. Mind you, even Alpine’s /sbin/init</code> is actually BusyBox:
% file $(which init) /sbin/init: symbolic link to /bin/busybox </code></pre> I then checked out BusyBox’s manual page to try and understand more:

Init is the first process started during boot. It never exits. It (re)spawns children according to /etc/inittab.
Signals:

HUP: reload /etc/inittab</li>
TSTP: stop respawning until CONT</li>
QUIT: re-exec another init</li>
USR1/TER, /USR2/INT: run halt/reboot/poweroff/Ctrl-Alt-Del script</li> </ul> </blockquote>
I knew Alpine heavily leaned on BusyBox, but didn’t realize it was to this extent. So, following this trail, there was Alpine’s /etc/inittab</code>, where I found this:
::sysinit:/sbin/openrc sysinit ::sysinit:/sbin/openrc boot ::wait:/sbin/openrc default </code></pre> Meaning, Alpine’s init is BusyBox’s init, which in turn calls on OpenRC to bring services up. Working around musl…</h2> Willing to give OpenRC a try, I had to clear up my only concern with Alpine: lack of compatibility with glibc software. This took a long time to evaluate, because you need to actually try to do what you usually do in real situations to see if things will work out or not. If you are questioning whether Alpine can reliably run a full desktop environment or window manager, play and record audio, share your screen, run GUI applications including the latest browsers, act as the host of virtualization and development environments, the answer is an unwavering yes. If you are questioning whether on top of that Alpine can play DRM video and games (e.g., Netflix and Steam), the answer is also yes, but you’ll need a glibc layer around it. If you like Flatpak, that might actually be trivial. But for more involved usage of software that requires glibc, in particular novel and experimental stuff you are compiling yourself or exploring and prototyping with, your mileage may vary. It is more likely that a vendor will not provide a musl build of their software. Some stuff works just by installing gcompat</code>, a compatibility layer that will make glibc software run as-is. Some however won’t work with it. Sometimes I’d just compile something myself, meaning I made my own musl-linked executable of something that was available only for glibc. That works, but compilation for musl is often less documented and may not be possible at all depending on whether or not musl provides all the needed features. The latter case was however quite rare in my experience. Some other options are running things through Flatpak, or Nix, or Distrobox. But sometimes, if all failed, I just had to accept it wasn’t possible. Or it would work and then crash when using a specific feature at runtime. And that’s great, having ways to circumvent musl. But you could also just… run stuff directly? It’s nice to have options, but it’s also a lot of friction. And sadly that is what ended up exhausting me from using Alpine as a daily driver. I think that, if you have a very consistent usage of Alpine, where you are mostly doing the same thing and using the same tools, you could find a comfy workflow there. But I am extremely moved by experimentation and curiosity. And something tells me that I could devise a more fluid workflow to run my experiments and try different tools inside isolated glibc-friendly environments, but in the end it’s still friction. And it’s hard to shake the feeling that, if you were running a more compatible system to begin with, that wouldn’t even be something to think about. Getting software from the distribution’s package manager is the way to go for numerous reasons. But sometimes that is just not an option. Sometimes the package is just nonexistent in the repositories. Sometimes you need to compile it yourself because you need a specific version. Or because you don’t trust the provided pre-compiled binary. Sometimes you need a specific compilation with a specific feature. Sometimes you are in a hurry, or just curious to try something you can securely run in isolation. So not being able to install it the conventional way is friction in itself. Having to compile something every time is extra friction. Having to recompile on upgrades is more friction. And to not be able to compile or run it even by compiling it yourself, that is just compounded friction with some frustration sprinkled on top.
… though I’d rather not</h2> This is not about Alpine, but it’s also inseparable from Alpine. Alpine has an excellent, fast package manager that never broke my setup. When things did break, it was able to fix them by itself. The six months release cycle is to me a really sweet spot at which to space versions — not too often, not too seldom. A lot of software is available. Developers are security conscious. It’s extremely lean on resources.And in the spirit of being so lightweight, Alpine shines on virtualized and embedded systems. This focus on being small is why musl is a great fit, since it’s a lot lighter than glibc, among other advantages</a>. For me though, while I appreciate the smaller size, it’s not that critical that I’d sacrifice compatibility and versatility over it. So I have zero complaints about Alpine, musl aside. It is perfect in everything I signed up for when I decided to try it. It delivered on every feature and never lied to me. It never said to me it was going to be smooth and easy to run glibc programs. So it’s not so much that I didn’t like Alpine, it’s that I wish there were a glibc Alpine just like there is a glibc Void. But I’m well aware that’s not happening. Choosing a path beyond</h2> It was excellent to learn hands-on about how Alpine works and what it can offer. I learned a lot about not just its components, such as OpenRC and BusyBox, which were basically strangers to me, known only conceptually. Alpine also taught me a lot about all the things it doesn’t have. Because, seriously, if you do a chroot installation of Alpine and don’t even run its setup scripts, you’ll end up with something so trimmed down you’ll wonder if you’re midway through LFS</a>. You’ll have to work your way up to a usable system piece by piece, finding out what you need, why you need it and how to set it up. And that is a huge learning opportunity for those interested For now though, I decided to backtrack a little bit and then fully reverse course if I can’t find a working compromise. That means giving Void another try and, if stability keeps being a problem, go further back and land on Debian. If I shed the non-systemd preference aside, Debian would deliver on the stability and compatibility that I am looking for. You see, I am not looking for the next distribution to try as if it were a hobby and I wanted to try novelty out of FOMO. This is actually a lot of work! I’d rather do other things with the operating system rather than install and configure it, as I mentioned before</a>. And if at the other end of the bell curve it’s Debian, whatever. Perfect is the enemy of good and being picky is exhausting. I mentioned curiosity is a big factor in wanting to try things too. And frankly, after a while holding this preference for non-systemd operating systems and using systems without it, I might as well seize the opportunity to learn more about systemd as well. At my current internship, systemd is what all the distros we have in production are using. In future work, that is very likely to be the case too. So there is nothing to lose. I really wish, though, that package managers were more capable of differentiating between security and feature upgrades. If they were, we could run a rolling distro in “Debian mode” at will. This thought may not sit well with some rolling distributions, say, in Arch Linux you have the notion that “Partial upgrades are unsupported”</a>. There may be the expectation that the whole repository at any given time is the end state for all machines subscribed to it, but if dependencies could also be at the version and not only package level, that could also be solved, though I won’t undersell the consequences and added complexity in such a change. Just like mail clients, all operating systems suck. Some just suck less for some specific use case. Also, if you find out how to reclaim that identity, do share. Asking for a friend. ↩</a> </li> It’s no surprise a distribution’s main init system has better support, as it sure is a lot of work for packagers to support and test multiple init systems. ↩</a> </li> </ol>

Void on ZFS

2024-06-09T00:00:00+00:00

June is here. It brings the usual cold weather and some extra rhinitis complications. With that I find myself in a recovery mood Sunday, wrapped in a blanket with a mug of tea, a screwdriver, some notes on paper, a flash drive, a couple of NVMe cards and the trio of Unix-powered machines that will help me get this done.

The mission is to get a root-on-ZFS EFI installation of Void Linux with ZFSBootMenu on a Dell Latitude 7480.

To my left, a ceiling-collapse-survivor Sony VAIO is running NetBSD with spectrwm. It’s split with sakura and tmux on a terminal to one side, where Neovim is storing these words, and Firefox on the other, ready to fetch all the docs. In the middle, the object of today’s operation. And to my right, a headless PC board runs FreeBSD with ZFS, holding all the backups needed for the post-install tasks.

This lengthy post, written not after the fact but during it, is my way of documenting and also sharing how it all went. Additionally, it’s a way to delve deeper into many of the things the ZFSBootMenu docs leave unexplained, an urge I already had yesterday as I just tried it out without much modification.

Last night, I ran through the ZFSBootMenu documentation guide for Void</a> and followed it both on a VM and then on an external SATA HDD plugged through a USB case, taking some notes and getting a general idea of the process.

The Void installer does not support ZFS out of the box, so the Void Handbook</a> itself recommends the ZFSBootMenu documentation before its own</a> (a manual chroot installation) when it comes to doing a ZFS-on-root install. This guide from ZFSBootMenu is what we’ll be following throughout this post.

Do note that, while comprehensive, my account is no replacement for the original guide</a>. Although more concise, it contains certain notes not included in this post and covers a larger set of possibilities than I did here. Some of the code blocks you’ll see here are identical to the ones from the guide, but many others are specific to how I did things, so keep that in mind and try things before going with your final installation.

Why Void?</h2>
I don’t really enjoy distro-hopping. I usually will spend a few years on the same OS and only switch for good reason and after some thorough testing. And after some Debian time, I felt interested in trying Void for a few reasons:

rolling, but stable</li>
runit init system</li>
BSD-like rc files</li>
BSD-like handbook documentation</li>
numerous, up to date, but stable packages</li> </ul>
After trying it, some other features made me settle:

fast and feature-packed package manager</li>
very fast startup time (kudos to runit)</li>
first-class support in ZFSBootMenu</li> </ul>
The Void package manager, xbps</a>, has several interesting features. One of my favorites, for a taste, is `xbps-query --cat</code>, which shows the original contents of a given file in a package.`
`For example, xpbps-query --cat /etc/zfsbootmenu/config.yaml zfsbootmenu</code> will show you the original content of the config.yaml</code> file in the zfsbootmenu</code> package. You can use it for very core packages like base-system</code> or runit-void</code> to determine the original version of files shipped by them.`
And why ZFS?</h2> My first contact with ZFS was when using FreeBSD, which provides it as an option in its installer, making it a bit too easy not to try. Having a server on ZFS means all the data it holds can be safeguarded and transferred in robust ways, and mistakes are also easier to recover from. Aside from all the data integrity features and flexibility it brings, the features that interest me the most are the ones for managing snapshots. ZFS snapshots allow you to store the filesystem state at a given point in time, and to compare against, access the content of, and fully revert to this state. After the guide has been followed throughout, an extra section at the end of this post has some snapshot basics.Getting in</h2> So, first things first, open the machine up and swap the NVMe cards. For me, that means getting my 128 GB NVMe stick, which I use basically for tests, and replace it with the 256 GB one which currently has Debian on it. Yes, I get by just fine with that much. While a bit dusty, the machine was overall in good state. The release date for the model is 2017, which for my computing standards is very recent. It has a single NVMe slot, one 16 GB RAM stick and one unused RAM slot. If you look closely, you can notice a dent on the vent tube connecting the cooler to the CPU. Despite this, it very rarely heats up. Next up is to boot up hrmpf</a> in EFI mode. hrmpf is a Void-based rescue system maintained by a Void team member and distributed as a bootable image that can accomplish many things, some of them being a full Void installation, entering a proper chroot, and being ZFS-ready with the needed drivers and tools. Once booted into it, EFI support can be confirmed by filtering the output of dmesg</code>: dmesg | grep -i efivars </code></pre> The output should contain “Registered efivars operations”. Make sure you have an Internet connection at this point. Most of the following steps will run fine without one, but closer to the end, when installing the Void base system, it will all go to waste if we can’t reach a package mirror. Setting up the installation environment</h2> The ZFSBootMenu guide uses some variables in order to avoid mistakes and make the instructions more portable across the different storage types and supported operating systems. /etc/os-release</code></h3> The /etc/os-release</code> file typically contains information on the operating system you are running. In the hrmpf live system, these are its contents: NAME="Void" ID="void" PRETTY_NAME="Void Linux" HOME_URL="https://voidlinux.org/" DOCUMENTATION="https://docs.voidlinux.org/" LOGO="void-logo" ANSI_COLOR="0;38;2;71;128;97" DISTRIB_ID="void </code></pre> For comparison, here is FreeBSD’s os-release</code> file: NAME=FreeBSD VERSION="14.0-RELEASE" VERSION_ID="14.0" ID=freebsd ANSI_COLOR="0;31" PRETTY_NAME="FreeBSD 14.0-RELEASE" CPE_NAME="cpe:/o:freebsd:freebsd:14.0" HOME_URL="https://FreeBSD.org/" BUG_REPORT_URL="https://bugs.FreeBSD.org/" </code></pre> In contrast, NetBSD has no such file. For the purposes of the ZFSBootMenu guide, only the $ID</code> value appears to be used. And because the file already is structured as shell-compatible variable assignments, we just source it: source /etc/os-release export ID </code></pre> hostid</code></h3> Required by ZFS intallations, a host ID is a 32-bit hexadecimal value that, supposedly, will uniquely identify a machine. Considering the number of existing machines and the 32-bit range, you might guess why I say supposedly. If your machine has the hostid</code> utilities, you can see the host ID by simply running hostid</code>. Prior to generation, my hrmpf live system reports 00000000</code>. It can’t provide a real guarantee that it will be unique, so it’s up to you to take care that it is unique among your machines. Read on for why that’s hardly an issue. From the gethostid(3)</code> man page: […] a unique 32-bit identifier for the current machine. The 32-bit identifier was intended to be unique among all UNIX systems in existence. This normally resembles the Internet address for the local machine, as returned by gethostbyname(3)</code>, and thus usually never needs to be set. </blockquote> This seems to be more or less a legacy feature. In Void’s man page for gethostid(3)</code>, you see this in the history section: 4.2BSD; dropped in 4.4BSD. SVr4 and POSIX.1-2001 include gethostid() but not sethostid(). </blockquote> Still, it is something that OpenZFS requires to be set: At time of import or creation, the pool stores the system’s unique host ID and for the purposes of supporting multipath, import into other systems will fail unless forced. — OpenZFS docs, Introduction to ZFS: Storage pools</a> </blockquote> zgenhostid</code>, which is shipped by OpenZFS, according to its man page “emulates the genhostid(1)</code> utility and is provided for use on systems which do not include the utility or do not provide the sethostid(3)</code> function.” When used without arguments, these commands will generate a random host ID. But they can also be passed a hexadecimal value, which gets stored by default in /etc/hostid</code> unless another path is given with -o</code>. Considering this information, it threw me off a bit that the ZFSBootMenu guide tells you to specify an arbitrary host ID rather than generate a random one: zgenhostid -f 0x00bab10c </code></pre> If they must be unique, that seems odd. The value 0x00bab10c</code> actually has significance in the context of OpenZFS as an identifier (and leetspeak) for its uberblock. However, it apparently is totally unrelated to host IDs. Should you be curious still, you can refer to this GitHub discussion</a> where a ZFSBootMenu user brought this exact question to the developers. According to the answer given above, the uniqueness of host IDs is useful for “multipathed SAS enclosures with two discrete head unis attached”, which is an enterprise-grade storage solution. The value 0x00bab10c</code> is indeed unrelated and chosen for easy identification. Any value may be used, but when using the pre-built ZFSBootMenu images it may make the process slightly slower (around 250ms) as ZFSBootMenu will have to “discover the hostid every boot”. Disk variables</h3> Here too, the ZFSBootMenu guide works with a set of variables</a> to make it easier covering different possible storage types: BOOT_DISK</code>, BOOT_PART</code> and BOOT_DEVICE</code></li> POOL_DISK</code>, POOL_PART</code> and POOL_DEVICE</code></li> </ul> My target device is an NVMe at nvme0n1</code>, so I’ll have: BOOT_DISK="/dev/nvme0n1"</code></li> BOOT_PART="1"</code></li> BOOT_DEVICE="${BOOT_DISK}p${BOOT_PART}"</code> which evaluates to /dev/nvme0n1p1</code> </li> </ul> </li> POOL_DISK="/dev/nvme0n1"</code></li> POOL_PART="2"</code></li> POOL_DEVICE="${POOL_DISK}p${POOL_PART}"</code> which evaluates to /dev/nvme0n1p2</code></li> </ul> </li> </ul> While this may seem silly at first, it allows using the values separately in the next steps. It also makes the docs a lot more concise while covering several possible disk setups. Confirming the environment setup</h3> At this point, we should be able to print something like this in our environment: # env | grep ID ID=void # hostid 00bab10c # echo $BOOT_DEVICE /dev/nvme0n1p1 # echo $POOL_DEVICE /dev/nvme0n1p2 </code></pre> Take care to keep this same environment for all the next steps as they depend on it. For instance, the hrmpf live system ships tmux. While that is great and I have used it throughout, you must be careful to use a single pane for all the actual steps, and the other panes just for secondary things like looking up man pages or checking file contents. Filesystem setup</h2> Wiping</h3> The first step is to clear the current ZFS label information from the device: zpool labelclear -f "$POOL_DISK" </code></pre> The -f</code> option will “treat exported or foreign devices as inactive”, per the man page. This step fails consistenly for me, which I assume is because the previous filesystem was not ZFS to begin with. Next, we will use wipefs</code> to erase the current filesystem signature. This command is not ZFS-specific, but part of the kernel utilities. It does not erase the filesystems themselves, nor their content, but the signatures that aid in their detection. Without any options, it will list all the filesystems that are still visible: # wipefs "$BOOT_DISK" DEVICE OFFSET TYPE UUID LABEL nvme0n1 0x200 gpt nvme0n1 0x3d9e655e00 gpt nvme0n1 0x1fe PMBR </code></pre> The -a</code> option is for erasing all signatures. This means it will “scan the device again after each modification until no magic string [signature] is found”, as per its man page. In my case: wipefs -a "$POOL_DISK" wipefs -a "$BOOT_DISK" </code></pre> Along the guide, commands are sometimes repeated for both $POOL_DISK</code> and $BOOT_DISK</code>. If you are using the same disk for both, this may be redundant, although also harmless. This is my case, so I am not typically running it twice. I’ll still leave it as is however, so as not to mislead the reader. Now, when listing the signatures again with wipefs "$BOOT_DISK"</code>, there should be no output. Finally, the current MBR and GPT tables must be destroyed. For this, the ZFSBootMenu guide uses sgdisk</code>. This is also not ZFS-specific. sgdisk --zap-all "$POOL_DISK" sgdisk --zap-all "$BOOT_DISK" </code></pre> The --zap-all</code> option contrasts with --zap</code> in that it will destroy both MBR and GPT partition tables. Partitioning</h3> In the ZFSBootMenu guide, sgdisk</code> is used again for creating the partitions: sgdisk \ -n "${BOOT_PART}:1m:+512m" \ -t "${BOOT_PART}:ef00" "$BOOT_DISK" sgdisk \ -n "${POOL_PART}:0:-10m" \ -t "${POOL_PART}:bf00" "$POOL_DISK" </code></pre> In the commands above, option -n</code> is short for --new</code>, and is specifying the start and end sectors by using relative kibibyte measures. The format is --new partnum:start:end</code>. Breaking it down: 1m</code> 1 mebibyte from the start of the disk</li> +512m</code> 512 mebibytes after the default start sector</li> -10m</code> 10 mebibytes before the last available sector</li> 0</code> the default value</li> </ul> In the list above, “default” is “the start of the largest available block for the start sector and the end of the same block for the end sector”, as per the sgdisk</code> man page. 1:1m:+512m</code>, therefore, means that partition 1 will start 1 mebibyte from the start of the disk and end 512 mebibytes after the start of the largest available block. 2:0:-10m</code>, in turn, means partition 2 will begin at the start of the largest available block and end 10 mebibytes before the last available sector. Option -t</code> is for setting the typecode for each partition. Typecode ef00</code> is for the EFI system partition, and typecode bf00</code> is for “Solaris root”, the Unix system upon whose ZFS implementation OpenZFS was based. For a list of typecodes, see sgdisk -L</code>. While just running these commands as-is is your safest option, you might have a different layout in mind or prefer an interactive UI. For one thing, I’ve had issues in the past with the boot partition being too small, so I’ll be using 2g</code> instead of 512m</code> for it. sgdisk</code> has a friendlier counterpart named gdisk</code>, which you can use just by passing it the disk path, as in gdisk /dev/sda</code>. At this point, you should be safe to try partitioning and going back to wiping as needed until you are satisfied. When you are done, you can use lsblk</code> to confirm the results. The following will show you the options just configured: lsblk -o NAME,SIZE,TYPE,PARTTYPENAME </code></pre> Creating the pool</h3> This part of the guide was the one that actually made me want to delve deeper and understand what each option meant. With little knowledge about ZFS still, I wanted to understand precisely what was happening here, but also what a pool even is and what its creation meant. Here’s the zpool(8)</code> man page: A storage pool is a collection of devices that provides physical storage and data replication for ZFS datasets. All datasets within a storage pool share the same space. </blockquote> The definition of a dataset is then indicated to be at zfs(8)</code>: A dataset is identified by a unique path within the ZFS namespace: pool[/component]/component</code> for example: rpool/var/log</code> </blockquote> Here, it’s also explained that a dataset can be a file system, logical volume, snapshot or bookmark. Further information is also hinted to be found at zpoolconcepts(7)</code>. At this point you start to notice the breadth of knowledge available in the documentation. The man pages are not only comprehensible, but sometimes contain several examples on how to apply their concepts. Each command has their own man page named with a hyphen for separation, as in zpool-create</code>. We’ll be exploring only the zpool-create(8)</code> command in depth, in particular the options used in the ZFSBootMenu guide: -f</code> force the use of virtual devices, even if they appear in use</li> -o feature=value</code> set a pool feature</li> -O property=value</code> set a file system property in the root file system of the pool</li> -o compatibility=off|legacy|file[,file]</code> specify a compatibility feature set</li> -m mountpoint</code> the mountpoint (default: /pool</code>)</li> pool</code> the pool</li> vdev</code> the virtual device</li> </ul> The listing with pool features (including compatibility feature sets) is at zpool-features(7)</code>. Pool properties are at zpoolprops(7)</code> and file system properties at zfsprops(7)</code>. In the guide, these are the options given: zpool create -f \ -o ashift=12 \ -O compression=lz4 \ -O acltype=posixacl \ -O xattr=sa \ -O relatime=on \ -o autotrim=on \ -o compatibility=openzfs-2.1-linux \ -m none zroot "$POOL_DEVICE" </code></pre> Among the options above, several pool features and system properties are set: -o ashift=12</code> “Alignment shift”, used to calculate physical sector sizes.This is discussed at greater length in the online documentation on Workload Tuning</a></li> -O compression=lz4</code> Sets the compression algorithm used (LZ4</a>)</li> -O acltype=posixacl</code> Whether ACLs</a> are enabled and what type to use. The value posixacl</code> is equivalent to posix</code> (default on Linux: off)</li> -O xattr=sa</code> Enables extended attributes. If value is on</code>, uses directory-based extended attributes, while sa</code> uses system-attribute-based. The latter has performance benefits, and is important for ACLs and SELinux usage</li> -O relatime=on</code> “Causes the access time to be updated relative to the modify or change time.” Also, “access time is only updated if the previous access time was earlier than the current modify or change time or if the existing access time hasn’t been updated within the past 24hours”</li> -o autotrim=on</code> Automatically reclaims unused blocks from time to time. Can put the filesystem under some stress</li> </ul> The last option, the compatibility feature set, specifies in this case a relative filename to /usr/share/zfs/compatibility.d</code>: -o compatibility=openzfs-2.1-linux</code></li> </ul> zpool-create(8)</code> also states: By default, all supported features are enabled on the new pool. The -d</code> option and the -o</code> compatibility property […] can be used to restrict the features that are enabled, so that the pool can be imported on other releases of ZFS. </blockquote> The compatibility option openzfs-2.1-linux</code> is described as a “conservative” choice in the ZFSBootMenu guide and in my tests had little impact, so I decided to not use it for this installation. Creating the filesystems</h3> Once the pool is ready, the filesystems can be created. For this task, the zfs</code> command is used with create</code>: zfs create -o mountpoint=none zroot/ROOT zfs create -o mountpoint=/ -o canmount=noauto zroot/ROOT/${ID} zfs create -o mountpoint=/home zroot/home </code></pre> The ZFSBootMenu guide explains at this point that if canmount=noauto</code> is not set on file systems with the /</code> mountpoint, the OS will try to mount them all and fail. It goes on to say: Automatic mounting of / is not required because the root file system is explicitly mounted in the boot process. </blockquote> After the filesystems have been created, the boot file system must be set. zpool set bootfs=zroot/ROOT/${ID} zroot </code></pre> Essentially, this is saying “set zroot</code>’s bootfs</code> property to zroot/ROOT/void</code>” Export, reimport and mount</h3> The next steps consist in exporting and then importing the pool with a given mountpoint. zpool export zroot zpool import -N -R /mnt zroot </code></pre> From what I gather, exporting means putting the pool in a more portable state. According to the zpool-export(8)</code> man page, “the devices [marked as exported] can be moved between systems […] and imported as long as a sufficient number of devices are present.” If zfs import</code> is used without any arguments, it will list the exported pools available to be imported. The -N root</code> option imports the pool without mounting any of its file systems, the -R</code> option “sets the cachefile</code> property to none</code> and the altroot</code> property to root</code>”. In this case, that root</code> will be /mnt</code>. altroot</code> stands for the alternate root directory. In zpoolprops(7)</code>, this becomes clearer when it is stated that “this directory is prepended to any mount points within the pool.” Once re-imported, we can mount: zfs mount zroot/ROOT/${ID} zfs mount zroot/home </code></pre> And verify that all is mounted correctly with mount | grep mnt</code>: # mount | grep mnt zroot/ROOT/void on /mnt type zfs (rw,relatime,xattr,posixacl,casesensitive) zroot/home on /mnt/home type zfs (rw,relatime,xattr,posixacl,casesensitive) </code></pre> Lastly, we request the device events from the kernel to update the device symlinks: udevadm trigger </code></pre> Setting up Void</h2> Installation</h3> So far, not much here was Void-specific. This is when we start bootstrapping the void system into the filesystem we laid out. XBPS_ARCH=x86_64 xbps-install \ -S -R https://repo-fastly.voidlinux.org/current \ -r /mnt base-system </code></pre> Here, we are passing an environment variable to set the architecture to x86_64</code>, then use xbps-install</code> from the xbps package manager to fetch the Void base system. -S</code> takes care of synchronizing the data from the mirror so that package data is fetched, -R</code> allows us to manually specify the repository for this run, and -r</code> allows choosing a different root directory to act upon. Here, I chose the Fastly mirror over the ServerCentral one. Any working mirror</a> should do. Note that not all mirrors have the same content at the root of their URL. Some point directly to a Void repo, some don’t. You can access the mirror in a browser or otherwise inspect it to find the path to the current</code> directory. With this done, we can copy the host ID file, which will also be required in our final system, and we are ready to chroot. cp /etc/hostid /mnt/etc </code></pre> chrooting</h3> We will chroot into the system mounted at the /mnt</code> directory using xchroot</code>, which is part of the xbps xtools</code> package and should already be available on hrmpf. It provides a more sane</a> chroot than the plain one, in particular regarding the required mountpoints: xchroot /mnt </code></pre> This is a good time to get back to the notes I mentioned taking the day before. Reconfiguring packages</h4> After chrooting, it may be a good idea to run xbps-reconfigure</code> to make sure packages are properly configured. This is because in the bootstrap process some packets may have tried to configure themselves while relying on directories that were not mounted anywhere. This is particularly true for dracut</code></a>, which is a tool that generates initramfs and initrd images, therefore being critical to the early boot process. You might see error messages related to it in your first run of xbps outside of the chroot, when installing the base system. To reconfigure all packages, just run xbps-reconfigure -fa</code>. If you’d rather only reconfigure dracut</code>, go with xpbs-reconfigure -f dracut</code>. root password</h4> As early as possible is a good time to run passwd</code> and set the root password. rc.conf</code></h4> runit</code> reads the /etc/rc.conf</code> file during startup to configure the system, setting up things like the keymap, hardware clock and terminal font. For your reference, here is what I added to mine during the installation: HARDWARECLOCK="UTC" KEYMAP="br-abnt2" FONT="ter-120n" </code></pre> Time zone and locale</h4> To configure your local time zone, create a symlink at /etc/localtime</code> that points to the corresponding time zone in the /usr/share/zoneinfo</code> directory. ln -sf /usr/share/zoneinfo/<timezone> /etc/localtime </code></pre> Unless you are using musl</code>, you also want to set and generate the glibc</code> locales. Edit /etc/default/libc-locales</code> and uncomment the desired locales, then run xbps-reconfigure -f glibc-locales</code>. dracut</h4> dracut</code> generates file system images used by the kernel at the very early stages of boot. We have to make it able to identify our ZFS root filesystem by enabling the proper modules. This is accomplished by creating /etc/dracut.conf.d/zol.conf</code> with: nofsck="yes" add_dracutmodules+=" zfs " omit_dracutmodules+=" btrfs " </code></pre> Notice the spaces surrounding the module names. Installing and configuring ZFSBootMenu</h2> We are now ready to install both ZFS and ZFSBootMenu. Let’s start with ZFS: xbps-install -R https://repo-fastly.voidlinux.org/current zfs </code></pre> Now, before installing ZFSBootMenu, we set the kernel commandline. This is the command line that will be used by the Linux kernel, so any options you are used to go here. The ZFSBootMenu guide has only the quiet</code> option. In my case, I added net.ifnames=0</code> to have the classic eth0</code>, wlan0</code> network interface names, and fbcon=nodefer video=efifb:nobgrt</code>, which prevents the manufacturer’s logo from showing after boot and sometimes obscuring the boot process output. zfs set org.zfsbootmenu:commandline="quiet net.ifnames=0 fbcon=nodefer video=efifb:nobgrt" zroot/ROOT </code></pre> We also need a vfat</code> filesystem on our boot device: mkfs.vfat -F32 "$BOOT_DEVICE" </code></pre> Now we can add an /etc/fstab</code> entry pointing to it, and mount: echo "$(blkid | grep "$BOOT_DEVICE" | cut -d ' ' -f 2) /boot/efi vfat defaults 0 0" >> /etc/fstab mkdir -p /boot/efi mount /boot/efi </code></pre> Into this directory we just mounted, we can now install ZFSBootMenu. The guide provides two different paths here: a prebuilt image or the Void package, which you can get through xbps. While there are advantages to both, I decided to go with the prebuilt image since I’d rather the package manager not touch the boot manager on updating. This has the downside of you having to take care of being aware of any relevant versions and when to upgrade to them. xbps-install curl mkdir -p /boot/efi/EFI/ZBM curl -o /boot/efi/EFI/ZBM/VMLINUZ.EFI -L https://get.zfsbootmenu.org/efi cp /boot/efi/EFI/ZBM/VMLINUZ.EFI /boot/efi/EFI/ZBM/VMLINUZ-BACKUP.EFI </code></pre> If you’d rather use the repository package, see the corresponding instructions in the guide</a>. Finally, a second choice has to be made between rEFind</code> or plain efibootmgr</code> for managing the boot entries. I prefer to go with the simpler one, but you may find rEFind</code> more feature-packed. First, install efibootmgr</code> using xbps-install efibootmgr</code>, then run the following commands: efibootmgr -c -d "$BOOT_DISK" -p "$BOOT_PART" \ -L "ZFSBootMenu (Backup)" \ -l '\EFI\ZBM\VMLINUZ-BACKUP.EFI' efibootmgr -c -d "$BOOT_DISK" -p "$BOOT_PART" \ -L "ZFSBootMenu" \ -l '\EFI\ZBM\VMLINUZ.EFI' </code></pre> If you’d prefer to use rEFInd, see the guide’s relevant section</a>. zbm-kcl</code> is mentioned here in passing. This utility allows you, among other things, to set ZFSBootMenu options, such as the delay before automatically booting. I am not sure if it comes included with the ZFSBootMenu package, as I went for the pre-built image, but you can nonetheless get it from GitHub: curl -O https://raw.githubusercontent.com/zbm-dev/zfsbootmenu/master/bin/zbm-kcl chmod +x zbm-kcl </code></pre> Now, if you want to change an option, you can use its -a</code> option to append an argument to the target image’s command line: zbm-kcl -a 'zbm.timeout=2' /boot/efi/EFI/ZBM/VMLINUZ.EFI </code></pre> In the example above, the timeout before automatically booting is set from its 10 seconds default to 2 seconds. Getting out</h2> We are all done. It’s time to exit the chroot, unmount and export the pool. exit umount -n -R /mnt zpool export zroot </code></pre> If all above went well, we can now run reboot</code>, remove the flash drive used for installation, and log in for the first time into our new system. ZFS snapshot basics</h2> Something you might want to do at this point is to take a snapshot of the current state, since it can serve as a baseline before any further tweaking, allowing you to go back or access the files in this state as you make important changes that could potentially break the system. zfs snapshot -r zroot/ROOT/void@baseline </code></pre> Note that, if you followed the ZFSBootMenu guide in creating a separate dataset for your home directory, this snapshot will not include the contents inside and under /home</code> (which at this point should be empty anyways). You can access the contents of a snapshot at any time in the .zfs</code> directory at the root of a given dataset. For the ones we previously set up, those would be /.zfs</code> and /home/.zfs</code>. Note that these directories are not only hidden in the traditional way, but they won’t show up even if you use ls -a</code>. You need to actually cd</code> into the apparently absent directory for it to work. ZFS snapshots start taking virtually no space at all, but grow with time as the snapshot drifts from the present system state. For that reason, keeping a snapshot of the very first moment of your system can take up significant space. Depending on your storage resources, you might eventually decide to destroy this snapshot: zfs destroy -r zroot/ROOT/void@baseline </code></pre> You may also want to list your current snapshots. While typically you can use zfs list -t snap</code>, I tend to use the following command in order to get more relevant output: zfs list -t snap -o creation,name,used,written,referenced,refcompressratio -S creation </code></pre> Finally, you might want to rename a snapshot: zfs rename -r zroot/ROOT/void@baseline @day0 </code></pre> Combined, these commands can get you as far as an automatic, rolling snapshot system. Say, for instance you add the following to rc.local</code>: zfs destroy -r zroot/ROOT/void@fallbackBoot zfs rename -r zroot/ROOT/void@previousBoot @fallbackBoot zfs rename -r zroot/ROOT/void@currentBoot @previousBoot zfs snapshot -r zroot/ROOT/void@currentBoot </code></pre> This would give you a per-boot snapshot trail to rely on. The zfs-snapshot(8)</code> man page provides a similar example for daily snapshots. Considering how simple the zfs CLI is, scripting several snapshot schemes can be quite easy, be them per boot, daily, or even every so many minutes using cron. Because ZFS snapshots grow as they drift from the present state, rotating them is optimal when storage space is a concern. That’s it! I hope this was helpful to you in either learning about ZFS or about Void installations with Root on ZFS. Originally written June 2nd, 2024

Setting up the installation environment</h2>
The ZFSBootMenu guide uses some variables in order to avoid mistakes and make the instructions more portable across the different storage types and supported operating systems.</p>

Filesystem setup</h2>

Setting up Void</h2>

jutty.dev - Posts

Half an year on Alpine: just musl aside

On self-hosting being a patch

[Meta] Notice on RSS feeds

Giving up simplicity

Introducing tori

Void on ZFS

Meeting the BSD family

jutty.dev - Posts

Half an year on Alpine: just musl aside

On self-hosting being a patch

[Meta] Notice on RSS feeds

Giving up simplicity

Introducing tori

Void on ZFS

Setting up the installation environment</h2> The ZFSBootMenu guide uses some variables in order to avoid mistakes and make the instructions more portable across the different storage types and supported operating systems.</p>

Filesystem setup</h2>

Setting up Void</h2>

Time zone and locale</h4> To configure your local time zone, create a symlink at /etc/localtime</code> that points to the corresponding time zone in the /usr/share/zoneinfo</code> directory.</p>

Installing and configuring ZFSBootMenu</h2> We are now ready to install both ZFS and ZFSBootMenu. Let’s start with ZFS:</p>

Meeting the BSD family

Setting up the installation environment</h2>
The ZFSBootMenu guide uses some variables in order to avoid mistakes and make the instructions more portable across the different storage types and supported operating systems.</p>