How the system protects itself

This page explains from the beginning how Zainium OS keeps its base safe. You do not need to know how a Linux system works inside to follow it.

The problem

On most Linux systems, the operating system and the software you install are mixed together in the same folders. /usr/bin holds programs from the system and from your packages. Anything running as root can change any of them.

That causes three kinds of trouble:

• Updates can break the system. If the power goes off in the middle of an update, some files are new and some are old, and the machine may not boot.
• Mistakes are permanent. One wrong command as root can delete something the system needs.
• Malware can hide. A program that becomes root can replace login or sudo, and nothing will look wrong.

An immutable system answers this by keeping the core of the system read-only while it runs.

Three layers, three owners

Zainium OS splits the system into layers under /overlayer, and every file has one owner:

There is no /usr, /bin or /lib at the root, so there is no shared place for these files to get mixed. Your own files stay in /home. This layout is an open specification, the Dualayer Hierarchy Standard.

The base never receives a file from the other layers. If you deleted all your installed software, the base would still boot.

One view of two layers

Programs still need to see everything in one place. For that the system uses OverlayFS, a Linux feature that shows two directories as one. The base is the lower layer, your software is the upper layer, and the result appears at /overlayer/syshub. A program in the base and a program you installed both show up in /overlayer/syshub/bin.

This view is built at every boot by quantra-ramfs, before the system starts.

Why plain OverlayFS was not enough

A normal read-only overlay has two problems here.

The first is that root can make it writable again with one command, mount -o remount,rw. That stops accidents, but not a compromised root.

The second is that the kernel does not expect the upper layer to change while the overlay is in use. It remembers what it has looked up, so a newly installed program might not appear, and a directory that is in use can fail with Stale file handle. Other systems avoid this by asking you to reboot.

So we added a small kernel patch with a new mount option, sealed. The kernel page has the technical details. A sealed overlay:

• is read-only for good, and a remount fails, for root too;
• lets zex write straight into the upper layer, and shows new, replaced and removed files at once;
• in the folders that hold programs and libraries (bin, sbin, libexec, lib, drivers and the musl folder), always shows the base's file when there is one. The upper layer can add new files there, but it can not replace or hide a file of the base.

This is why you can install software with no reboot while the base stays out of reach.

fs-verity: the kernel checks the files

The sealed overlay stops writes through the view, but the base is also a set of files on a disk. fs-verity protects those. It is a kernel feature that seals a file with a tree of hashes. After that the kernel refuses to change the file's content, and it checks each part of the file when it is read.

quantra-ramfs makes sure every file of the base and the boot layer is sealed at every boot. The first boot takes a moment for this, and later boots skip what has not changed.

How an update stays safe

Base upgrades never touch the running system:

1. zex downloads the new packages and checks their signatures.
2. It puts them in a staging area that waits for the next boot.
3. At the next boot, before anything uses the base, quantra-ramfs applies them and keeps a journal of every step.
4. If the upgrade fails or the power goes off, the journal is used to undo it. If it can not be undone, the boot stops, so the system never starts on a half upgraded base.

The kernel works in a similar way: the one you were running stays in the boot menu as a fallback.

At boot, one more check

Before the view is built, quantra-ramfs looks through your software layer for anything that sits over a program or library of the base. If it finds something, it moves it to /overlayer/zexlib/shadowed and writes a warning to the boot log. Nothing is deleted, so you can look at what was found.

When something goes wrong

• A bad program: remove it with zex, or undo the change with zex rollback.
• A broken userland: add zainium.overlay=off to the boot line, and the system starts on the base alone. See troubleshooting.
• A bad kernel: choose the fallback entry in the boot menu.

What this does not protect against

No system that gives you root can fully protect itself from root, and we would rather say so:

• /overlayer/syshub/etc and var are writable on purpose, because they hold your users, passwords, network and settings. Root can change them, including how login works.
• Root can still add new programs, and can change data in share, where zex keeps generated files such as the MIME cache.
• Root can use bind mounts to show another file over a path until the next reboot.
• Software you install runs with your own rights, and can read your files. Immutability protects the system, not your documents.

The promise is this: the base stays clean, and you can always get back to it.