Overview of ZFS Replication Tools

Published: 2021-12-19

After running the server for years, I still do not have a real backup. I have an automatic snapshot regimen using zfs-auto-snapshot (from the zfstools package), but these snapshots are not yet replicated elsewhere. This setup follows the recommendations from “FreeBSD Mastery: ZFS” by Michael W. Lucas and Allan Jude.

This vacation I finally take the time to make it a proper backup¹, and start by looking at what is there.

Requirements

I want to pull snapshots, for security reasons and because the backup server lives behind a NAT.

I would prefer to have separate programs for snapshotting and replication purposes, as I think a program doing both will do them worse than two programs doing only each (i.e. I am a strong believer in the Unix philosophy).

The process should run automated, which means that the pulling system needs to figure out what the latest snapshot is on the server.

Rotation

Only the latest hourly snapshot needs to be fetched, quarterhourly snapshots need not be fetched at all. I do not need a that precise history of what happened, an hour precision is sufficient for me. But I want all daily and monthly snapshots to be fetched.

Existing Solutions

My first instinct was to roll a custom script to download the snapshots, but I quickly realized that figuring out which snapshots to send and which not is not exactly that easy. And before I waste hours of trying to find a whacky solution, I decided to first survey the space of already existing solutions.

sanoid: Singing and dancing do all, end all solution. Seems to contain every imaginable feature, especially taking periodic snapshots. Uses mbuffer(1) if available. Written in Perl. Available via pkg(8).
zxfer: A pure shell solution, originally forked from the original zfs-replicate. Also contains a lot of features, which means its huge (~2kLOC). Uses mbuffer(1) if available. Available via pkg(8).
zfs-replicate: A Python-based re-implementation of the original zfs-replicate. Seems to only handle replication, which is nice. Available via pkg(8).
zrep: Another pretty large shell-based solution. Seems to be pretty old, originally written for Solaris.
zrepl: Go-based solution with lots of features. Uses a daemon, which makes me suspicious. Available via pkg(8).
znapzend: Another pretty big Perl-based solution. Has kind of an annoying documentation “joke” of replacing the s in headlines with z. Probably seen as a cute idea by the developers, but a turn off for me. Available via pkg(8).

Since zxfer(8) is suggested by “FreeBSD Mastery: Advanced ZFS” (also by Michael W. Lucas and Allan Jude), and it does not need funny scripting languages installed², I will try this first.

The success of failure will be reported in the next post.

Although it still won’t satisfy the criteria set by Allan Jude, who wants at least three copies for a proper backup. While he is most probably right for business data, I think a copy of my servers data at home should be sufficient for my private use. ↩︎
I use iocage for jail management, but I am sick and tired of the ever changing Python dependencies it has. This experience made me very wary of these “super-productive” scripting languages, I want my server to use stable and boring tech, because I do not have the time to check on the latest developments all the time. Zfs-auto-snapshot works fine though, as it only depends on the Ruby interpreter. ↩︎