Setting up Encrypted ZFS on NetBSD
I just set up an encrypted ZFS volume on my NetBSD install. This is my journal on how I did it. I’m using a zpool on top of NetBSD’s cgd Cryptographic Disk layer (similar to LUKS on linux), because NetBSD’s ZFS does not have ZFS’s native encryption available. Most of this post is about setting up cgd.
So before I get into things: I haven’t yet done an install with a ZFS rootfs. I also haven’t done an install with an encrypted rootfs. I certainly haven’t done both together. It is possible, but we’re not covering it here. I’ll put links in the references at the bottom of the page that may help if you want to give that a go, but it gets more complicated because you need to set up a ramdisk.
Also, since it is 2026, I must unfortunately remind you. GPT, in this article, means “GUID Partition Table”, as it has meant since the 90s.
Before we get into setting up the disks, here’s something that will make life easier. Add devpubd=YES to your rc.conf, and turn it on with /etc/rc.d/devpubd start. This will add all your partitions to /dev/wedges so they’re easier to get at.
The NetBSD Guide describes setting up cgd on an MBR-partitioned drive, but mine is GPT-partitioned, and is bigger than 2TB. So, my disklabel is a little funny:
4 partitions:
# size offset fstype [fsize bsize cpg/sgs]
a: 4294967295 0 4.2BSD 0 0 0 # (Cyl. 0 - 4294967295*)
d: 4294967295 0 unused 0 0 # (Cyl. 0 - 4294967295*)
If you’ve seen a normal disklabel you know that ain’t normal. I think it’s just a stub since disklabel doesn’t support 64-bit sector offsets/sizes. You can maybe guess, we aren’t going to be touching disklabel at all.
Instead we’re using the gpt command. To follow along with me, you need a disk with a GPT partition table, and free space for however big you want your zpool to be. Come back when you have that. For me, I only have one disk, so I told NetBSD’s installer to give me a reasonable sized / partition, and leave the rest of the drive empty.
If you’ve got something that looks like this you’re in good shape:
deimos# gpt show wd0
start size index contents
0 1 PMBR
1 1 Pri GPT header
2 32 Pri GPT table
34 2014 Unused
2048 262144 1 GPT part - EFI System
264192 614400000 2 GPT part - NetBSD FFSv1/FFSv2
614664192 16615424 3 GPT part - NetBSD swap
631279616 6870196879 Unused
7501476495 32 Sec GPT table
7501476527 1 Sec GPT header
We want to fill that big chunk of Unused space at the end. We’ll use the cgd type (Cryptographic Disk).
deimos# gpt add -a 4096 -b 631279616 -t cgd -l sepulchre wd0
deimos# gpt show wd0
start size index contents
0 1 PMBR
1 1 Pri GPT header
2 32 Pri GPT table
34 2014 Unused
2048 262144 1 GPT part - EFI System
264192 614400000 2 GPT part - NetBSD FFSv1/FFSv2
614664192 16615424 3 GPT part - NetBSD swap
631279616 6870196872 4 GPT part - NetBSD Cryptographic Disk
7501476488 7 Unused
7501476495 32 Sec GPT table
7501476527 1 Sec GPT header
Now, thanks to devpubd, we’ve got /dev/wedges/sepulchre linking to a /dev/dk wedge device:
deimos# ls -l /dev/wedges/sepulchre
lrwxr-xr-x 1 root wheel 8 Oct 23 06:04 /dev/wedges/sepulchre -> /dev/dk4
Setting up Encryption
If you’re coming from Linux and you’re used to LUKS, things are a bit different here. With LUKS, metadata about the encryption is stored in a header block at the start of the encrypted partition. Not so with NetBSD’s cgd. Instead, that metadata is stored in a normal file on disk.
To make that file, we use cgdconfig -g:
deimos# cgdconfig -g -o /etc/cgd/sepulchre -V gpt -k argon2id aes-xts 512
calibrating argon2id parameters......... done
deimos# cat /etc/cgd/sepulchre
algorithm aes-xts;
iv-method encblkno1;
keylength 512;
verify_method gpt;
keygen argon2id {
iterations 16;
memory 84825;
parallelism 4;
version 19;
salt AAAAgMfSmdb/UER41hcsaA51JWw=;
};
Explaining the parameters:
-V gpt- Since there’s no metadata installed to the partition, cgd needs a way to tell whether a key is correct when you open an encrypted partition, by looking for some expected data.-V gpttells it to look for a GPT header (which is not there yet, but we’ll put it there shortly).-k argon2id- This is the method for deriving the 512-bit key from your passphrase. argon2id is the current gold standard here, but is not the default on NetBSD for whatever reason.aes-xts 512- AES encryption with a 512-bit key (this actually gets split into two 256-bit keys). You could useaes-xts 256here for theoretically somewhat worse security, in exchange for better performance, but I’m not the one to tell you if you need that. If you’re on a device without hardware-accelerated AES, consider usingadiantum 256instead (see my performance comparisons on a raspi3 under linux)
The paramsfile we just generated will be required to decrypt your disk, so back it up now! To quote the NetBSD Guide:
Consider this file being SACRED, BACK IT UP , and BACK IT UP AGAIN!
Now we can open the encrypted partition for the first time. We’re going to do it a special way the first time since there’s no data on there yet, using the re-enter verification method. Please understand: if you EVER use re-enter again, you are asking for your data to be eaten. It’s only safe the first time you set up the partition, since it ignores any data on the disk.
deimos# cgdconfig -V re-enter cgd0 NAME=sepulchre /etc/cgd/sepulchre
After entering a passphrase, you’ll have an encrypted partition open!
If you ever need to manually open this partition later, after your filesystem is set up on it, run that same command but without -V re-enter:
deimos# cgdconfig cgd0 NAME=sepulchre /etc/cgd/sepulchre
But you don’t need to run that right now.
Adding a GPT to the Encrypted Partition
I told you to set up your paramsfile with the gpt verification method, and so we’re going to put a GPT partition table onto this encrypted partition. Yes, you can nest partition tables like this, it’s fairly normal.
In theory this isn’t necessary, because cgd has a zfs verification method. But, when I tried putting a pool directly onto the partition and using the zfs method, cgd could not verify it correctly at all. No idea why, but that’s why we’re going to create a GPT partition table and then add a single partition to it for ZFS:
deimos# gpt create cgd0
deimos# gpt add -a 4096 -l tome -t zfs cgd0
deimos# gpt show cgd0
start size index contents
0 1 PMBR
1 1 Pri GPT header
2 32 Pri GPT table
34 6 Unused
40 6870196792 1 GPT part - ZFS
6870196832 7 Unused
6870196839 32 Sec GPT table
6870196871 1 Sec GPT header
Now we have /dev/wedges/tome
ZFS
Here’s the easy part!
zpool create tome /dev/wedges/tome
Now you have a ZFS dataset at /tome! That’s it!
Mounting at Boot
First, you need /etc/cgd/cgd.conf:
# cgd target [paramsfile]
cgd0 NAME=sepulchre /etc/cgd/sepulchre
Then, make sure your rc.conf has cgd and zfs enabled:
cgd=YES
zfs=YES
Try rebooting and you should be prompted for your disk password.
You don’t need any configuration to automatically mount the ZFS zpool because, after CGD unlocks your disk, ZFS will scan your disks and pick it up on its own. That’s standard ZFS behavior.
Automatically Unlocking the Disk
If this is an unattended server and you want to automatically unlock the disk, well there are many ways, and you will need to get creative depending on how you want to store your key. Here is what I can tell you that will help you out.
First, you can ask cgdconfig to provide the derived key, based on your passphrase and paramsfile:
deimos# cgdconfig -t /etc/cgd/sepulchre
key's passphrase:
ubVdy9uOg4YG9VVSRvrWhbSAO47Cd46NcxTE8Pk2YZ+AashObO1tBygwwRIc1zs1euZMEc9pK8Tt425Yp+zyPQ==
Un-base64 that string and you will have the raw key.
Second, you can use a keygen block like this in a paramsfile, to run an arbitrary command to retrieve your key data:
keygen shell_cmd {
cmd "command passed to /bin/sh"
};
Note that the command should return the binary key data. NOT BASE64.
The third thing to know is that by default, cgd unlocks encrypted partitions after boot, but before any partition other than / is mounted. So you cannot rely on data on other file systems.
Armed with this knowledge, you can devise whatever scheme you want for automatic unlocking. But watch out!
One place to start might be fidocrypt(1) from pkgsrc. The NetBSD wiki has a page that describes using fidocrypt in a shell_cmd to secure a key with fido2.