Like the title says, how do you decrypt a headless server at boot time that has it’s root partition LUKS encrypted.. while not on site? I’ve been running my server for almost two years and I knew this problem would come up eventually so it’s been at the back of my mind for a while. And you know it had to come up at the most inopportune time… 15 minutes before I leave town for a weekend away. After a weekend without access to my network or any of the services I host I finally started working on solving this.
Quick primer on the typical Linux boot process: when you start your computer it loads a minimal OS called initramfs that knows how to decrypt your root and data partitions and start the boot process. initramfs is configured a few different ways, I’m on a RHEL based distro so I used dracut to configure it. Arch based distros use mkinitramfs. Unsure about Ubuntu since I prefer real operating systems ;)
There’s really two routes to take to solving this problem: fully automatic (bad security posture) and manual intervention (good security but inconvenient). The automatic path means writing the keys in plain text in the initramfs and the thought of plain text keys makes me full body cringe. It’s like having a safe with your combination on a Sticky Note next to the dial: why even have a safe? So I knew I needed to find a way to reach my box remotely during the initial boot process.
Luckily, this was fairly easy to implement. I found a very good Github repo https://github.com/gsauthof/dracut-sshd with full instructions. It does have a couple different options so here’s what I did:
I initially tried to go the systemd-networkd path but was not able to get it working. I instead had to add a network module to initramfs:
~$ echo 'add_dracutmodules+=" network "' >> /etc/dracut.conf.d/90-networkd.conf
Next you’ll want to put your public key in /root/.ssh/authorized_keys. You should generally generate a new public/private keypair for each new user/service:
~$ ssh-keygen -t ed25519 && cat ~/.ssh/id_ed25519.pub >> /root/.ssh/authorized_keys
Then it was as simple as adding the 46sshd folder from the dracut-sshd repo to /usr/lib/dracut/modules.d/ and rebuilding initramfs:
~$ git clone https://github.com/gsauthof/dracut-sshd.git ~/dracut-sshd
~$ mv ~/dracut-sshd/46sshd /usr/lib/dracut/modules.d
~$ dracut -v -f
The Permission denied (public key) issue described in the FAQ happened to me. Fixed by:
~$ usermod -p '*' root
That’s it! Restart your server and SSH in, run the Systemd ask password agent:
~$ systemd-tty-ask-password-agent
I hope this helps someone. If I would’ve known it was this easy I would’ve done it two years ago and never worried about how to reach home after the power goes out!