1 Creating a minimal file system to ease the boot process
2 =======================================================
4 :date: 1970-01-01 00:00
5 :category: System Administration
6 :tags: initramfs, boot, luks, gpg, ssh
7 :authors: Maximilian Friedersdorff
8 :summary: Booting a system with full disk encryption can be tricky.
14 My personal systems use full disk encryption according to the scheme
15 `LVM on LUKS`_. This works great, but obviously requires some way of supplying
16 (or unlocking) the decryption key during the early boot process.
21 Using a passhrase to protect the decryption key is easy, provided the user is
22 sitting in front of the machine. If the machine needs to be started remotely,
23 then it becomes necessary to install a SSH server like dropbear_, or decrypt the
24 LUKS partition in some other way.
29 One can use a key file that either contains the decryption key directly, or just
30 contains a passphrase that unlocks the decryption key. This requires the key to
31 be available during the early boot process.
33 This is tricky, because if it
34 exists in plaintext on the machine it is supposed to unlock, then there is not
35 much point in encrypting in the first place. If instead the key is on a
36 different machine, the key has to be transmitted accross the network, and the
37 other machine has to be on and running. Alternatively the key might be stored
38 on a USB device and carried on one's person at all times, but USB devices can be
39 unreliable. This is also sensitive to loss.
44 I use a Yubikey (a USB SmartCard_) to store my GPG key. This enables a
45 convenient compromise.
47 The solution I propose is to store a GPG encrypted key file on the encrypted
48 machine. When starting the machine remotely, the key can be transmitted to the
49 remote client, decrypted on the SmartCard_ and transmitted back to the encrypted
50 machine. Transmission happens over SSH authenticated with a GPG authentication
51 key stored on the same SmartCard_.
53 When starting the machine locally, the decryption takes place directly on the
56 This has an additional benefit. The SmartCard_ is protected by a passphrase
57 forcing the use of two different factors to decrypt the drive.
62 To get all of this to work, several pieces of software need to be available
63 during the early boot process:
65 * GnuPG, preferably V2.1 or above
66 * pcscd, the PC/SC Smart Card Daemon
68 * LVM, Logical Volume Manager
69 * SSH server, for instance dropbear_
71 Some of these can be built with static linking. This makes their inclusion in
72 an initramfs relatively straightforward. However neither pcscd nor recent
73 version of GnuPG can be built this way. They must be dynamically linked to
74 at least glibc. Including dynamic executable in an initramfs is substantially
75 more tedious as all of their library dependencies have to be included also.
80 Building an initramfs that can do these things is tedious. Existing automated
81 tools that build an initramfs would require substantial additions to make this
82 possible. Doing it manually or building an automated tool from scratch is lots
83 of work. An initramfs like this will also be fairly large (since it contains
84 lots of libraries) and has to be loaded in it's entirety at boot time.
86 Instead I will build a minimal, self hosted root file system. This way I can
87 use a proper package manager to manage the file system, I don't have to worry
88 too much about it's overall size (it doesn't all have to be read at boot
89 time) and I have easy access to any package included in the distribution I base
90 this on. I will call this an initfs.
92 I use gentoo on all of my personal systems, so I will base the initfs on gentoo
93 also. An initramfs is typically built from source anyway, so this makes sense.
94 This is also an opportunity to play with alternative libc's like uClib-ng_ or
95 musl_ which gentoo has some support for.
104 I start by building a very minimal Gentoo system. Initially I follow the amd64
105 handbook_. All of this is done on a fully installed gentoo system. As such the
106 network is already set up, the bootloader is installed, the kernel is configured
107 and compiled and the disk is already partially partitioned.
109 First I add an additional small partition that is *outside* the encrypted LUKS
110 volume, formatting it to ext4 and mounting it somewhere convenient::
112 $ sudo fdisk /dev/sda
115 Welcome to fdisk (util-linux 2.33.1).
116 Changes will remain in memory only, until you decide to write them.
117 Be careful before using the write command.
120 Command (m for help): n
121 Partition number (2-4,7-128, default 2): 2
122 First sector (534528-500118158, default 534528):
123 Last sector, +/-sectors or +/-size{K,M,G,T,P} (534528-83509247, default 83509247): +1G
125 Created a new partition 2 of type 'Linux filesystem' and of size 1 GiB.
127 Command (m for help): p
128 Disk /dev/sda: 238.5 GiB, 256060514304 bytes, 500118192 sectors
129 Disk model: SanDisk SD8SNAT2
130 Units: sectors of 1 * 512 = 512 bytes
131 Sector size (logical/physical): 512 bytes / 512 bytes
132 I/O size (minimum/optimal): 512 bytes / 512 bytes
134 Disk identifier: 699BC6B3-E694-4D9B-AE65-2C7101253C99
136 Device Start End Sectors Size Type
137 /dev/sda1 2048 534527 532480 260M EFI System
138 /dev/sda2 534528 2631679 2097152 1G Linux filesystem
139 /dev/sda5 83509248 84557823 1048576 512M Linux filesystem
140 /dev/sda6 84557824 500117503 415559680 198.2G Linux filesystem
142 Command (m for help): w
143 The partition table has been altered.
146 $ sudo mkfs.ext4 /dev/sda2
147 mke2fs 1.45.0 (6-Mar-2019)
148 Discarding device blocks: done
149 Creating filesystem with 262144 4k blocks and 65536 inodes
150 Filesystem UUID: 97904c89-51e3-4a6b-b3f3-24bb5f083f60
151 Superblock backups stored on blocks:
152 32768, 98304, 163840, 229376
154 Allocating group tables: done
155 Writing inode tables: done
156 Creating journal (8192 blocks): done
157 Writing superblocks and filesystem accounting information: done
159 $ sudo mount /dev/sda2 /mnt/initfs
161 I then move on to installing a stage3 tarball. The main system will be in a
162 chroot anyway, so a No-multilib (pure 64-bit) system is appropriate here and will
163 save some small amount of space. For a first attempt I choose the uclibc stage3
164 tarball. I download the file tarball and it's signed DIGESTS file, verify the
165 DIGESTS file and finally check the hash::
168 # wget http://distfiles.gentoo.org/releases/amd64/autobuilds/20190405/uclibc/stage3-amd64-uclibc-vanilla-20190405.tar.bz2
172 # wget http://distfiles.gentoo.org/releases/amd64/autobuilds/current-stage3-amd64-uclibc-vanilla/stage3-amd64-uclibc-v
173 anilla-20190405.tar.bz2.DIGESTS.asc
177 # gpg --verify stage3-amd64-uclibc-vanilla-20190405.tar.bz2.DIGESTS.asc
178 gpg: Signature made Sat 06 Apr 2019 07:13:22 BST
179 gpg: using RSA key 534E4209AB49EEE1C19D96162C44695DB9F6043D
180 gpg: Good signature from "Gentoo Linux Release Engineering (Automated Weekly Release Key) <releng@gentoo.org>" [unknown]
181 gpg: WARNING: This key is not certified with a trusted signature!
182 gpg: There is no indication that the signature belongs to the owner.
183 Primary key fingerprint: 13EB BDBE DE7A 1277 5DFD B1BA BB57 2E0E 2D18 2910
184 Subkey fingerprint: 534E 4209 AB49 EEE1 C19D 9616 2C44 695D B9F6 043D
185 # sha512sum -c ./stage3-amd64-uclibc-vanilla-20190405.tar.bz2.DIGESTS.asc
186 stage3-amd64-uclibc-vanilla-20190405.tar.bz2: OK
187 stage3-amd64-uclibc-vanilla-20190405.tar.bz2: FAILED
188 sha512sum: stage3-amd64-uclibc-vanilla-20190405.tar.bz2.CONTENTS: No such file or directory
189 stage3-amd64-uclibc-vanilla-20190405.tar.bz2.CONTENTS: FAILED open or read
190 sha512sum: stage3-amd64-uclibc-vanilla-20190405.tar.bz2.CONTENTS: No such file or directory
191 stage3-amd64-uclibc-vanilla-20190405.tar.bz2.CONTENTS: FAILED open or read
192 sha512sum: WARNING: 14 lines are improperly formatted
193 sha512sum: WARNING: 2 listed files could not be read
194 sha512sum: WARNING: 1 computed checksum did NOT match
196 Some things to note: I verified the fingerprint of the "Gentoo Linux Release
197 Engineering" GPG key from the official `gentoo website`_, accessed via https.
198 I can be reasonably confident that that is proper. The error messages in the
199 output of sha512sum exist because there are some other files listed in the
200 DIGESTS, because for each file there is also a WHIRLPOOL hash and there is an
201 inline GPG signature in the file.
203 Extracting the tarball::
205 # tar xpf stage3-*.tar.bz2 --xattrs-include='*.*' --numeric-owner
207 And editing the ``/etc/portage/make.conf`` to have sensible values::
209 # cat etc/portage/make.conf
211 The portage tree is definitely *not* required during boot and is quite large. I
212 just mount it from the host system and chroot::
214 # mount --types proc /proc proc
215 # mount --rbind /sys sys
216 # mount --make-rslave sys
217 # mount --rbind /dev dev
218 # mount --make-rslave dev
219 # mount --rbind /usr/portage usr/
221 # mount --rbind /usr/portage usr/portage
222 # mount --make-rslave usr/portage
223 # chroot /mnt/initfs /bin/bash
224 # source /etc/profile
225 # export PS1="(chroot) ${PS1}"k
227 Because I am building a *minimal* gentoo system, and because I will be writing
228 my own init script anyway, I can remove the installed init system::
230 (chroot) # emerge -C openrc virtual/service-manager netifrc
231 (chroot) # echo 'virtual/service-manager' >> /etc/portage/package.mask
232 (chroot) # emerge -av --depclean
234 Replacing ``openssh`` with ``dropbear``::
236 (chroot) # emerge -C openssh
237 (chroot) # emerge -av dropbear
239 Updating the whole system::
241 (chroot) # emerge -avuDN --with-bdeps y world
243 And finally installing the software required for booting the encrypted drive::
245 (chroot) # emerge -av cryptsetup lvm2 pcsc-lite gnupg ccid
247 Unfortunately ``pcsc-lite`` requires boost, which does inflate the size of the
248 whole thing a fair bit.
253 .. _`LVM on LUKS`: https://wiki.archlinux.org/index.php/Dm-crypt/Encrypting_an_Entire_System#LVM_on_LUKS
254 .. _dropbear: https://matt.ucc.asn.au/dropbear/dropbear.html
255 .. _SmartCard: https://wiki.gnupg.org/SmartCard
256 .. _UClib-ng: https://uclibc-ng.org/
257 .. _musl: https://www.musl-libc.org/
258 .. _handbook: https://wiki.gentoo.org/wiki/Handbook:AMD64
259 .. _gentoo website: https://www.gentoo.org/downloads/signatures/