After being bit by FreeBSD loader half a decade ago, I started preferring GNU GRUB as my bootloader for booting all the operating systems, I ever wrote or decided to try out. Recently I started using ZFS extensively, initially for just my /home
, and later for /
(root). It is great except for one issue that I needed a UFS2
as /
volume for booting. Not a big deal, just created a tiny UFS2
partition and got started…
This worked great for me until yesterday, when due to abnormal power cut which my box experienced, and caused my UFS2
/
to literally wipe off after I fsck
ed it :(. Fortunately there wasn’t much stuff on it, except /etc
, /boot
, /lib{,exec}
, and /{s,}bin
just enough to boostrap my FreeBSD box and mount my ZFS root which has /usr
, /var
, /tmp
etc. volumes. I was able to reinstall my root, and got back to old setup in just an hour. But to avoid this for future, I decided to give a shot to ZFS-only setup again. The normal way to have a ZFS only system requires having gptzfsboot
(for GPT drives, ofcourse) as boot loader, zfsloader
as loader (which does some magic so you can set vfs.root.mountfrom
as a ZFS volume without getting any root device not found error), but since I use GRUB2, I can’t use gptzfsboot
, but I can use zfsloader
although that requires a bit of configuration, like creating a loader.conf(5)
etc. Anyways, I decided to go zfsloader
way, and started /boot/zfsloader
from GRUB2 command-line:
grub> kfreebsd (hd0,7)/boot/zfsloader
This presented me with a loader menu, so instead of continuing with loader, I came back to loader prompt. Then I did lsdev
and it displayed ZFS volumes. And then I did lsmod
to verify if it added zfs.ko
, which it didn’t, but I found something else there. A module entry for /boot/zfs/zpool.cache
, which revealed the whole magic zfsloader
was doing.
So I rebooted and decided to replicate this with GRUB2 taking inspiration from a similar situation I was in few months ago when I was trying to boot dragonflybsd kernel to boot from encrypted HAMMER
volume, and needed an initrd
to mount root from. Following is the final GRUB2 configuration which worked for me, with changes in bold:
menuentry "FreeBSD 9.0-RC3" {
set root=(hd0,7)
kfreebsd /boot/kernel/kernel
kfreebsd_loadenv /boot/device.hints
kfreebsd_module /boot/zfs/zpool.cache type=/boot/zfs/zpool.cache
kfreebsd_module_elf /boot/kernel/opensolaris.ko
kfreebsd_module_elf /boot/kernel/zfs.ko
kfreebsd_module_elf /boot/kernel/coretemp.ko
kfreebsd_module_elf /boot/kernel/geom_journal.ko
kfreebsd_module_elf /boot/kernel/sound.ko
kfreebsd_module_elf /boot/kernel/snd_hda.ko
kfreebsd_module_elf /boot/kernel/uhid.ko
kfreebsd_module_elf /boot/kernel/ukbd.ko
kfreebsd_module_elf /boot/kernel/ums.ko
kfreebsd_module_elf /boot/modules/nvidia.ko
# set kFreeBSD.vfs.root.mountfrom=ufs:ad14p7
set kFreeBSD.vfs.root.mountfrom=zfs:zroot
set kFreeBSD.vfs.zfs.prefetch_disable=1
set kFreeBSD.vfs.zfs.txg.timeout=5
set kFreeBSD.maxvnodes=250000
set kFreeBSD.zfs.txg.write_limit_override=805306368
}
In the above configuration (hd0,7)
is my UFS2 partition which hosts /boot
, which I’ll soon rm -rf
once I upgrade my GRUB to 1.99
(which has support for ZFS), ufs:ad14p7
was my old UFS2
root, and zroot
is the my ZFS volume I’m using as /
.
chateau.d.if!abbe:~ % mount |grep -v zfs
devfs on /dev (devfs, local, multilabel)
devfs on /var/named/dev (devfs, local, multilabel)
/dev/ad14p7 on /boot (ufs, local, read-only)
chateau.d.if!abbe:~ % zfs list
NAME USED AVAIL REFER MOUNTPOINT
home 147G 538G 79.6G /home
home/abbe 65.3G 538G 65.3G /home/abbe
home/tscache 2G 540G 16K -
zroot 18.0G 178G 351M legacy
zroot/tmp 124K 178G 124K /tmp
zroot/usr 15.7G 178G 6.66G /usr
zroot/usr/ports 8.67G 178G 244M /usr/ports
zroot/usr/ports/distfiles 6.55G 178G 6.55G /usr/ports/distfiles
zroot/usr/ports/packages 1.88G 178G 1.88G /usr/ports/packages
zroot/usr/src 351M 178G 351M /usr/src
zroot/var 1.60G 178G 502M /var
zroot/var/crash 31.5K 178G 31.5K /var/crash
zroot/var/db 895M 178G 846M /var/db
zroot/var/db/pkg 49.6M 178G 49.6M /var/db/pkg
zroot/var/empty 31K 178G 31K /var/empty
zroot/var/log 11.2M 178G 11.2M /var/log
zroot/var/mail 54K 178G 54K /var/mail
zroot/var/run 89.5K 178G 89.5K /var/run
zroot/var/squid 231M 11.8G 231M /var/squid
zroot/var/tmp 663K 178G 663K /var/tmp
I hope it doesn’t happen again. Fingers crossed….;)