Tag Archives: grub2

Booting from LVM over mdraid

Recently while setting up a GNU/Linux host which had /boot over LVM over md RAID, I used following command-line to install GRUB2:

# grub-install --modules="biosdisk part_gpt raid chain part_msdos ext2 linux search help mdraid1x configfile ata normal" --boot-directory=/boot /dev/sda

Following is an excerpt from grub.cfg(5) menuentry block:

set root='(VG-Boot)'
linux /vmlinuz root=/dev/mapper/VG-Root ro  quiet
initrd  /initrd.gz

where VG is the name of volume group, and Boot, and Root are the names of logical volumes containing /boot, and / respectively.

HTH

Booting into ZFS only FreeBSD from GRUB2

After being bit by FreeBSD loader half a decade ago, I started preferring GNU GRUB as my bootloader for booting all the operating systems, I ever wrote or decided to try out. Recently I started using ZFS extensively, initially for just my /home, and later for / (root). It is great except for one issue that I needed a UFS2 as / volume for booting. Not a big deal, just created a tiny UFS2 partition and got started…

This worked great for me until yesterday, when due to abnormal power cut which my box experienced, and caused my UFS2 / to literally wipe off after I fscked it :(. Fortunately there wasn’t much stuff on it, except /etc, /boot, /lib{,exec}, and /{s,}bin just enough to boostrap my FreeBSD box and mount my ZFS root which has /usr, /var, /tmp etc. volumes. I was able to reinstall my root, and got back to old setup in just an hour. But to avoid this for future, I decided to give a shot to ZFS-only setup again. The normal way to have a ZFS only system requires having gptzfsboot (for GPT drives, ofcourse) as boot loader, zfsloader as loader (which does some magic so you can set vfs.root.mountfrom as a ZFS volume without getting any root device not found error), but since I use GRUB2, I can’t use gptzfsboot, but I can use zfsloader although that requires a bit of configuration, like creating a loader.conf(5) etc. Anyways, I decided to go zfsloader way, and started /boot/zfsloader from GRUB2 command-line:

grub> kfreebsd (hd0,7)/boot/zfsloader

This presented me with a loader menu, so instead of continuing with loader, I came back to loader prompt. Then I did lsdev and it displayed ZFS volumes. And then I did lsmod to verify if it added zfs.ko, which it didn’t, but I found something else there. A module entry for /boot/zfs/zpool.cache, which revealed the whole magic zfsloader was doing.

So I rebooted and decided to replicate this with GRUB2 taking inspiration from a similar situation I was in few months ago when I was trying to boot dragonflybsd kernel to boot from encrypted HAMMER volume, and needed an initrd to mount root from. Following is the final GRUB2 configuration which worked for me, with changes in bold:

menuentry "FreeBSD 9.0-RC3" {
        set root=(hd0,7)
        kfreebsd /boot/kernel/kernel
        kfreebsd_loadenv /boot/device.hints
        kfreebsd_module /boot/zfs/zpool.cache type=/boot/zfs/zpool.cache
        kfreebsd_module_elf /boot/kernel/opensolaris.ko
        kfreebsd_module_elf /boot/kernel/zfs.ko
        kfreebsd_module_elf /boot/kernel/coretemp.ko
        kfreebsd_module_elf /boot/kernel/geom_journal.ko
        kfreebsd_module_elf /boot/kernel/sound.ko
        kfreebsd_module_elf /boot/kernel/snd_hda.ko
        kfreebsd_module_elf /boot/kernel/uhid.ko
        kfreebsd_module_elf /boot/kernel/ukbd.ko
        kfreebsd_module_elf /boot/kernel/ums.ko
        kfreebsd_module_elf /boot/modules/nvidia.ko
#       set kFreeBSD.vfs.root.mountfrom=ufs:ad14p7
        set kFreeBSD.vfs.root.mountfrom=zfs:zroot
	set kFreeBSD.vfs.zfs.prefetch_disable=1
	set kFreeBSD.vfs.zfs.txg.timeout=5
	set kFreeBSD.maxvnodes=250000
	set kFreeBSD.zfs.txg.write_limit_override=805306368
}

In the above configuration (hd0,7) is my UFS2 partition which hosts /boot, which I’ll soon rm -rf once I upgrade my GRUB to 1.99 (which has support for ZFS), ufs:ad14p7 was my old UFS2 root, and zroot is the my ZFS volume I’m using as /.

chateau.d.if!abbe:~ % mount |grep -v zfs
devfs on /dev (devfs, local, multilabel)
devfs on /var/named/dev (devfs, local, multilabel)
/dev/ad14p7 on /boot (ufs, local, read-only)
chateau.d.if!abbe:~ % zfs list
NAME                        USED  AVAIL  REFER  MOUNTPOINT
home                        147G   538G  79.6G  /home
home/abbe                  65.3G   538G  65.3G  /home/abbe
home/tscache                  2G   540G    16K  -
zroot                      18.0G   178G   351M  legacy
zroot/tmp                   124K   178G   124K  /tmp
zroot/usr                  15.7G   178G  6.66G  /usr
zroot/usr/ports            8.67G   178G   244M  /usr/ports
zroot/usr/ports/distfiles  6.55G   178G  6.55G  /usr/ports/distfiles
zroot/usr/ports/packages   1.88G   178G  1.88G  /usr/ports/packages
zroot/usr/src               351M   178G   351M  /usr/src
zroot/var                  1.60G   178G   502M  /var
zroot/var/crash            31.5K   178G  31.5K  /var/crash
zroot/var/db                895M   178G   846M  /var/db
zroot/var/db/pkg           49.6M   178G  49.6M  /var/db/pkg
zroot/var/empty              31K   178G    31K  /var/empty
zroot/var/log              11.2M   178G  11.2M  /var/log
zroot/var/mail               54K   178G    54K  /var/mail
zroot/var/run              89.5K   178G  89.5K  /var/run
zroot/var/squid             231M  11.8G   231M  /var/squid
zroot/var/tmp               663K   178G   663K  /var/tmp

I hope it doesn’t happen again. Fingers crossed….;)

Let there be dragonflies

After an inspiring post by Siju George on bsd-india list, I decided to use DragonFlyBSD 2.10.1 for NAS (a long-time pending project). The installation using their installer was super-simple, and went fine. The userland is same as FreeBSD userland, and I used HAMMER filesystem as root. The NAS is not finished yet, have to install/configure BackupPC yet.

After installing on NAS, I decided to install on my notebook side-by-side with GNU/Linux. My notebook’s HDD is a GPT disk, with GNU GRUB2 used as bootloader. To install DragonFlyBSD on it, I booted with DragonFlyBSD’s USB image, started shell, and created a large partition of type DragonFly HAMMER, and a small UFS2 (to be used as /boot) using gpt(8). Encrypted partitions using dm-crypt. After that I newfs_hammer(8)-ed it, mount(8)-ed it, cpdup(1)-ed USB filesystem onto it as per instructions, and created an initrd using mkinitrd(8). Following is GRUB2 configuration I used to boot into it.

insmod bsd

menuentry "DragonFlyBSD 2.10-RELEASE" {
        set root='(hd0,6)'

        kfreebsd /kernel/kernel

        kfreebsd_module /kernel/initrd.img type=md_image

        kfreebsd_module_elf /kernel/acpi.ko
        kfreebsd_module_elf /kernel/ahci.ko
        kfreebsd_module_elf /kernel/ehci.ko
        kfreebsd_module_elf /kernel/dm.ko
        kfreebsd_module_elf /kernel/md.ko
        kfreebsd_module_elf /kernel/dm_target_crypt.ko
        kfreebsd_module_elf /kernel/wlan.ko
        kfreebsd_module_elf /kernel/wlan_ccmp.ko
        kfreebsd_module_elf /kernel/wlan_tkip.ko
        kfreebsd_module_elf /kernel/ath_hal.ko
        kfreebsd_module_elf /kernel/ath_rate.ko
        kfreebsd_module_elf /kernel/if_ath.ko
        kfreebsd_module_elf /kernel/coretemp.ko
        kfreebsd_module_elf /kernel/sound.ko
        kfreebsd_module_elf /kernel/snd_hda.ko

        set kFreeBSD.vfs.root.mountfrom=ufs:md0s0
        set kFreeBSD.vfs.root.realroot=crypt:hammer:/dev/da0s4:dragon
}
### END /etc/grub.d/01_bsd ###

Thanks to alexh on #dragonflybsd on EFnet for help with installation.