Note: This information is geared toward advanced users. Don’t try this at home.
This weekend it was time to perform some maintenance on my main server. Its single IDE hard drive had developed the metallic whine indicative of bearing failure, and the power supply began the constant whistling telling the world that it was doomed.
Original storage/power config:
- Western Digital 200GB 8MB cache ATA-133
- Onboard AMD-75xx IDE controller
- 250W ATX PSU
- 2x Seagate 250GB 16MB cache SATA/300
- Promise TX2 SATA controller
- 600W Enermax NoiseTaker II ATX12V 2.0 PSU
The typical setup for this is: create partitions of matching size on each SATA drive and pair these off, creating numerous RAID1 arrays. This has always struck me as inelegant. But, every HOWTO I could find insisted that at the very least, /boot and swap must be created this way. You can put / on LVM if you’re a cowboy, but you’re pushing your luck.
Knowing something of the way Linux RAID, LVM and LILO work, I decided this didn’t make any sense.
The resulting config is much more elegant than the method prescribed by conventional wisdom. A single RAID array hosts my logical volumes, everything including the filesystems, kernel image and swap are hosted within these, and the bootloader understands the whole thing.
Caveat: GRUB cannot understand LVM2, but LILO can. So much for GRUB being newer and more advanced – while most everyone now swears by GRUB, LILO has continued to add innovative features behind the scenes.
What I used:
- Distribution: Gentoo Linux
- Kernel: Linux 2.6.22
- Bootloader: LILO 22.8
- LVM: LVM2 2.02.10
- MDADM: mdadm 2.6.2
And this initrd script.
I will shortly investigate how this would be done under a Red Hat based distribution. This is of particular interest to me, because Red Hat’s installer insists up and down that /boot must not be on anything more exotic than an MD RAID1 based partition, which is obviously not a requirement based on technical limitations of the underlying software.
Getting Started – RAID
I booted the server from a LiveCD (the Gentoo installcd in this case).
Both SATA disks were prepped for integration into a RAID array by using
fdisk to create a single large partition on each disk and setting the type of the partition to ‘fd’, which is the type code for ‘Linux RAID Autodetect’.
mdadm -C -l1 -n2 /dev/md0 /dev/sda1 /dev/sdb1
Time to set up LVM. If you’re not familiar with LVM, I highly recommend that you read some documentation before staking your reputation on it. The LVM2 Resource Page and LVM HOWTO on TLDP are great reading.
livecd:/ # pvcreate /dev/md0
livecd:/ # vgcreate TMA-1 /dev/md0
livecd:/ # lvcreate -L1G -nswap TMA-1
livecd:/ # lvcreate -L300M -nroot TMA-1
livecd:/ # lvcreate -L25G -nhome TMA-1
livecd:/ # lvcreate -L25G -nvar TMA-1
livecd:/ # lvcreate -L128M -ntmp TMA-1
livecd:/ # lvcreate -L181G -nusr TMA-1
livecd:/ # mkswap /dev/TMA-1/swap
livecd:/ # mke2fs -j /dev/TMA-1/root
You get the idea. I then copied the contents of the filesystem on the original, dying IDE drive to this fresh, new filesystem once I mounted all volumes.
Kernel and INITRD
Next step: Kernel has to be built with SATA controller driver, SCSI disk support, Device Mapper and RAID1 built in. Install the kernel and modules as normal.
Use the lvm2create_initrd script as follows:
sh lvm2create_initrd.sh -M gentoo -c /etc/lvm/lvm.conf -r /dev/md0 -R /etc/mdadm.conf 2.6.22-gentoo-r1
If you’re using Red Hat or Debian, specify ‘redhat’ or ‘debian’ as the -M option. Specify config file locations and kernel version string as needed.
Bootloader – LILO
I used version 22.8. I’m unsure whether earlier versions support the device mapper (the kernel component used to read LVM style partition boundaries). I will list here the deviations from a standard LILO config file. Please familiarize yourself with other aspects LILO configuration.
Two lilo.conf files need to be created, or you can modify the one. The reason is as follows:
boot = /dev/sda
boot = /dev/sdb
LILO needs to be written to the MBR of both (or all) RAID array members. Specifying
boot = /dev/md0 will cause it to mutter racial epithets and quit. Also, you can’t use two boot parameters. So don’t paste the above verbatim into your config file. Silly goose.
image = /boot/vmlinuz-2.6.22
root = /dev/ram0
initrd = /boot/initrd-lvm2-2.6.22-gentoo-r1.gz
append = "lvm2root=/dev/TMA-1/root"
label = Gentoo
The important lines in the above config snippet are
initrd =, and
append =. Specify the filename given to you by the lvm2create_initrd script in the
Remember to edit your fstab file. Your device paths for logical volumes are now named according to the volume group you created. In my case, for example:
/dev/TMA-1/root / xfs noatime 0 0
/dev/TMA-1/swap none swap sw 0 0
Write LILO to your MBR’s, reboot and you’re set. Gentoo’s init scripts understand how to start and stop RAID and LVM, as do many other modern distributions.
There’s something terribly satisfying about a state-of-the-art configuration. 😉