Saturday, September 22, 2012

mdadm external bitmap boot hack

Update 3/1/2014: I have updated my procedure for Debian 7.4.

Linux's mdadm gives you the ability to use a write intent bitmap, which helps speed resync if a drive fails and is then re-added. There are two bitmap storage options, internal and external. The internal storage stores the bitmap on the raid array disks and can slow writing significantly. External storage uses a file on an ext2 or ext3 file system. It can be quicker than internal storage during writing but causes big problems during boot.

In order for the bitmap to function, it must be writable at the time that mdadm assembles the array. If it is not writable, mdadm will fail to start the array. At the time that mdadm assembles the array, typically no partitions are writable.  My solution to this was to shift the array assembly to some time period after mountall (mounts the file systems found in fstab) had executed.

First I prevent any raid partitions from mounting during boot, noauto in fstab.

I copied /etc/mdadm/mdadm.conf to /etc/mdadm/mdadm.manual.conf.  Then I edited /etc/mdadm/mdadm.conf changing DEVICE partitions to DEVICE /dev/null. This will make mdadm scan for raid partitions within the null device, in which it will find none. Raid devices will no longer assemble during the boot process.

Then I created a new script in /etc/rcS.d/ named S02mountRaid (don't forget execute permissions). The script contains the following lines.
mdadm --assemble --scan --config=/etc/mdadm/mdadm.manual.conf mount /mnt/raid5
This will cause mdadm to scan the copy of our mdadm.conf which we did not modify. Mdadm will correctly assemble the raid devices found in that file along with assigning the external bitmap to the proper array. The raid device is then mounted.

This script runs at every run level, and runs before any other init.d script. Future work, I need to find a way to make init.d wait for script completion.