Write-Intent Bitmaps for MD devices

I’ve been meaning to write about this for a while and since it’s come up recently (Write Intent Bitmaps, Software RAID) I figured now would be a good time. Basically what the write-intent bitmap does is keep track of which blocks are being used so that in the event that a resync is needed (power failure, kernel crash, disk temporarily offline) only the blocks that are out of sync are copied. This drastically improves resync times. Of course there is a downside.

The downside is that the md driver for a particular device will be far more active. In the server I’m currently testing on the mdx_raid1 process with the write-intent bitmap enabled uses far more system time than it did before the write-intent bitmap was enabled. Of course, the system load on that particular server is still extremely low. I wish I had done kept some statistics before and after to have better numbers. At the moment it seems as the the increase is significant int hat it goes from virtually nothing to something but still insignificant enough to be concerned about. I would have to test on a server with much higher disk IO to see if there were any problems using write-intent bitmaps under load.

If the impact of this change proves to be minimal across systems then it seems like a worthwhile tradeoff. I have been bitten not infrequently by long md resync times, during which the systems are far less responsive. Sometimes in an effort to take advantage of cutting edge software I’ll run a kernel that is less than stable (such as the Debian vserver kernels a year or so ago). Sometimes a piece of hardware will be flaky and it takes time to identify while in the meantime the server experiences kernel panics or system freezes, I admit that this comes from sometimes using less than reliable hardware due to budget constraints and a desire to reuse as much as possible. I’ve also had power failures that have outlasted the UPS on a set of systems. But however it happens, each of these instances require an md resync after the crash. And that process is slow and resource intensive.

I’ve also found that in an effort to diagnose the problem I often need to reboot before the resync is done, which forces the resync to begin again. In the past to alleviate this issue I’ve partitioned my disks to keep the system partition separate and small. These days I’m enjoying the use of LVM snapshots to facilitate backups and would prefer a simple disk layout utilizing LVM. To this end I’ve begun partitioning systems with a small (256MB) boot partition and using the rest of the disk space for LVM (of course RAID is used for both partitions). With write-intent bitmaps enabled I can keep recovery time down while maximizing flexibility. Of course I’m only testing this one 1 server at the moment. I need to examine what happens under load, as well as what a recovery is like before committing to other machines. At least the process is easy to do.

To enable write intent bitmaps use the following command:

• mdadm -G /dev/mdx -b internal
  • -G /dev/mdx = “grow” md device
  • -b internal = create and enable bitmap in metadata

To disable write-itnent bitmaps is just as easy:

• mdadm -G /dev/mdx -b none