Performance impact of Logical Volume Manager (LVM) under Ubuntu 6.04 (Dapper Drake)

I’ve always been concerned about the potential performance impact of a system confgured to use LVM. Reports on the internet have been conflicting, with some people reporting a significant loss of performance while others report no difference, or even a s light improvement, in performance. LVM adds a level of manageability and flexibility to partitioning that is welcome to most system administrators. However one must also be concerned with the performance of a filesystem, especially on servers where the filesystems are the bottleneck. To satisfy my curiosity I decided to run some benchmarks and see what the performance delta between a system with LVM and one without would be.

Test System

Hardware

• AMD Athlon 64 3000+ (1.8GHz)
• nForce 4 Motherboard
• 512MB RAM
• Wester Digital WD800JB 80GB Hard Disk

Software

• Ubuntu 6.04 (Dapper Drake) AMD64
• linux-image-2.6.15-29-amd64-generic - 2.6.15-29.60
• bonnie++ - 1.03abuild1
• lvm2 - 2.02.02-1ubuntu1.5
• lvm-common - 1.5.20ubuntu4

Methodology

I ran the tests on a scratch partition that was either formatted with an ext3 filesyste, using the command mkfs.ext3 or setup with LVM with a logical volume that filled the partition and an ext3 filesystem on top of that. I used the [[bonnie++] program with the following paramenters:

• bonnie++ -d /mnt/hda3 -s 1024 -n 16:102400:0:1024 -x 5

This sets the program to use a 1GB file for the first set of tests and to use over 16,000 files from ranging in size from 0 to 100K in 1024 directories for the second set of tests. And finally the tests are rerun 5 times. In my calculations I used the average of the results in the 5 runs.

Results

The first graph represents most of the tests design to simulate a “database” like situation. Interestingly while writes were quicker without LVM, reads were actually a bit quicker with LVM. However none of the differences was by more than 5% so I’d call that pretty even.

The second graph has the seeks results which actually are part of the first test but graph better with the second test. The rest of the results are from a workload designed to simulate the load on a mail, news, or proxy server. In this case we can see that the partition without LVM is faster across the range of tests. The most significant differences are in the sequential tests where the performance without LVM exceeded with LVM by 17% in the 3 tests. In the random tests the difference is below 5%.

Conclusions

I’m a little suspicious about the results as the performance advantage of not having LVM seems to be inflated by 1 data point in the sequential delete (seq_del) which could be an outlier. Unfortunately the random files generated in these tests are different between runs which could lead to favorable set of files to be generated in a particular run. It would seem wise in a program like this to seed the random number generator with a known value, and allow that value to be overridden by the user therefore allowing tests to be completely reproducible.

Given that concern I’d say that my fears that LVM would be a significant detriment to performance have largely been allayed. It is also important to note that the version of ext3 in this Ubuntu release is not nearly as optimized as later versions. I plan on retrying these tests when Ubuntu 7.10 (Gutsy Gibbon) is released and see what if anything is different. Some other thoughts for future testing include:

• consider other benchmarking programs (iozone, tiobench, dbench, postmark)
• consider testing other file systems (ext2, jfs, xfs)
• seek ways to improve upon testing methodology

Further Reading

• LVM2 Resource Page
• LVM HOWTO
• Performance comparison between LVM and a straight disk
• Benchmarking an encrypted dm-crypt/LVM/ext3/SELinux hard drive with bonnie++ and hdparm
• LVM performance problems
• Differences in performance between file and LVM based images