Skip to main content

LVM Rescue Procedure

Tobi Oetiker, 2007-05-04

Issue

Add a bunch of new disks to your box while it is running. In order to add them to your LVM volume group, edit/etc/lvm.conf to allow the new devices to be picked up. All goes well. Create two new Volumes.

Now for some reason the old version of /etc/lvm.conf gets restored. Nothing happens until the next reboot, when LVM cannot re-assemble all the disks for the volume group.

Don't try this on a real system, unless things look really bad! Don't come whining when you destroy your gear.

Recovery instructions

If you realize that /etc/lvm.conf is not picking up all the disks, you are already on the winning side. Just edit the file, and run lvmdiskscan to make lvm check again. If everything is back where it should be, run lvscan to see if any of the volumes are inactive. Reactivate them with:

lvchange -a y /dev/volume_group/lv_name

If you have already destroyed some config by stroking pvcreate the wrong way, there is still hope. Go to/etc/lvm/archive/ and find the most recent backup of lvm.conf. The backups are plain text! Before you do anything else though, make sure that all disks are visible now! With this established, read man vgcfgrestoreto restore the vg configuration to its former glory.

pvcreate  --restorefile /etc/lvm/archive/filename --uuid uuid
vgcfgrestore --file /etc/lvm/archive/filename

It may be, that you are still missing one logical volume. The LVM commands create an automatic backup prior to any configuration changes. This means that there is no backup of the configuration AFTER the last change. So how to get that last volume back. If you look at the last backup file, you will see the command that was used to create the last volume. The good news is, that you can issue this same command with the -Znoption to get your volume back (YES READ THE MANUAL PAGE!). LVM seem to be pretty predictable in the way it creates new volumes. Obviously this will only work, when nothing else has been changed.

lvcreate --size 20G -Zn --name lv_name volume_group
fsck -f /dev/volume_group/lv_name

And yes, this is not just an academic example. My adrenalin is still pumping.

 

Sorry

It seems JavaScript has been disabled in your browser. This site, like most of the Internet contents these days, does not work properly without JavaScript. Please re-enable JavaScript, or use a different Browser.


If you need help fixing your browser, please send mail to support@oetiker.ch