[Solved] “Logical volume is used by another device”

tl;dr: use dmsetup remove before trying lvremove

Note: Volume group and logical volume names have been substituted here. I’m not entirely sure it’s necessary, but better safe than sorry. If following this, please use the names of your volume group[s] and logical volume[s]

I am in the process of combining fileserver information, and so I have been touching parts of the system not usually looked at in the normal case of day-to-day operations. For some reason, on one of my logical volumes I had created a partition table and added a partition. Of course, that worked normally so there was no reason to be aware of this — clearly I had blanked the fact that I did it at all not long after doing so — until recently.

The Problem

Logical volume vg/lv-old is used by another device.

After copying the data over to a new logical volume, I wanted to remove the now-unnecessary original logical volume that contained the partition. Easy, right?

# lvremove -v /dev/vg/lv-old
    DEGRADED MODE. Incomplete RAID LVs will be processed.
    Using logical volume(s) on command line
  Logical volume vg/lv-old is used by another device.

Okay, what’s using it? cat /proc/mounts reports that it isn’t mounted. lsof and fuser return nothing. Maybe retrying the command will work*… nope.

There are a bunch of posts around this, mostly saying “make sure it is umounted first”, or “try using -f with lvremove“. And the old favourite: “a reboot fixed it”.

Find Out device-mapper’s Mapping

Well, the culprit in this case seemed to be device-mapper creating a mapping which counted as ‘in-use’. Check for the mapping via:

# dmsetup info -c | grep old
vg-lv--old       253   9 L--w    1    2      1 LVM-6O3jLvI6ZR3fg6ZpMgTlkqAudvgkfphCyPcP8AwpU2H57VjVBNmFBpL

Find Out Mapped Device

Then use that to find out what is holding it:

$ ls -la /sys/dev/block/253\:9/holders

drwxr-xr-x 2 root root 0 Dec 12 01:07 .
drwxr-xr-x 8 root root 0 Dec 12 01:07 ..
lrwxrwxrwx 1 root root 0 Dec 12 01:07 dm-18 -> ../../dm-18

Remove Device (via `dmsetup remove`)

Then do a dmsetup remove on that device-mapper device:

# dmsetup remove /dev/dm-18

Retry `lvremove`

And you’re good to go with lvremove:

# lvremove -v /dev/vgraid6/lv-old
    DEGRADED MODE. Incomplete RAID LVs will be processed.
    Using logical volume(s) on command line
Do you really want to remove active logical volume lv-old? [y/n]: y
    Archiving volume group "vg" metadata (seqno 35).
    Removing vg-lv--old (253:9)
    Releasing logical volume "lv-old"
    Creating volume group backup "/etc/lvm/backup/vg" (seqno 36).
  Logical volume "lv-old" successfully removed

Bish bash bosh!


*: I’m not sure of the thought process behind “just try it again”.

I’m reminded of a short bit of Darrell Hammond’s stand up (paraphrased):

“You know that message you get when you dial the wrong number that tells you to ‘check you have the right number and dial again’? Well, women will check the number and try again. Men will try the same number, but this time we’ll push the buttons a ******** harder…”

[Solved] “Filesystem is already n blocks long. Nothing to do!”

tl;dr: if you’re sure you did everything right, use lsblk or parted (etc) to see if a partition table is present on your logical volume.

So I am in the process of merging the content of two fileservers, and had the need to extend a logical volume to accommodate some additional data. No problem- that’s one of the benefits of using LVM!

Except after resizing, I ran into a problem:

$ lvextend +150G /dev/vg/lvinquestion
$ resize2fs /dev/vg/lvinquestion
> The filesystem is already 268435200 (4k) blocks long. Nothing to do!

Wait, what? Aside from the fact I could have combined the comments by including the --resizefs option to lvextend, why was resize2fs complaining that there was “Nothing to do!”?

Fortunately SE Arquade user ToxicFrog had the answer:

@bertieb parted reports the partition size, not the filesystem size
If it’s a partitioned LV you need to resize the partition after expanding the VL

Ah, whoops! I’m not sure why I partitioned the LV (it only had one partition) but I must have done so.

lsblk confirmed the partition:

sdh                                8:112  0   2.7T  0 disk
??sdh1                             8:113  0   2.7T  0 part
  ??md1                            9:1    0  10.9T  0 raid6
    ??vg-lv                      253:9    0   1.2T  0 lvm
    ? ??vg-lv1                   253:18   0   1.1T  0 part

So, then what? Well, I used dd to copy the filesystem to a new logical volume, then extended that, and finally removed the original:

# dd if=/dev/dm-18 bs=1M | pv -s 1T |  dd of=/dev/vg/lv-new bs=1M
# lvextend --resizefs -L 1.15T /dev/vg/lv-new
# lvremove /dev/vg/lv
# lvrename vg lv-new lv

(pv was included to give a nice progress indicator, rather than faffing around with SIGUSR1)

And that was that. There was a slight problem with removing the original logical volume, but more on that later…