On LVM

This isn’t as much an article as it is a collection of notes, many of which were taken from the online course Mastering Linux Storage Using LVM2 by Andrew Mallett.

Configuration
Metadata
Install
Logical Volume
Demo
Thin Provisioning
Shrinking A Non-Root Volume
Expanding A Non-Root Volume
References

Configuration

Configuration file is in /etc/lvm/lvm.conf.

Use the lvmconfig command to display and manipulate configuration information.

Get the default settings:

$ lvmconfig --type default

Show the config we’ve made:

$ lvmconfig

Show the device filters:

$ lvmconfig --type default | grep filter
  WARNING: Running as a non-root user. Functionality may be unavailable.
        # filter=["a|.*|"]
        # global_filter=["a|.*|"]

Note that the default is to allow device to work with the LVM system on the host.

The “a” in the braces in the rule stands for “allow”, while a “reject” would have an “r”.

To validate the config file after any changes have been made:

$ sudo !!
sudo lvmconfig --validate
  LVM configuration valid.

Metadata

The information about the volume groups and is stored in the physical volumes.

Commands that are useful to see and backup metadata:

pvdisplay
vgdisplay
vgcfgbackup

There is a metadata service to keep the metadata in memory:

$ sudo systemctl status lvm2-lvmetad.service

Actually, this wasn’t found on my Debian 12 (bookworm) distro.

In addition, here are the first 10 lines of one of the metadata backups:

$ sudo head /etc/lvm/backup/kilgore-trout-vg
# Generated by LVM2 version 2.03.16(2) (2022-05-18): Tue Oct  8 00:30:25 2024

contents = "Text Format Volume Group"
version = 1

description = "Created *after* executing 'vgcfgbackup'"

creation_host = "kilgore-trout" # Linux kilgore-trout 6.1.0-25-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.106-3 (2024-08-26) x86_64
creation_time = 1728361825      # Tue Oct  8 00:30:25 2024

In LVM2, backups are created by default.

Install

ssh-keyscan $host | sudo tee -a /etc/ssh/ssh_known_hosts

Install LVM2 if it’s not already:

$ sudo lvm version || sudo apt-get install lvm2 -y
  LVM version:     2.03.16(2) (2022-05-18)
  Library version: 1.02.185 (2022-05-18)
  Driver version:  4.47.0
  Configuration:   ./configure --build=x86_64-linux-gnu --prefix=/usr --includedir=${prefix}/include --mandir=${prefix}/share/man --infodir=${prefix}/share/info --sysconfdir=/etc --localstatedir=/var --disable-option-checking --disable-silent-rules --libdir=${prefix}/lib/x86_64-linux-gnu --runstatedir=/run --disable-maintainer-mode --disable-dependency-tracking --libdir=/lib/x86_64-linux-gnu --sbindir=/sbin --with-usrlibdir=/usr/lib/x86_64-linux-gnu --with-optimisation=-O2 --with-cache=internal --with-device-uid=0 --with-device-gid=6 --with-device-mode=0660 --with-default-pid-dir=/run --with-default-run-dir=/run/lvm --with-default-locking-dir=/run/lock/lvm --with-thin=internal --with-thin-check=/usr/sbin/thin_check --with-thin-dump=/usr/sbin/thin_dump --with-thin-repair=/usr/sbin/thin_repair --with-udev-prefix=/ --enable-applib --enable-blkid_wiping --enable-cmdlib --enable-dmeventd --enable-editline --enable-lvmlockd-dlm --enable-lvmlockd-sanlock --enable-lvmpolld --enable-notify-dbus --enable-pkgconfig --enable-udev_rules --enable-udev_sync --disable-readline

List block devices:

$ lsblk
NAME                            MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINTS
nvme0n1                         259:0    0 476.9G  0 disk
├─nvme0n1p1                     259:1    0   512M  0 part  /boot/efi
├─nvme0n1p2                     259:2    0   488M  0 part  /boot
└─nvme0n1p3                     259:3    0   476G  0 part
  └─nvme0n1p3_crypt             253:0    0 475.9G  0 crypt
    ├─kilgore--trout--vg-root   253:1    0  27.9G  0 lvm   /
    ├─kilgore--trout--vg-swap_1 253:2    0   976M  0 lvm   [SWAP]
    └─kilgore--trout--vg-home   253:3    0   447G  0 lvm   /home

List device-mapper devices:

$ sudo dmsetup ls
kilgore--trout--vg-home (253:3)
kilgore--trout--vg-root (253:1)
kilgore--trout--vg-swap_1       (253:2)
nvme0n1p3_crypt (253:0)
$
$ sudo dmsetup ls --tree
kilgore--trout--vg-home (253:3)
 └─nvme0n1p3_crypt (253:0)
    └─ (259:3)
kilgore--trout--vg-root (253:1)
 └─nvme0n1p3_crypt (253:0)
    └─ (259:3)
kilgore--trout--vg-swap_1 (253:2)
 └─nvme0n1p3_crypt (253:0)
    └─ (259:3)

Layered storage in LVM

physical
- the actual block storage devices
- disks, partitions and raw files
- pv* commands
- pvs, pvmove, pvcreate, pvscan
volume groups
- act as storage pools, aggregating storage together and overcoming the limitations of physical storage size
- vg* commands
- vgs, vgdisplay, vgcreate, vgextend, vgremove
logical volumes
- dev mapper devices which are formatted and presented as a block device
- lv* commands
- lvs, lvdisplay, lvcreate, lvresize, lvextend, lvremove

Logical Volume

List the device mappers:

$ sudo dmsetup ls
kilgore--trout--vg-home (253:3)
kilgore--trout--vg-root (253:1)
kilgore--trout--vg-swap_1       (253:2)
nvme0n1p3_crypt (253:0)

Get information on device mapper devices:

$ sudo dmsetup info

Logical volumes are created as /dev/dm-NUMBER, such as:

$ ls /dev/dm-*
/dev/dm-0  /dev/dm-1  /dev/dm-2  /dev/dm-3

There are symbolic links that are easier to remember that link to these numbered devices:

$ ls -l /dev/mapper/
crw------- 1 root root 10, 236 Oct 13 13:26 /dev/mapper/control
lrwxrwxrwx 1 root root       7 Oct 13 13:26 /dev/mapper/kilgore--trout--vg-home -> ../dm-3
lrwxrwxrwx 1 root root       7 Oct 13 13:26 /dev/mapper/kilgore--trout--vg-root -> ../dm-1
lrwxrwxrwx 1 root root       7 Oct 13 13:26 /dev/mapper/kilgore--trout--vg-swap_1 -> ../dm-2
lrwxrwxrwx 1 root root       7 Oct 13 13:26 /dev/mapper/nvme0n1p3_crypt -> ../dm-0

And:

$ ls -l /dev/kilgore-trout-vg/
total 0
lrwxrwxrwx 1 root root 7 Oct 13 13:26 home -> ../dm-3
lrwxrwxrwx 1 root root 7 Oct 13 13:26 root -> ../dm-1
lrwxrwxrwx 1 root root 7 Oct 13 13:26 swap_1 -> ../dm-2

Resize the logical volume and the filesystem in one fell swoop:

$ sudo lvextend -l +50 -r vg1/lv1

The -r switch calls resize2fs for us.

Demo

$ fallocate -l 500M /tmp/file1
$ sudo losetup -f /tmp/file1
$ losetup -l
NAME       SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE  DIO LOG-SEC
/dev/loop0         0      0         0  0 /tmp/file1   0     512
$ sudo pvcreate /dev/loop0
  Physical volume "/dev/loop0" successfully created.
$ sudo pvs
  PV         VG Fmt  Attr PSize   PFree
  /dev/loop0    lvm2 ---  500.00m 500.00m
$ sudo head /dev/loop0
LABELONE$ LVM2 001ZtarpzLi1SRyWRz5YoGtIi7d7sP5Rg7d@֎ LVM2 x[5A%r0N*>

Reads metadata that was written by pvcreate.

$ sudo vgcreate vg1 /dev/loop0
  Volume group "vg1" successfully created
$ sudo vgs
  VG  #PV #LV #SN Attr   VSize   VFree
  vg1   1   0   0 wz--n- 496.00m 496.00m

Clean up:

$ sudo vgremove vg1
  Volume group "vg1" successfully removed
$ sudo pvremove /dev/loop0
  Labels on physical volume "/dev/loop0" successfully wiped.

We can create a volume group without having first created the physical volume. It will be done automatically:

$ sudo vgcreate vg1 /dev/loop0
  Physical volume "/dev/loop0" successfully created.
  Volume group "vg1" successfully created
$ sudo pvs
  PV         VG  Fmt  Attr PSize   PFree
  /dev/loop0 vg1 lvm2 a--  496.00m 496.00m
$ sudo vgs
  VG  #PV #LV #SN Attr   VSize   VFree
  vg1   1   0   0 wz--n- 496.00m 496.00m

Create the first logical volume and take all the available space:

$ sudo lvcreate -l 100%FREE -n lv1 vg1
  Logical volume "lv1" created.
$ sudo lvs
  LV   VG  Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  lv1  vg1 -wi-a----- 496.00m
$ sudo vgs
  VG  #PV #LV #SN Attr   VSize   VFree
  vg1   1   1   0 wz--n- 496.00m    0
$ sudo pvs
  PV         VG  Fmt  Attr PSize   PFree
  /dev/loop0 vg1 lvm2 a--  496.00m    0
$ sudo dmsetup ls
vg1-lv1 (253:0)
$ sudo dmsetup ls --tree
vg1-lv1 (253:0)
 └─ (7:0)

-l = extents, -L = size (the extent size is the minimum block storage size that can be allocated within the volume group)

The loopback device has major device 7 and minor device 0.

Cleanup:

$ sudo lvremove vg1/lv1
Do you really want to remove active logical volume vg1/lv1? [y/n]: y
  Logical volume "lv1" successfully removed.
$ sudo vgremove vg1
  Volume group "vg1" successfully removed
$ sudo pvremove /dev/loop0
  Labels on physical volume "/dev/loop0" successfully wiped.
$ sudo losetup -d /dev/loop0
$ rm /tmp/file1

Thin Provisioning

$ sudo apt-get install thin-provisioning-tools

Create a thin pool:

$ sudo lvcreate -l 100 --thin-pool tpool vg1

Create a thin volume:

$ sudo lvcreate -V 2g --thin -n thin_lv vg1/tpool

Shrinking A Non-Root Volume

Reboot and login as root (so the home logical volume isn’t in use).

First, unmount the logical volume:

# umount /dev/kilgore-trout-vg/home

Verify the filesystem. Note that this should only be done after unmounting the block device (otherwise, you’ll get a scary error about proceeding).

# e2fsck -fy /dev/kilgore-trout-vg/home

Next, reduce the size of the filesystem, so no data is lost. It is very important that the size of the filesystem is not greater than the size of the volume after it’s resized. This will also lead to data loss and corruption.

# resize2fs /dev/kilgore-trout-vg/home 250G

The extra space that is created when the filesystem has been resized to a size less than the size of the eventual reduced logical volume will be reclaimed when the filesystem is again resized, so no need to worry about losing any extra space.

Next, reduce the size of the logical volume itself. In my case, I want to reduce the size by 150G:

# lvreduce -L -150G /dev/kilgore-trout-vg/home

Then, resize the filesystem again to reclaim the previously-mentioned extra space:

# resize2fs /dev/kilgore-trout-vg/home

Finally, remount the volume:

# mount -a

Of course, there’s a simpler way to do this. We can resize both the filesystem and the volume in one step:

# lvresize –size 5G –resizefs /path/to/device

Nice.

Expanding A Non-Root Volume

$ sudo lvextend -L +150G /dev/kilgore-trout-vg/root
$ resize2fs /dev/kilgore-trout-vg/root

$ lsblk
NAME                            MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINTS
nvme0n1                         259:0    0 476.9G  0 disk
├─nvme0n1p1                     259:1    0   512M  0 part  /boot/efi
├─nvme0n1p2                     259:2    0   488M  0 part  /boot
└─nvme0n1p3                     259:3    0   476G  0 part
  └─nvme0n1p3_crypt             253:0    0 475.9G  0 crypt
    ├─kilgore--trout--vg-root   253:1    0 177.9G  0 lvm   /
    ├─kilgore--trout--vg-swap_1 253:2    0   976M  0 lvm   [SWAP]
    └─kilgore--trout--vg-home   253:3    0   297G  0 lvm   /home

$ df -h
Filesystem                           Size  Used Avail Use% Mounted on
udev                                 7.7G     0  7.7G   0% /dev
tmpfs                                1.6G  2.1M  1.6G   1% /run
/dev/mapper/kilgore--trout--vg-root  175G   26G  143G  16% /
tmpfs                                7.7G   84K  7.7G   1% /dev/shm
tmpfs                                5.0M  8.0K  5.0M   1% /run/lock
/dev/nvme0n1p2                       456M  153M  279M  36% /boot
/dev/mapper/kilgore--trout--vg-home  292G   41G  237G  15% /home
/dev/nvme0n1p1                       511M   14M  498M   3% /boot/efi
tmpfs                                1.6G  2.5M  1.6G   1% /run/user/1000