Shulou(Shulou.com)06/03 Report--

On the left above is my personal Wechat. For further communication, please add Wechat. On the right is my official account "Openstack Private Cloud". If you are interested, please follow us.

Last time, a physical machine was installed with PVE. Because there was no good planning at the beginning, there was a problem when configuring and using storage, and then it was simply reinstalled. When reinstalled, it was found that in addition to the formatting of the system disk, the partition information of other original disks still existed, especially a large amount of lvm information remained, so manual intervention was still needed. Make a record here.

According to the general order in which lvm deletes lv vg pv, first delete lv under vg, then delete vg, and finally delete pv, but in general, a large number of lv will be generated during the use of pve, so when deleting, you can use the vgremove command to delete vg directly, and delete the lv under vg at the same time.

After the reinstallation is completed, the pve node will create a vg called pve. In addition, there will be the original residual vg, as shown below:

Root@pve:~# vgdisplay | more-Volume group-VG Name ceph-d910d1d3-3595-4c5a-93ed-579e4a0968b4 System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 17 VG Access read/write VG Status resizable MAX LV 0 Cur LV 1 Open LV 0 Max PV 0 Cur PV 1 Act PV 1 VG Size 30.00 GiB PE Size 4.00 MiB Total PE 7679 Alloc PE / Size 7679 / 30.00 GiB Free PE / Size 0 / 0 VG UUID Ngu9qK-BILc-sY4K-BFLG-0ICd-k4Fp-5mVXpg-Volume group-- VG Name ceph-3ab6b8cb-a06c-458f-9947-eaaf40fc0525 System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 17 VG Access read/write VG Status resizable MAX LV 0 Cur LV 1 Open LV 0 Max PV 0 Cur PV 1 Act PV 1 VG Size 30.00 GiB PE Size 4.00 MiB Total PE 7679 Alloc PE / Size 7679 / 30.00 GiB Free PE / Size 0 / 0 VG UUID T8ReNU-YOdO-Bver-KdYe-NG1p-qoiv-T6tUO1-Volume group-VG Name ceph-aa50bad9-b042-40bd-b060-438cddaf8ff2 System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 17 VG Access read/write VG Status resizable MAX LV 0 Cur LV 1 Open LV 0 Max PV 0 Cur PV 1 Act PV 1 VG Size 30.00 GiB PE Size 4.00 MiB Total PE 7679 Alloc PE / Size 7679 / 30.00 GiB Free PE / Size 0 / 0 VG UUID Wnn3OH-X11z-TovS-VFxn-sbz7-btsH-jPbUUa-Volume group-VG Name ceph-38029056-30a1-4bb8-b94f-da32dde62217 System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 17 VG Access read/write VG Status resizable MAX LV 0 Cur LV 1 Open LV 0 Max PV 0 Cur PV 1 Act PV 1 VG Size 30.00 GiB PE Size 4.00 MiB Total PE 7679 Alloc PE / Size 7679 / 30.00 GiB Free PE / Size 0 / 0 VG UUID IT06de-ABnj-yE07-4VFS-WqES-0cep-7j3O4I-Volume group-VG Name vg-sdb System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 4 VG Access read/write VG Status Resizable MAX LV 0 Cur LV 1 Open LV 0 Max PV 0 Cur PV 1 Act PV 1 VG Size 32.00 GiB PE Size 4.00 MiB Total PE 8191 Alloc PE / Size 7696 / 30.06 GiB Free PE / Size 495 / 1 .93 GiB VG UUID QDHI0E-K5Vd-DZ77-o6N9-JRGC-AezH-eIai7Q-Volume group-VG Name pvevg2 System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 493VG Access read/write VG Status resizable MAX LV 0 Cur LV 23 Open LV 7 Max PV 0 Cur PV 1 Act PV 1 VG Size 556.93 GiB PE Size 4.00 MiB Total PE 142573 Alloc PE / Size 140836 / 550.14 GiB Free PE / Size 1737 / 6.79 GiB VG UUID 0NWJ32-iGhh-jPU8-BwxZ-46Ie-5DDB-md0XFB- -- Volume group-VG Name ceph-18ccfe46-ab1a-44e4-9b5e-910d55679b2d System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 17 VG Access read/write VG Status resizable MAX LV 0 Cur LV 1 Open LV 0 Max PV 0 Cur PV 1 Act PV 1 VG Size 30.00 GiB PE Size 4.00 MiB Total PE 7679 Alloc PE / Size 7679 / 30.00 GiB Free PE / Size 0 / 0 VG UUID 6A7ViB-O1UM-sbBw-iCSn-RR3D-IA2o-Uf5nMW-Volume group-- VG Name ceph- Ca21f994-6ea2-4d97-9377-b71e1713089d System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 17 VG Access read/write VG Status resizable MAX LV 0 Cur LV 1 Open LV 0 Max PV 0 Cur PV 1 Act PV 1 VG Size 30.00 GiB PE Size 4.00 MiB Total PE 7679 Alloc PE / Size 7679 / 30.00 GiB Free PE / Size 0 / 0 VG UUID bBvYWE-skn5-lD6G-A0aa-HvC1-cufv-HPh69e-Volume group-VG Name pve System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 7 VG Access read/write VG Status resizable MAX LV 0 Cur LV 3 Open LV 2 Max PV 0 Cur PV 1 Act PV 1 VG Size 556.68 GiB PE Size 4.00 MiB Total PE 142509 Alloc PE / Size 138413 / 540.68 GiB Free PE / Size 4096 / 16.00 GiB VG UUID 3rZ2nK-8oci-O0gh-c6ac-1XXX-slCA-5kh4jZ

Next, delete all vg except pve:

Root@pve:~# vgremove ceph-d910d1d3-3595-4c5a-93ed-579e4a0968b4Do you really want to remove volume group "ceph-d910d1d3-3595-4c5a-93ed-579e4a0968b4" containing 1 logical volumes? [Yzone]: yDo you really want to remove and DISCARD active logical volume ceph-d910d1d3-3595-4c5a-93ed-579e4a0968b4/osd-block-8b281dbd-5dac-40c7-86a9-2eadcd9d876b? [Yzone]: y Logical volume "osd-block-8b281dbd-5dac-40c7-86a9-2eadcd9d876b" successfully removed Volume group "ceph-d910d1d3-3595-4c5a-93ed-579e4a0968b4" successfully removedroot@pve:~# root@pve:~# vgremove ceph-3ab6b8cb-a06c-458f-9947-eaaf40fc0525Do you really want to remove volume group "ceph-3ab6b8cb-a06c-458f-9947-eaaf40fc0525" containing 1 logical volumes? YDo you really want to remove and DISCARD active logical volume ceph-3ab6b8cb-a06c-458f-9947-eaaf40fc0525/osd-block-7b89b395-2506-4997-a545-14eb1a5a029f? [YBO]: y Logical volume "osd-block-7b89b395-2506-4997-a545-14eb1a5a029f" successfully removed Volume group "ceph-3ab6b8cb-a06c-458f-9947-eaaf40fc0525" successfully removedroot@pve:~# root@pve:~# vgremove-f ceph-aa50bad9-b042-40bd-b060-438cddaf8ff2 Logical volume "osd-block-c1ccede6-d03b-409b-952d-57fd702b08fd" successfully removed Volume group "ceph-aa50bad9-b042-40bd-b060-438cddaf8ff2" successfully removedroot@pve:~# root@pve:~# vgremove-f ceph- 38029056-30a1-4bb8-b94f-da32dde62217 Logical volume "osd-block-a2331e8c-907b-48dd-a053-071d0e1d88ca" successfully removed Volume group "ceph-38029056-30a1-4bb8-b94f-da32dde62217" successfully removedroot@pve:~# vgremove-f vg-sdb Logical volume "lvm-sdb" successfully removed Volume group "vg-sdb" successfully removedroot@pve:~# vgremove-f pvevg2 Logical volume "vm-112-disk-3" successfully removed Logical volume "vm-112-disk-4" successfully removed Logical volume "vm-111-disk-5" Successfully removed Logical volume "vm-112-disk-5" successfully removed Logical volume "vm-113-disk-5" successfully removed Logical volume "vm-112-disk-7" successfully removed Logical volume "snap_vm-112-disk-2_pve52init1" successfully removed Logical volume "snap_vm-112-disk-6_pve52init1" successfully removed Logical volume "vm-111-state-pve52ceph2" successfully removed Logical volume "vm-113-state-pve52ceph2" successfully removed Logical volume "vm-111-state-pve52cephok" successfully removed Logical volume "vm-113 State-pve52cephok "successfully removed Logical volume pvevg2/vm-111-disk-2 is used by another device.root@pve:~# vgremove ceph-18ccfe46-ab1a-44e4-9b5e-910d55679b2d-f Logical volume" osd-block-7e68233c-97b2-4bd7-8d24-a48212e02943 "successfully removed Volume group" ceph-18ccfe46-ab1a-44e4-9b5e-910d55679b2d "successfully removedroot@pve:~# vgremove ceph-ca21f994-6ea2-4d97-9377-b71e1713089d-f Logical volume" osd-block-a489f6f9-e24d-408c-ada2-850dbe876c23 "successfully removed Volume group "ceph-ca21f994-6ea2-4d97-9377-b71e1713089d" successfully removedroot@pve:~#

Check vg and pv again:

Root@pve:~# vgdisplay-Volume group-VG Name pve System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 7 VG Access read/write VG Status resizable MAX LV 0 Cur LV 3 Open LV 2 Max PV 0 Cur PV 1 Act PV 1 VG Size 556.68 GiB PE Size 4.00 MiB Total PE 142509 Alloc PE / Size 138413 / 540.68 GiB Free PE / Size 4096 / 16.00 GiB VG UUID 3rZ2nK-8oci-O0gh-c6ac-1XXX-slCA-5kh4jZ root@pve:~#root@pve:~# pvdisplay-Physical volume- -PV Name / dev/sda3 VG Name pve PV Size 556.68 GiB / not usable 1.98 MiB Allocatable yes PE Size 4.00 MiB Total PE 142509 Free PE 4096 Allocated PE 138413 PV UUID 6LqVrb-YLWy-doYc-RMVR-XA58-6XuQ-sSVpwD "/ dev/ Sdb1 "is a new physical volume of" 556.93 GiB-NEW Physical volume-PV Name / dev/sdb1 VG Name PV Size 556.93 GiB Allocatable NO PE Size 0 Total PE 0 Free PE 0 Allocated PE 0 PV UUID 45XFJa-Uixa-4zi7-2Bbe-LnEs-vRea-XAKjiB

Clean up successfully!

In addition, record how to delete some lv like dog skin plaster to remind you how to delete the device when the device is busy and occupied. I referred to the contents of this website:

Http://blog.roberthallam.org/2017/12/solved-logical-volume-is-used-by-another-device/comment-page-1/

Dmsetup is mainly used as a lower-level instruction, and the main steps are as follows:

# lvremove-v / dev/vg/lv-old DEGRADED MODE. Incomplete RAID LVs will be processed. Using logical volume (s) on command line Logical volume vg/lv-old is used by another device. # dmsetup info-c | grep oldvg-lv--old 253 9 LUV w 12 1 LVM-6O3jLvI6ZR3fg6ZpMgTlkqAudvgkfphCyPcP8AwpU2H57VjVBNmFBpLTis8ia0NE$ ls-la / sys/dev/block/253\: 9/holdersdrwxr-xr-x 2 root root 0 Dec 12 01:07. Drwxr-xr-x 8 root root 0 Dec 12 01:07.. lrwxrwxrwx 1 root root 0 Dec 12 01:07 dm-18->.. / dm-18# dmsetup remove / dev/dm-18# lvremove-v / dev/vgraid6/lv-old

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.