Shulou(Shulou.com)06/01 Report--

In this issue, the editor will bring you about how to understand the LVM logic volume management of Linux system. The article is rich in content and analyzes and describes for you from a professional point of view. I hope you can get something after reading this article.

I. preface

Every Linux user will encounter this dilemma when installing Linux: when partitioning the system, how to accurately evaluate and allocate the capacity of each hard disk partition, because the system administrator should not only consider the current capacity of a partition, but also foresee the maximum capacity that may be needed in the future. Because if the estimate is not accurate, the administrator may even have to back up the entire system, clear the hard disk, repartition the hard disk, and then restore the data to the new partition when a partition is insufficient.

Although there are many tools available to dynamically adjust disks, such as PartationMagic, etc., it does not completely solve the problem, because a partition may be exhausted again On the other hand, this requires a reboot of the system, downtime is unacceptable for many critical servers, and for adding new hard drives, when you want a file system that spans multiple hard drives, the partition adjustment program won't solve the problem.

Therefore, the perfect solution should be to adjust the size of the file system freely under the premise of zero downtime, which is convenient for the file system to span different disks and partitions. Fortunately, the logical volume management (LVM,LogicalVolumeManager) mechanism provided by Linux is a perfect solution.

LVM is the abbreviation of logical disk Volume Management (LogicalVolumeManager). It is a mechanism for managing disk partitions in Linux environment. LVM is a logical layer based on hard disks and partitions to improve the flexibility of disk partition management. LVM system administrators can easily manage disk partitions, such as connecting several disk partitions into a single block volume group (volumegroup) to form a storage pool. Administrators are free to create logical volume groups (logicalvolumes) on volume groups and further create file families on logical volume groups. Administrators can easily resize storage volume groups through LVM, and disk storage can be named, managed, and allocated according to groups, such as "development" and "sales", instead of using physical disk names "sda" and "sdb". And when the system adds a new disk, the LVM administrator does not have to move the files from the disk to the new disk to make full use of the new storage space, but directly extends the file system across the disk.

II. Brief introduction to LVM

LVM (Logical Volume Manager, logical Volume Manager) is a method of allocating hard drive space into logical volumes, so that hard drives can be simply resized without using partitions. Traditionally, a partition size is static. If a user has no space on this partition, he can either repartition (which may require the entire operating system to be reinstalled) or use the assembly machine like a symbolic link. With LVM, hard drives or hard drive collections are assigned to one or more physical volumes (Physical Volume). Physical volumes are merged into logical volume groups (Logical Volume Group), with the only exception being the / boot partition. Because physical volumes cannot span more than one drive, if you want a logical volume group to span more than one drive, you should create one or more physical volumes on the drive. Logical volume groups are divided into logical volumes, with mount points assigned (such as / home and /), and file system types (such as ext3). When the partition reaches their limit, the free space in the logical volume group can be added to the logical volume to increase the size of the partition. When a new hard disk drive is added to the system, it can also be added to the logical volume group, which is an expandable partition. Because LVM allows logical volumes to be created outside the machine's physical storage resources, and logical volumes are extensible partitions in logical volume groups. Because LVM allows logical volumes to be created outside the machine's physical storage resources, and logical volumes can be expanded and reduced while the system is still running, it provides Linux system administrators with the storage flexibility they dream of.

The role of LVM: the actual operation of LVM varies according to different practices. But LVM usually includes physical storage clustering (Physical Storage Grouping), resizing logical sectors (Logical Volume Resizing), and data transfer (Data Migrating).

III. Why use LVM

For Linux users, one of the problems encountered when installing a Linux machine is to estimate and allocate enough hard disk space to each partition. This is a very common problem for both a system administrator who is looking for space for a server and an ordinary user who is running out of disk. Generally speaking, the solution we think of is:

Attach a new hard drive, and then use a symbolic link to link to the new hard drive. Some tools for resizing partitions are used to partition non-invasive data.

However, these are only temporary solutions, and they all require the machine to stop running or the partition work that lasts for a long time, so that the service cannot be provided properly. Even if you alleviate the hard disk space problem, you will still face the same problem in the near future.

For a large site, there are a large number of customers, and connected to the Internet, the server shutdown for one minute, will bring great losses to the company. In addition, using this method, the system has to be restarted each time after the partition table has been modified. The adoption of the new technology LVM (logical volume manager) can help us solve these problems.

IV. History of LVM

Of course, LVM is not the latest technology. As early as in the era of UNIX operating system, VM can be seen on HP and IBM AIX. As the flagship product of IBM, AIX supports dynamic logical partitioning (DVM) a long time ago. Of course, its DVM design is quite powerful. Since then, in AIX 5L, it reconstructs the UNIX kernel and adds functions such as logical volume management (LVM) and log file system (JFS). To make its AIX more powerful. In a variety of commercial UNIX systems, such as AIX, HP-UNIX, Tru64 UNIX and other systems, logical volume management has been widely used and has become a de facto standard.

The functionality of LVM is supported in the developing version of the 2.3kernel. In January 2001, the Linux2.4.0 kernel was released and formally supported logical volume management, which made the new Linux kernel more suitable for server applications. Previous versions of Linux had to patch the kernel to implement LVM functionality. Now, we see that LVM is supported at the kernel level since Redhat Linux 9.0. Therefore, we can use LVM to help us manage disks more efficiently. It should be noted that there are two versions of LVM, LVM 1 and LVM 2, and there are some differences in related tools and settings. This article will use the environment of LVM 2 to introduce the use. The LVM 1 command can only work on the 2.4 kernel. You cannot use the LVM 1 command when running the 2. 6 kernel. For more information about LVM 2, see / usr/share/doc/lvm2*/WHATS_NEW. A complete LVM 2 command is installed under / usr/sbin/. In a startup environment where / usr/ is invalid, each command needs to be preceded by / sbin/lvm.static (for example, / sbin/lvm.static vgchange-ay). In an environment where / usr is valid, it is no longer necessary to precede each command with lvm (for example, / usr/sbin/lvm vgchage-ay becomes / usr/sbin/vgchange-ay). New LVM2 commands (for example, / usr/sbin/vgchange-ay and / sbin/lvm.static vgchange-ay) will detect whether you are running the2.4 kernel. If so, it invokes the old LVM 1 command.

V. LVM structure and classification

LVM is the abbreviation of logical disk Volume Management (Logical Volume Manager). It is a mechanism for managing disk partitions in Linux environment. It aggregates multiple physical partitions into a single volume group (Volume Group), and these physical volumes can be of different sizes or even types (such as SCSI, SATA disks). The volume group is like a large hard disk, and then a piece of logical volume (Logical Volume) is split from it, and a file system is further created on the logical volume group, as shown in the following figure.

The biggest advantage of LVM is that it considers the characteristics of physical disks differently, with a layer of "insulation", or, in modern terms, the service layer, which provides disk space services. Without this middle layer, there must be a limit on the size of the physical disk, which is an insurmountable fact. The partition on this disk must be restricted by physical conditions, and it is very inconvenient to store data. In an era when a computer has only one 100MB disk, this is not a big problem. At the beginning of LVM design, we realized the nature of human laziness and achieved a good balance in terms of performance, manageability, compatibility, functional support, and so on.

LVM supports logical volumes in two modes, as shown in the following figure. One is concatenation mode (Concatenation) and the other is strip mode (Striping). The system defaults to concatenation mode. What's the difference between the two models? For example, there are two 30GB hard drives / dev/hdb and / dev/hdc with IDE interface, which together form a volume group vg1, on which a logical volume lv1 of 40GB is created. If the logical volume is in concatenated mode, the data will be stored sequentially on two hard drives, and the other hard drive will not be used until one hard disk is full. If the logical volume is in stripe mode, the data will be divided into fixed-size strips and then spread over two hard drives. This means that you have more effective disk bandwidth and the speed of reading and writing data will be greatly improved. Although the stripe mode brings us high performance, it also brings high risk. If any disk goes down, all logical volumes will be lost, and the result will be catastrophic. However, LVM technology has long been on guard against this kind of disaster. The implementation of LVM can be implemented not only on disk partitions, but also on RAID volumes. Both hard disk RAID and software RAID,LVM are well supported.

VI. Snapshots (snapshot) features of LVM

LVM provides a very good feature: snapshots (snapshots). It allows administrators to build a block device: the device is an exact copy of a logical volume frozen at some point. This feature is usually used in batch processes such as backups where logical volumes need to be processed, but the system cannot be stopped. When the operation is complete, the snapshots (snapshot) device can be removed. This feature requires that logical volumes be in a compatible state when a snapshots (snapshot) device is established. With LVM, we can take an instant snapshot of LV, then mount it, and then back it up. Please note that the snapshot is not permanent. If you remove the LVM or restart it, they are lost and need to be recreated. The following figure is a schematic diagram of LVM snapshots.

There are two ways to take a snapshot: one is read-only, the other is read-write. Read-only snapshots look good if you just copy data, but read-write snapshots have several advantages. First of all, there is no additional processing of the log file system-you can simply implement log recovery on the snapshot. Read-only snapshots must ensure that the file system is synchronized with the device before starting the snapshot, so log reproduction is required.

VII. Terms commonly used in LVM

1. Physical storage media (The Physical media)

In this case, the storage device of the system: the hard disk or the partition on the hard disk, such as / dev/hda1, / dev/sda, etc., is the storage unit at the lowest level of the storage system.

2. Physical Volume (Physical Volume,PV)

A physical volume refers to a hard disk partition or a device that logically has the same function as a disk partition (such as RAID). It is the basic storage logic block of LVM, but compared with the basic physical storage media (such as partitions, disks, etc.), it contains management parameters related to LVM.

3. Volume Group (Volume Group,VG)

The highest abstraction layer in LVM, consisting of one or more physical volumes. There can be only one volume group or multiple volume groups in a logical volume management system. LVM volume groups are similar to physical hard disks in non-LVM systems and are made up of physical volumes.

4. Logical Volume (Logical Volume,LV)

LVM's logical volumes are similar to hard disk partitions in non-LVM systems, and file systems can be built on top of logical volumes (such as / home or / usr, etc.). Multiple logical volumes in the system can belong to the same volume group or to different volume groups.

5.PE (Physical Extent,PE)

Each physical volume is divided into equal-sized basic units called PE (Physical Extents). The physical area is the smallest storage unit in the physical volume that can be allocated, and the size of the physical area can be specified when the physical volume is established according to the implementation. Once the physical area size is determined, it cannot be changed, and the physical area size needs to be the same for all physical volumes in the same volume group. The size of PE is configurable and defaults to 4MB.

6.LE (Logical Extent,LE)

Logical volumes are also divided into basic addressable units called LE (LogicalExtents). In the same volume group, the size of LE and PE are the same and correspond one to one.

7.VGDA (volume group descriptor area)

Just as non-LVM systems store metadata containing partition information in the partition table at the beginning of the partition, the metadata related to logical volumes and volume groups is stored in the VGDA at the beginning of the physical volume. VGDA includes the following: PV descriptors, VG descriptors, LV descriptors, and some PE descriptors. When the system starts LVM, activate VG and load the VGDA into memory to identify the actual physical storage location of the LV. When the system performs an iCandle O operation, it accesses the actual physical location according to the mapping mechanism established by VGDA.

VII. Install LVM

First determine if the lvm tool is installed on the system:

# rpm-qa | greplvm

Lvm-1.0.3-4

If the command result input is similar to the above example, then the system has installed the LVM management tool; if the command has no output, it means that the LVM management tool is not installed, then you need to download it from the network or install the LVMrpm toolkit from the CD.

After installing the RPM package for LVM, you also need to configure the kernel to support LVM to use LVM. The default kernel of RedHat supports LVM. If you need to recompile the kernel, you need to enter the Multi-deviceSupport (RAIDandLVM) submenu when configuring the kernel, and select the following two options:

[*] Multipledevicesdriversupport (RAIDandLVM)

Logicalvolumemanager (LVM) Support

Then recompile the kernel to add LVM support to the new kernel.

In order to use LVM, make sure that LVM is activated when the system starts. Fortunately, in later versions of RedHat7.0, the system startup script already has support for activating LVM, with the following in / etc/rc.d/rc.sysinit:

# LVMinitialization

If [- e/proc/lvm-a-x/sbin/vgchange-a-f/etc/lvmtab]; then

Action$ "SettingupLogicalVolumeManagement:" / sbin/vgscan&&/sbin

/ vgchange-ayfi

The key is two commands, the vgscan command scans all disks to get volume group information, and creates file volume group data files / etc/lvmtab and / etc/lvmtab.d/*;vgchange-ay commands to activate all volume groups of the system.

Eight. LVM command

LVM Command Table 1

LVM

Command

Description

Pv

Pvcreate

Build the physical partition as PV

Pvscan

Search for any disk with PV in the current system

Pvdisplay

Displays the PV status on the current system

Pvremove

Delete the PV attribute so that the partition does not have the PV attribute

Partprobe

This command allows the core to read the latest partition table immediately without having to reboot

Vg

Vgcreate

Establish a VG. It has a lot of parameters. I'll introduce it later.

Vgscan

Search for the existence of VG on the system

Vgdisplay

Displays the VG status on the current system

Vgextend

Add extra PV to the VG

Vgreduce

Delete PV within VG

Vgchange

Sets whether VG starts (active)

Vgremove

Delete a VG

Lv

Lvcreate

Establish LV

Lvscan

Query LV on the system

Lvdisplay

Displays the LV status on the system

Lvextend

Increase capacity in LV

Lvreduce

Reduce capacity in LV

Lvremove

Delete a LV

Lvresize

Resize the capacity of LV

LVM Command Table 2

Task

PV

VG

LV

Search (scan)

Pvscan

Vgscan

Lvscan

Establish (create)

Pvcreate

Vgcreate

Lvcreate

List (display)

Pvdisplay

Vgdisplay

Lvdisplay

Add (extend)

Vgextend

Lvextend

Decrease (reduce)

Vgreduce

Lvreduce

Delete (remove)

Pvremove

Vgremove

Lvremove

Change capacity (resize)

Vgresize

IX. LVM configuration practice

There are two ways to realize LVM in Linux: one is to use Disk Druid program under the graphical interface when installing Linux, the other is to use LVM command to realize under the character interface, the following process is based on the latter (character interface).

1. Prepare the physical partition

First, we need to select the physical storage for LVM. These are usually standard partitions, but they can also be Linux Software RAID volumes that have been created. Here, I use the fdisk command to partition sdb and sdc into sdb1 and sdc1, and specify the partition as 8e type (Linux LVM) through the t of fdisk, as shown in the following figure.

2. Create a physical volume PV

Volumes are created on a physical partition of a disk or on a device (such as RAID) that has the same function as a disk partition. It only delineates a special area in the physical partition to record the management parameters related to LVM.

The command to create a physical volume is pvcreate:

# pvcreate / dev/sdb1

Physical volume "/ dev/sdb1" successfully created

# pvcreate / dev/sdc1

Physical volume "/ dev/sdc1" successfully created

The above command initializes / dev/sdc1 and / dve/sdd1 into physical volumes, respectively. Use the physical volume display command pvdisplay to view the physical volumes as follows:

# pvdisplay

-NEW Physical volume

PV Name / dev/sdb1

VG Name

PV Size 36.00 GB

Allocatable NO

PE Size (KByte) 0

Total PE 0

Free PE 0

Allocated PE 0

PV UUID QDmnUd-tuvH-U4Hn-n5Ry-zGRT-OlyK-67Dxbb

-NEW Physical volume

PV Name / dev/sdc1

VG Name

PV Size 36.00 GB

Allocatable NO

PE Size (KByte) 0

Total PE 0

Free PE 0

Allocated PE 0

PV UUID NDBf68-6qrD-9hE6-Rotv-RdxL-Azvv-7Nlcos

3. Create a volume group VG

A volume group is a combination of one or more physical volumes. Volume groups combine multiple physical volumes to form a manageable unit that is similar to a physical hard disk in a non-LVM system.

The command to create a volume group is vgcreate, which is used to create a volume group called "lvmdisk" that contains / dev/sdb1 and / dev/sdc1 two physical volumes.

# vgcreate lvmdisk / dev/sdb1 / dev/sdc1

Volume group "lvmdisk" successfully created

Use the volume group view command vgdisplay to display the volume group:

# vgdisplay

-Volume group

VG Name lvmdisk

System ID

Format lvm2

Metadata Areas 2

Metadata Sequence No 1

VG Access read/write

VG Status resizable

MAX LV 0

Cur LV 0

Open LV 0

Max PV 0

Cur PV 2

Act PV 2

VG Size 71.98GB

PE Size 4.00MB

Total PE 18428

Alloc PE / Size 0 / 0

Free PE / Size 18428 / 71.98GB

VG UUID SARfuj-wAUI-od81-VWAc-Alnt-aaFN-JWaPVf

When multiple physical volumes are combined into a volume group, LVM does similar formatting work on all physical volumes, cutting each physical volume into a piece of space, which is called PE (Physical Extent), and its default size is 4MB.

Due to kernel restrictions, a logical volume can only contain a maximum of 65536 PE, so the size of a PE determines the maximum capacity of a logical volume. The PE of 4MB determines that the maximum capacity of a single logical volume is 256GB. If you want to use a logical volume greater than 256GB, you need to specify a larger PE when creating a volume group.

For example, if you want to create a volume group using 64MB's PE, so that the maximum capacity of logical volumes can be 4TB, the command is as follows:

# vgcreate-64mb lvmdisk / dev/sdb1 / dev/sdc1

4. Create a logical volume LV

A logical volume (Logical Volumes,LV) is a logical region divided in a volume group, similar to a hard disk partition in a non-LVM system.

The command to create the logical volume is lvcreate. With the following command, we create a logical volume named pldy1 on the volume group lvmdisk, the size is 15GB, and its device entry is / dev/lvmdisk/pldy1.

# lvcreate-L 15G-n pldy1 lvmdisk

Logical volume "pldy1" create

You can also use the-l parameter to set the logical partition size by specifying the number of PE.

For example, if you want to create a logical volume that uses all space, you need to check the total number of PE in the volume group first. The total number of PE in the current volume group is found to be 18428 through the above vgdisplay command, as shown in the command:

# lvcreate-l 18428-n pldy1 lvmdisk

"when the logical volume is successfully created, you can use the lvmdisplay command to view the logical volume:"

# lvdisplay

-Logical voume

LV Name / dev/lvmdisk/pldy1

VG Name lvmdisk

LV UUID FQcnm3-BMyq-NkJz-hykw-9xg1-Qy8d-8UeGCN

LV Write Access read/write

LV Status available

# open 0

LV Size 15.00GB

Current LE 3840

Segments 1

Allocation inherit

Read ahead sectors 0

Block device 253:0

Like volume groups, logical volumes are divided into chunks of space called LE (Logical Extents). In the same volume group, the size of LE and PE are the same and correspond one to one.

5. Create a file system

Create an ext3 file system on the logical volume:

# mkfs-t ext3 / dev/lvmdisk/pldy1

Once you have created the file system, you can load and use:

# mkdir / opt/Oracle

# mount / dev/lvmdisk/pldy1 / opt/Oracle

In order to load the file system automatically when the system starts, you also need to add something to / etc/fstab:

# dev/lvmdisk/pldy1 / opt/Oracle ext3 defaults 1 2

6. Manage LVM

The biggest advantage of LVM is that it can resize the partition dynamically without having to restart the machine. Let's try it! Continuing with the above example, let's assume that the logical volume / dev/lvmdisk/pldy1 does not have enough space and needs to increase its size, which we discuss in two cases.

(1) check whether there is any space left in the volume group

The vgdiskplay command allows you to check the current volume group space usage:

# vgdisplay

-Volume group

VG Name lvmdisk

System ID

Format lvm2

Metadata Areas 2

Metadata Sequence No 2

VG Access read/write

VG Status resizable

MAX LV 0

Cur LV 1

Open LV 0

Max PV 0

Cur PV 2

Act PV 2

VG Size 71.98GB

PE Size 4.00MB

Total PE 18428

Alloc PE / Size 3840 / 15.00GB

Free PE / Size 14588 / 56.98GB

VG UUID SARfuj-wAUI-od81-VWAc-Alnt-aaFN-JWaPVf

Determine the current volume group remaining space 56.98GB, the remaining number of PE is 14588. Here all the remaining space is added to the logical volume / dev/lvmdisk/pldy1.

# lvextend-l + 14588 / dev/lvmdisk/pldy1

Extending logical volume pldy1 to 56.98 GB

Logical volume pldy1 successfully resized

The above command uses the-lumped 14588 argument, which means to add 14588 PE to the specified logical volume. If you don't use all the space, you can use other forms of lvextend commands.

For example, add logical volume / dev/lvmdisk/pldy1 to 5GB so that its space reaches 20GB, which can be written as: "# lvextend-L20G / dev/lvmdisk/pldy1" or "# lvextend-L20G / dev/lvmdisk/pldy1". After you increase the logical volume capacity, you will modify the size of the file system through the ext2online command.

# ext2online / opt/Oracle/

After the conversion, let's take a look at the current state of the file system:

# df-lh

Filesystem 1k-blocks Used Available Use% mounted on

/ dev/sba1 7.4G 1.8G 5.3G 25% /

None 135m 0 135m 0% / dev/shm

/ dev/mapper/lvmdisk-pldy1 71G 81m 68G 1% / opt/Oracle

(2) insufficient space in the volume group

When there is not enough space in the volume group to expand the size of the logical volume, you need to increase the capacity of the volume group, and the only way to increase the capacity of the volume group is to add new physical volumes to the volume group.

First of all, add a new hard disk (36GB SCSI hard disk), and complete its partition, create physical volumes and other work. Next, use the vgextend command to add the new physical volume (/ dev/sdd1) to the volume group.

The command to extend the volume group is as follows:

# vgextend lvmdisk / dev/sdd1

Volume group "lvmdisk" successfully extended

Use the vgdisplay command to view the volume group lvmdisk:

# vgdisplay

-Volume group

VG Name lvmdisk

System ID

Format lvm2

Metadata Areas 3

Metadata Sequence No 3

VG Access read/write

VG Status resizable

MAX LV 0

Cur LV 1

Open LV 0

Max PV 0

Cur PV 3

Act PV 3

VG Size 107.97GB

PE Size 4.00MB

Total PE 27640

Alloc PE / Size 3840 / 15.00GB

Free PE / Size 23800 / 92.97GB

VG UUID 18Rfuj-udUI-od81-VWcT-Alnt-aaFN-JWaPVf

After completing the expansion of the volume group, we can complete the expansion of the logical volume according to the method of the first case, and finally realize the dynamic adjustment of the partition.

(3) how to use the LVM tool under the rescue mode

General Linux distributions use LVM2 commands, for example, vgscan and vgcreate are actually linked to lvm.static. For example:

# ls-l / sbin/vgscan

Lrwxrwxrwx 1 root root 10 Oct 8 16:06 / sbin/vgscan-> lvm.static

In rescue mode, these links are not automatically established and all these commands cannot be used. To execute the LVM command, use the following command:

Lvm

For example:

Lvm vgscan

(4) use dm-crypt to create an encrypted logical volume on LVM (version LVM2)

Use the command lvcreate to create a logical volume named CRYPTO (LV):

# lvcreate-n CRYPTO-lumped 100m DATA

Logical volume "CRYPTO" created

Use the cryptsetup command to set the logical volume CRYPTO as an encrypted block device:

# cryptsetup create CRYPTO / dev/DATA/CRYPTO

Enter passphrase:xxxxxx

Use the command cryptsetup to check the status:

# cryptsetup status DMCRYPT

/ dev/mapper/DMCRYPT is active:

Cipher:aes-plain

Keysize:256 bits

Devce:/dev/dm-6

Offset: 0 sectors

Size:204800 sectors

Use mke2fs to create a file system on DMCRYPT:

# mke2fs / dev/mapper/DMCRYPT

Mke2fs 1.35 (23-Feb-2008)

Max_blocks 104857600, rsv_groups=128000, rsv_gdb=256

...

...

Mount the file system and create files on the file system:

# mkdir / mnt/crypt

# mount / dev/mapper/DMCRYPT / mnt/crypt

# cd / mnt/crypt

Unmount the file system and delete the mapped encrypted block device. In this case, it is impossible to get the data without a password:

# umount / mnt/crypt

# cryptsetup remove DMCRYPT

The following can also be done after reboot.

Re-encrypt the block device and prompt for a password:

# cryptsetup create DMCRYPT / dev/DATA/CRYPTO

Enter passphrase:

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.