Shulou(Shulou.com)06/01 Report--

Most people do not understand the knowledge of this article "how to use VDO to optimize storage space in Centos8", so the editor summarizes the following content, detailed content, clear steps, and has a certain reference value. I hope you can get something after reading this article. Let's take a look at this "how to optimize storage space using VDO in Centos8" article.

Virtual data Optimizer (VDO) is a block virtualization technology that provides transparent data deduplication capabilities. By eliminating redundant data blocks, VDO can greatly reduce the disk capacity actually used.

System environment

Centos8

Install VDO

The following command installs vdo and related dependencies, and restarts the system after the installation is complete:

[root@localhost ~] # yum-y install vdo [root@localhost ~] # reboot

Create a VDO device

Make sure there is a free disk or partition available for VDO. Although VDO volumes can be created on top of LVM logical volumes, boot order issues occur when the system is rebooted. So create a vdo volume on the bare disk and then create a LVM logical volume on it.

Below, use a disk of 20GB, / dev/sda as vdo volume, and create vdo volume below:

[root@localhost] # vdo create-name=vdolvm-device / dev/sda-vdoLogicalSize 60G-writePolicy asyncCreating VDO vdolvm The VDO volume can address 16 GB in 8 data slabs, each 2 GB. It can grow to address at most 16 TB of physical storage in 8192 slabs. If a larger maximum size might be needed, use bigger slabs.Starting VDO vdolvmStarting compression on VDO vdolvmVDO instance 0 volume is ready at / dev/mapper/vdolvm

Let's break down the command to see the options used:

-device-specify which hard disk to create the vdo volume on-vdoLogicalSize-here we specify a capacity size of 60G, which is much larger than our actual disk 20g. Suppose we will get at least a 3:1 reduction from deduplication, which is quite conservative for most data, but if your data does not have a lot of duplication, the ratio should be different. Log files and other plain text files are usually well deduplicated and can get a deduplication rate of 10:1 or more. But binaries, such as video, audio, or compressed packages, will be much lower than 3:1 and, in some cases, 1:1. Vdo volumes are not recommended in this case. -writePolicy-vdo has three write strategies: sync: writes to VDO volumes are confirmed only after the data has been written to the physical device. Async: confirmed after the data has been written to the cache. If the cache is not flushed before a device failure or power outage, data loss may result. Auto: in this default mode, VDO checks the storage device and determines if it supports refresh. If so, VDO uses asynchronous mode. If not, it uses synchronous mode. View information related to the new VDO volume

As we saw in the output of the previous step, VDO created a new dm device called / dev/mapper/vdolvm. This is the device we will use when we create the lvm volume group.

[root@localhost ~] # ll / dev/mapper/vdolvmlrwxrwxrwx 1 root root 7 Mar 4 13:31 / dev/mapper/vdolvm->.. / dm-2

Let's see what kind of information you can get about vdo volumes using vdostats:

[root@localhost] # vdostats-- huDevice Size Used Available Use% Space saving%/dev/mapper/vdolvm 20.0G 4.0G 16.0G 20% Nash A

Since we have not written any data to the volume, the Space saving% field is NCMA. When we write some data later, we will see more useful information here.

As you can see in the picture above, we haven't written any data yet, but there is already 20% of the 4GB space in use! This is because the Universal Deduplication Index has been written to disk. This is basically a database for recording slab fingerprints and their location. This is what makes deduplication possible.

Set up VDO volumes as normal disk devices

Now that we have created the VDO device, we can partition and format it, or create a lvm logical volume on top of this vod volume. Let's create a logical volume:

# create physical volume [root@localhost ~] # pvcreate / dev/mapper/vdolvm Physical volume "/ dev/mapper/vdolvm" successfully created.# create volume group vdo_ vg [root @ localhost ~] # vgcreate vdo_vg / dev/mapper/vdolvm Volume group "vdo_vg" successfully created# View vdo_vg volume group information [root@localhost ~] # vgdisplay vdo_vg-- Volume group-- VG Name vdo_vg System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 1 VG Access read/write VG Status resizable MAX LV 0 Cur LV 0 Open LV 0 Max PV 0 Cur PV 1 Act PV 1 VG Size

As you can see from the above, lvm thinks our basic disk is 120GB, even though we know that it only has 40GB. Because LVM does not know the size of the VDO back-end disk.

Now, let's create a few logical volumes:

[root@localhost ~] # lvcreate-n vdo_lv01-L 15G vdo_ VG [root @ localhost ~] # lvcreate-L 15G-n vdo_lv02 vdo_ VG [root @ localhost ~] # lvcreate-L 15G-n vdo_lv03 vdo_vg# View the created logical volume [root@localhost ~] # lvs-o + devices create mount point and mount file system

Typically, after a file system is created, it runs Trim operations on the device. This is not an ideal choice when using VDO because disk capacity is allocated on demand. Therefore, we will tell mkfs not to discard blocks during file system creation. For XFS, use the-K option. For EXT4, use-E nodiscard. The following uses a for loop to format the three logical volumes into the XFS file system:

[root@localhost ~] # for i in `seq 1 3`; do mkfs.xfs-K / dev/vdo_vg/vdo_lv0 $I Donemeta-data=/dev/vdo_vg/vdo_lv01 isize=512 agcount=4, agsize=983040 blks = sectsz=4096 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=1, rmapbt=0 = reflink=1data = bsize=4096 blocks=3932160 Imaxpct=25 = sunit=0 swidth=0 blksnaming = version 2 bsize=4096 ascii-ci=0, ftype=1log = internal log bsize=4096 blocks=2560, version=2 = sectsz=4096 sunit=1 blks, lazy-count=1realtime = none extsz=4096 blocks=0, rtextents=0meta-data=/dev/vdo_vg/vdo_lv02 isize=512 agcount=4, agsize=983040 blks = sectsz=4096 attr=2 Projid32bit=1 = crc=1 finobt=1, sparse=1, rmapbt=0 = reflink=1data = bsize=4096 blocks=3932160, imaxpct=25 = sunit=0 swidth=0 blksnaming = version 2 bsize=4096 ascii-ci=0, ftype=1log = internal log bsize=4096 blocks=2560, version=2 = sectsz=4096 sunit=1 blks Lazy-count=1realtime = none extsz=4096 blocks=0, rtextents=0meta-data=/dev/vdo_vg/vdo_lv03 isize=512 agcount=4, agsize=983040 blks = sectsz=4096 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=1, rmapbt=0 = reflink=1data = bsize=4096 blocks=3932160 Imaxpct=25 = sunit=0 swidth=0 blksnaming = version 2 bsize=4096 ascii-ci=0, ftype=1log = internal log bsize=4096 blocks=2560, version=2 = sectsz=4096 sunit=1 blks, lazy-count=1realtime = none extsz=4096 blocks=0, rtextents=0

When we mount the new file system to the mount point, we tell XFS to discard the block, because this will greatly speed up the file deletion.

[root@localhost ~] # mkdir-p / data/ {01.03} [root@localhost ~] # for i in `seq 1 3`; do mount-o discard / dev/vdo_vg/vdo_lv0 $I / data/0$ I; done

Now that we have written a small amount of data to the device, we can check the VDO volume again to see if the situation has changed.

[root@localhost ~] # vdostats-- huDevice Size Used Available Use% Space saving%/dev/mapper/vdolvm 20.0G 4.0G 16.0G 20% 86%

Set to boot

The following three logical volumes are set to boot, and you need to add x-systemd.device-timeout=0 and x-systemd.requires=vdo.service to the fstab file.

Use blkid to view the UUID of these three logical volumes.

Using the UUID obtained above, add it to the / etc/fstab file:

[root@localhost systemd] # vim / etc/fstabUUID= "bd2c1c61-4656-4065-b5a0-3ca53ef0f949" / data/01 xfs defaults,x-systemd.device-timeout=0,x-systemd.requires=vdo.service 00UUID = "1e53579b-f1da-4f77-80e6-d61a40515525" / data/02 xfs defaults,x-systemd.device-timeout=0,x-systemd.requires=vdo.service 00UUID = "d41bf7e2-bf75-4db7-b323-a923375f6a6e" / data/03 xfs defaults,x-systemd.device-timeout=0 X-systemd.requires=vdo.service 0 0 is the content of the article on "how to use VDO to optimize storage space in Centos8". I believe we all have a certain understanding. I hope the content shared by the editor will be helpful to you. If you want to know more about the relevant knowledge, please pay attention to the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.