Shulou(Shulou.com)05/31 Report--

This article introduces the five Docker storage driver principles and application scenarios, the content is very detailed, interested friends can refer to, I hope it can be helpful to you.

Docker initially adopted AUFS as the file system, but also benefited from the concept of AUFS layering, so that multiple Container can share the same image. However, because AUFS is not incorporated into the Linux kernel and only supports Ubuntu, storage drivers are introduced in Docker version 0.7. at present, Docker supports five storage drivers: AUFS, Btrfs, Device mapper, OverlayFS and ZFS. As mentioned on the Docker official website, no single driver is suitable for all application scenarios. It is necessary to select the appropriate storage driver according to different scenarios in order to effectively improve the performance of Docker. How to choose a suitable storage driver, it is necessary to understand the principle of storage driver in order to make a better judgment. This paper introduces the detailed explanation of the five storage driver principles of Docker and the comparison of application scenarios and IO performance testing. Before we talk about the principle, let's talk about the two techniques of copying when writing and assigning when writing.

1. Copy when writing the principle description (CoW)

The technology used by all drivers-copy on write (CoW). CoW is copy-on-write, which means to copy only when you need to write. This is a modification scenario for existing files. For example, if you start multiple Container based on one image, if you allocate a file system like image for each Container, it will take up a lot of disk space. The CoW technology allows all containers to share the image file system, and all the data is read from the image. Only when you want to write to the file, copy the file to be written from the image to your own file system for modification. So no matter how many containers share the same image, the write operation is done on the copy copied from image to their own file system, and the source file of image will not be modified, and multiple containers operate on the same file, which will generate a copy in the file system of each container, and each container modifies its own copy, isolated from each other, and does not affect each other. Using CoW can effectively improve the utilization of the disk.

Time allocation (allocate-on-demand)

On the other hand, write-time allocation is used in scenarios where there is no such file, and space is allocated only when a new file is to be written, which can improve the utilization of storage resources. For example, starting a container does not pre-allocate some disk space for the container, but allocates new space as needed when a new file is written.

AUFS

AUFS (AnotherUnionFS) is a UnionFS that is a file-level storage driver. AUFS can transparently overwrite the layered file system of one or more existing file systems, merging multiple layers into a single-layer representation of the file system. To put it simply, it supports mounting different directories to a file system under the same virtual file system. This file system can overlay and modify files layer by layer. No matter how many layers are read-only, only the top file system is writable. When you need to modify a file, AUFS creates a copy of the file, uses CoW to copy the file from the read-only layer to the writeable layer for modification, and the result is also saved in the writable layer. In Docker, the bottom read-only layer is image, and the writable layer is Container.

Overlay

Overlay is a kind of Union FS supported by the Linux kernel after 3.18. Unlike the multi-layer of AUFS, Overlay has only two layers: a upper file system and a lower file system, which represent the mirror layer and container layer of Docker, respectively. When you need to modify a file, use CoW to copy the file from the read-only lower to the writable upper for modification, and the results are also saved in the upper layer. In Docker, the bottom read-only layer is image, and the writable layer is Container.

Device mapper

Device mapper is supported by Linux kernel 2.6.9 and provides a mapping framework mechanism from logical devices to physical devices. Under this mechanism, users can easily make management strategies for storage resources according to their own needs. The AUFS and OverlayFS mentioned earlier are file-level storage, while Device mapper is block-level storage, and all operations operate directly on blocks, not files. The Device mapper driver first creates a resource pool on the block device, and then creates a basic device with a file system on the resource pool. All mirrors are snapshots of this basic device, and containers are snapshots of mirrors. So in the container, you can see that the file system is a snapshot of the file system of the basic devices on the resource pool, and there is no space allocated for the container. When a new file is to be written, a new block is allocated and data is written to it in the container's mirror, which is called time allocation. When you want to modify an existing file, use CoW to allocate block space for the container snapshot, copy the modified data to the new block in the container snapshot and then modify it. By default, the Device mapper driver creates a 100g file containing images and containers. Each container is limited to a 10G volume and can be configured and adjusted on its own.

Btrfs

Btrfs is called the next generation write-time replication file system, which is incorporated into the Linux kernel and is also file-level storage, but it can directly manipulate the underlying devices like Device mapper. Btrfs configures part of the file system as a complete subfile system, called subvolume. With subvolume, a large file system can be divided into multiple sub-file systems that share the underlying device space and allocate it from the underlying device when disk space is needed, just as an application calls malloc () to allocate memory. To make flexible use of device space, Btrfs divides disk space into multiple chunk. Each chunk can use a different disk space allocation strategy. For example, some chunk only store metadata, and some chunk store only data. This model has many advantages, such as Btrfs support for dynamically adding devices. After you add a new disk to the system, you can use the Btrfs command to add the device to the file system. Btrfs treats a large file system as a resource pool and configures it into multiple complete sub-file systems. It can also add new sub-file systems to the resource pool, while the basic image is a snapshot of the sub-file system. Each sub-image and container has its own snapshot, and these snapshots are all subvolume snapshots.

When a new file is written, a new data block is assigned to it in the snapshot of the container, the file is written in this space, and this is called time allocation. When you want to modify an existing file, use CoW replication to assign a new original data and snapshot, change the data in this newly allocated space, and then update the relevant data structure to point to the new sub-file system and snapshot. The original data and snapshot are overwritten without a pointer.

ZFS

ZFS file system is a revolutionary and brand-new file system, which fundamentally changes the way of file system management. ZFS completely abandons "volume management" and no longer creates virtual volumes, but manages all devices in one storage pool and uses the concept of "storage pool" to manage physical storage space. In the past, file systems were built on physical devices. To manage these physical devices and provide redundancy for data, the concept of "volume management" provides a single-device image. ZFS is created on top of a virtual storage pool called "zpools". Each storage pool consists of several virtual devices (virtual devices,vdevs). These virtual devices can be raw disks, a RAID1 mirrored device, or a non-standard RAID-rated multi-disk group. The file system on zpool can then use the total storage capacity of these virtual devices.

Let's take a look at the use of ZFS in Docker. First, assign a ZFS file system from zpool to the base layer of the mirror, and the other mirror layers are clones of the ZFS file system snapshot, which is read-only, while the clone is writable. When the container starts, it generates a writeable layer at the top of the image.

When writing a new file, using on-demand allocation, a new data is quickly generated from the zpool, the new data is written to the block, and the new space is stored in the container (a clone of the ZFS).

When you want to modify an existing file, use copy on write, allocate a new space, and copy the original data to the new space to complete the modification.

2. Storage-driven comparison and adaptation scenario AUFS VS Overlay

Both AUFS and Overlay are federated file systems, but AUFS has multiple tiers, while Overlay has only two tiers, so AUSF may be slower if the file is large and has lower layers when doing a write copy operation. And Overlay is incorporated into linux kernel mainline,AUFS, so it may be faster than AUFS. But Overlay is too young to be used carefully in production. As the first storage driver of docker, AUFS has a long history, relatively stable, and has been practiced in a large number of production, with strong community support. Currently, open source DC/OS specifies to use Overlay.

Overlay VS Device mapper

Overlay is file-level storage, and Device mapper is block-level storage. When the file is very large and the modified content is very small, Overlay will copy the entire file regardless of the size of the modified content. It takes more time to modify and display large files than small files, while block-level files, whether large or small, only copy blocks that need to be modified, not the whole file. In this scenario, device mapper is obviously faster. Because the block level is a direct access logic disk, it is suitable for IO-intensive scenarios. For complex scenarios with large concurrency but few IO, the performance of Overlay is relatively stronger.

Device mapper VS Btrfs Driver VS ZFS

Both Device mapper and Btrfs operate directly on blocks and do not support shared storage, which means that when multiple containers read the same file, multiple replicas are needed, so this storage driver is not suitable for use on PaaS platforms with high-density containers. And in the case of many containers starting and stopping, it may lead to disk overflow, causing the host to fail to work. Device mapper is not recommended for production use. Btrfs can be very efficient in docker build.

ZFS was originally designed for Salaris servers with a large amount of memory, which will have an impact on memory when used, so it is suitable for environments with large memory. ZFS's COW aggravates the fragmentation problem. If some of the large files generated by sequential writes are changed randomly later, the physical address of the file on the hard disk will no longer be contiguous, and the performance of sequential reading will become poor in the future. ZFS supports multiple containers to share a cache block, which is suitable for PaaS and high-density user scenarios.

Third, the performance comparison of IO

Testing tool: IOzone (a file system benchmark tool that can test the read and write performance of file systems in different operating systems)

Test scenario: order from 4K to 1G files and random IO performance

Test method: start the container based on different storage drivers, install IOzone in the container, and execute the command:

. / iozone-a-n 4k-g 1g-I 0-I 1-I 2-f / root/test.rar-Rb. / iozone.xls definition and interpretation of test items

Write: test the performance of writing to a new file.

Re-write: tests the performance of writing to an existing file.

Read: test the performance of reading an existing file.

Re-Read: test the performance of reading a recently read file.

Random Read: test the performance of reading random offsets in a file.

Random Write: test the performance of writing random offsets in a file.

From the above performance data, you can see:

AUFS performs worse than Overlay in reading, but better than Overlay in writing.

The read-write performance of device mapper files above 512m is very poor, but the read-write performance of files below 512m is better.

The read and write performance of files above 512m of btrfs is very good, but the read and write performance of files below 512m is worse than that of other storage drivers.

The overall read and write performance of ZFS is worse than that of other storage drivers. Some data are simply tested, and the principle of the tested data needs to be further analyzed.

On the Docker five storage driver principles and application scenarios are shared here, I hope the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.