Shulou(Shulou.com)05/31 Report--

This article mainly explains "how to use Docker AUFS". Friends who are interested might as well take a look. The method introduced in this paper is simple, fast and practical. Now let the editor take you to learn how to use Docker AUFS.

AUFS is a kind of Union File System, the so-called UnionFS is to merge the directories of different physical locations mount into the same directory. One of the main applications of UnionFS is to combine a CD/DVD and a hard disk directory with the federated mount, and then you can modify the files on the read-only CD/DVD (of course, the modified files are stored in the directory on the hard drive).

AUFS, also known as Another UnionFS, later called Alternative UnionFS, may later feel that it is not domineering enough, called Advance UnionFS. Developed by Junjiro Okajima in 2006, AUFS completely rewrote the early UnionFS 1.x for reliability and performance, and introduced some new features, such as load balancing for writable branches. AUFS is fully compatible with UnionFS in use, and it is much better than the previous UnionFS in terms of stability and performance. Later, UnionFS 2.x began to copy the functions of AUFS. But he didn't go into the Linux trunk because Linus wouldn't let him, basically because there was a lot of code, and it was poorly written (compared to union mount with only 3000 lines and UnionFS with 10000 lines, and other VFS,AUFS with an average of only about 6000 lines of code, there were 30000 lines of code), so Linus kept improving the quality of the code, submitting it, and being rejected by Linus. To this day, AUFS still can't get into the Linux backbone (today you can see that AUFS's code is actually better, N times better than OpenSSL, either Linus has very high code quality requirements, or Linus just doesn't like AUFS).

However, fortunately, there are many distributions that use AUFS, such as Ubuntu 10.04 Gentoo Live CD Debian 6.0, Gentoo Live CD supports AUFS, so it is also OK.

All right, after all this gossip, let's look at an example (environment: Ubuntu 14.04)

First of all, we set up two catalogs (fruits and vegetables) and put some files in these two catalogues. there are apples and tomatoes in fruits and carrots and tomatoes in vegetables.

one

two

three

four

five

six

seven

eight

$tree

.

├── fruits

│ ├── apple

│ └── tomato

└── vegetables

├── carrots

└── tomato

Then, we enter the following command:

one

two

three

four

five

six

seven

eight

nine

ten

eleven

twelve

# create a mount directory

$mkdir mnt

# union mount the fruit and vegetable directories to the. / mnt directory

$sudo mount-t aufs-o dirs=./fruits:./vegetables none. / mnt

# View the. / mnt directory

$tree. / mnt

. / mnt

├── apple

├── carrots

└── tomato

We can see that there are three files in the. / mnt directory, Apple apple, Carrot carrots and Tomato tomato. The catalog of fruits and vegetables has been union to the. / mnt directory.

Let's modify the contents of the file:

one

two

three

four

five

$echo mnt >. / mnt/apple

$cat. / mnt/apple

Mnt

$cat. / fruits/apple

Mnt

In the above example, we can see that the content of. / mnt/apple has changed, and so has the content of. / fruits/apple.

one

two

three

four

five

$echo mnt_carrots >. / mnt/carrots

$cat. / vegetables/carrots

$cat. / fruits/carrots

Mnt_carrots

In the above example, we can see that we have modified the file content of. / mnt/carrots. / vegetables/carrots has not changed, but the carrots file appears in the directory of. / fruits/carrots, and its content is our content in. / mnt/carrots.

In other words, in the mount aufs command, we do not refer to its vegetables and fruits directory permissions. By default, the first (leftmost) directory on the command line is read and writable, and the rest is read-only. (generally speaking, the first directory should be writable, while the latter should be read-only.)

So, if we specify permissions to mount aufs as follows, you will find that there is a different effect (remember to delete the file above. / fruits/carrots):

one

two

three

four

five

six

seven

eight

nine

$sudo mount-t aufs-o dirs=./fruits=rw:./vegetables=rw none. / mnt

$echo "mnt_carrots" >. / mnt/carrots

$cat. / vegetables/carrots

Mnt_carrots

$cat. / fruits/carrots

Cat:. / fruits/carrots: No such file or directory

Now, in this case, if we want to modify the file. / mnt/tomato, which file will be rewritten?

one

two

three

four

five

six

seven

$echo "mnt_tomato" >. / mnt/tomato

$cat. / fruits/tomato

Mnt_tomato

$cat. / vegetables/tomato

I am a vegetable

As you can see, if there is a duplicate file name, the higher the priority on the mount command line, the higher the priority.

You can use this example to do a variety of experiments, I am here to give you a perceptual knowledge, so I will not carry out the experiment.

So, what's the use of this UnionFS?

Historically, there is a Linux distribution called Knoppix, which is mainly used for Linux presentations, CD-ROM teaching, system first aid, and demonstrations of commercial products. It does not require hard disk installation, but runs image on CD/DVD directly on a writable storage device (such as a U disk). In fact, it combines CD/DVD as a file system and USB as a writable system to mount. In this way, any changes you make to the image on the CD/DVD will be applied to the flash drive, so you can make any changes to the content on the CD/DVD, because the changes are on the flash drive, so you can't change the original.

We can use our imagination again, or you can put a directory, such as your source code, as a read-only template, with another your working directory to union, and then you can make all kinds of changes without fear of breaking the source code. Kind of like an ad hoc snapshot.

Docker uses UnionFS's imagination to mirror the container. Do you remember when I introduced Linux Namespace, I used mount namespace and chroot to copy a mirror image. Now when you look at this UnionFS technology, do you understand that you can use technologies like UnionFS to make layered images.

The following figure is from Layer, the official document of Docker, which well shows the hierarchical image built by Docker using UnionFS.

With regard to hierarchical mirroring for docker, btrfs, devicemapper and vfs are supported in addition to aufs,docker. You can use the-s or-storage-driver= options to specify the relevant mirror storage. Under Ubuntu 14.04, docker defaults to Ubuntu's aufs (devicemapper is used under CentOS7, which I will explain in a later article about devicemapper). You can view the images of each layer in the following directory:

one

/ var/lib/docker/aufs/diff/

After docker is executed (for example: docker run-it ubuntu / bin/bash), you can view the mount of aufs from the / sys/fs/aufs/si_ [id] directory. Here is an example:

one

two

three

four

five

six

seven

eight

nine

ten

eleven

twelve

thirteen

fourteen

fifteen

sixteen

seventeen

eighteen

nineteen

twenty

# ls / sys/fs/aufs/si_b71b209f85ff8e75/

Br0 br2 br4 br6 brid1 brid3 brid5 xi_path

Br1 br3 br5 brid0 brid2 brid4 brid6

# cat / sys/fs/aufs/si_b71b209f85ff8e75/*

/ var/lib/docker/aufs/diff/87315f1367e5703f599168d1e17528a0500bd2e2df7d2fe2aaf9595f3697dbd7=rw

/ var/lib/docker/aufs/diff/87315f1367e5703f599168d1e17528a0500bd2e2df7d2fe2aaf9595f3697dbd7-init=ro+wh

/ var/lib/docker/aufs/diff/d0955f21bf24f5bfffd32d2d0bb669d0564701c271bc3dfc64cfc5adfdec2d07=ro+wh

/ var/lib/docker/aufs/diff/9fec74352904baf5ab5237caa39a84b0af5c593dc7cc08839e2ba65193024507=ro+wh

/ var/lib/docker/aufs/diff/a1a958a248181c9aa6413848cd67646e5afb9797f1a3da5995c7a636f050f537=ro+wh

/ var/lib/docker/aufs/diff/f3c84ac3a0533f691c9fea4cc2ceaaf43baec22bf8d6a479e069f6d814be9b86=ro+wh

/ var/lib/docker/aufs/diff/511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158=ro+wh

sixty-four

sixty-five

sixty-six

sixty-seven

sixty-eight

sixty-nine

seventy

/ run/shm/aufs.xino

You will see that only the top layer (branch) is rw permission, and the rest is ro+wh permission read-only.

You can see the configuration of docker's aufs in the / var/lib/docker/repositories-aufs file.

Some features of AUFS

AUFS has all the features of Union FS, merging multiple directories into the same directory, and can specify the corresponding permissions for each directory that needs to be merged, and add, delete and modify directories that have been mount in real time. Moreover, he can load balance among multiple writable branch/dir.

In the above example, we have seen an example of AUFS's mount. Let's take a look at the relevant permissions of the directories (branches) that are union:

Rw stands for writable and readable read-write.

Ro stands for read-only, and if you don't mean permission, then ro is the default except for the first one, and for ro branches, they will never receive a write operation or an operation to find whiteout.

Rr stands for real-read-only, and unlike read-only, rr marks branches that are inherently read-only, so that AUFS can improve performance, such as not setting inotify to check file change notifications.

In permissions, we see a term: whiteout. Let me explain this term.

Generally speaking, branches of ro will have the attribute of wh, such as "[dir] = ro+wh". The so-called whiteout means that if a file deleted in union is actually located on a branch (directory) of readonly, then, in the union directory of mount, you will not see this file, but we cannot make any changes on the read-only layer, so we need to whiteout the files in this readonly directory. The whiteout implementation of AUFS is realized by establishing the corresponding whiteout hidden files under the writable directory of the upper layer.

Look at an example:

Suppose we have three directories and files as follows (test is an empty directory):

one

two

three

four

five

six

seven

eight

nine

# tree

.

├── fruits

│ ├── apple

│ └── tomato

├── test

└── vegetables

├── carrots

└── tomato

We have the following mount:

one

two

three

four

five

six

# mkdir mnt

# mount-t aufs-o dirs=./test=rw:./fruits=ro:./vegetables=ro none. / mnt

# # ls. / mnt/

Apple carrots tomato

Now let's create a hidden file .wh.apple of whiteout under the test directory with permission rw, and you will find that the file. / mnt/apple disappears:

one

two

three

four

# touch. / test/.wh.apple

# ls. / mnt

Carrots tomato

The above operation is the same as rm. / mnt/apple.

Related terms

Upload Branch-these are the directories to be union (that is, the command line argument to dirs that I used above)

The union Branch forms a stack according to the order in which it is union. Generally speaking, the top one is writable and the bottom one is read-only.

The stack of classic Branch can be modified after being mount, such as changing the order, adding a new branch, or deleting the branch in it, or directly modifying the permissions of the branch.

Whiteout and Opaque

"if a directory in UnionFS is deleted, it should not be visible, even if it is in the underlying branch.

A directory Whiteout is a directory with the same name in the lower layer overwritten by an upper directory. Files used to hide low-level branches are also used to prevent readdir from entering low-level branches.

"Opaque" means that a directory at any lower level is not allowed to be displayed.

In the case of hiding low-level files, the name of whiteout is' .wh.'

In the case of blocking readdir, the name is' .whr. OPQ'or'. Wh.__dir_opaque'.

Related problems

When you see the above, you must have a few questions:

First, you may ask, what if a file is modified in the same place? Will mount's catalogue be changed together? The answer is yes or no. Because you can specify a parameter called udba (full name: User's Direct Branch Access), this parameter has three values:

Udba=none-when this parameter is set, AUFS will run faster, because aufs will not synchronize those changes that do not occur in the mount directory, so there will be data errors.

Udba=reval-after setting this parameter, AUFS will check to see if the file has been updated, and if so, pull the changes to the mount directory.

Udba=notify-this parameter allows AUFS to register inotify for all branch, which allows AUFS to perform better when updating file changes.

Second, if multiple branch (directories) of rw are union, where will the aufs be created when I create the file? Aufs provides a parameter called create that allows you to configure an equivalent creation policy. Here are a few examples.

Create=rr | round − robin polling. As you can see in the following example, the newly created files are written to three directories in turn

one

two

three

four

five

six

seven

eight

nine

ten

Hchen$ sudo mount-t aufs-o dirs=./1=rw:./2=rw:./3=rw-o create=rr none. / mnt

Hchen$ touch. / mnt/a. / mnt/b. / mnt/c

Hchen$ tree

.

├── 1

│ └── a

├── 2

│ └── c

└── 3

└── b

Create=mfs [: second] | most − free − space [: second] choose the branch with the best available space. You can specify a time to check for free disk space.

Create=mfsrr:low [: second] choose a branch whose space is larger than low. If the space is less than low, then aufs will use round-robin method.

For more details about the usage parameters of AUFS, you can take a look at the various parameters and commands through man aufs directly under Ubuntu 14.04.

Performance of AUFS

Is the performance of AUFS slow? It's slow, not slow. Because AUFS will mount all the branches, it is relatively slow to find files. Because it traverses all the branch. It's an O (n) algorithm (obviously, this algorithm has a lot of room for improvement), so the more branch, the slower the performance of finding files. However, once AUFS finds the inode of the file, the subsequent reading, writing and manipulation of the original file are basically the same.

So, if your program runs under AUFS, there will be significant performance degradation in open and stat operations, the more branch, the worse the performance, but in write/read operations, there is no change in performance.

IBM's research center gave a very good performance report (PDF) "An Updated Performance Comparison of Virtual Machinesand Linux Containers" on the performance of Docker.

I took two pictures, the first one is reading and writing sequentially, the second one is reading and writing randomly. There is basically no problem of performance loss. And KVM is a little slow to read and write at random (but what if the hard drive is SSD? )

Sequential reading and writing

At this point, I believe you have a deeper understanding of "how to use Docker AUFS". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.