Shulou(Shulou.com)05/31 Report--

Today, I would like to share with you the relevant knowledge points about how to configure the joint file system of the core components of Docker. The content is detailed and the logic is clear. I believe most people still know too much about this knowledge, so share this article for your reference. I hope you can get something after reading this article.

1. Definition of federated file system

The federated file system (Union File System,Unionfs) is a hierarchical lightweight file system that jointly mounts the contents of multiple directories into the same directory to form a single file system. This feature allows users to use the federated file system as if they were using a single directory.

What is the existence of a federated file system for Docker? It can be said to be the basis of Docker images and containers, because it enables Docker to make the image into a hierarchical structure, so that each layer of the image can be shared. For example, if both business images are based on CentOS 7 images, these two business images only need to store the basic CentOS 7 image once on the physical machine, thus saving a lot of storage space.

There are three most commonly used federated file systems in Docker: AUFS, Devicemapper, and OverlayFS.

This article focuses on AUFS in the most commonly used federated file system in Docker. Why? Because AUFS is the earliest file system driver used by Docker, it is mostly used in Ubuntu and Debian systems. In the early days of Docker, OverlayFS and Devicemapper were relatively immature, and AUFS was the earliest and most stable file system driver.

two。 Configure AUFS mode for Docker

AUFS is not currently incorporated into the Linux kernel mainline, so only a few operating systems such as Ubuntu and Debian support AUFS. You can use the following command to see if your system supports AUFS:

$grep aufs / proc/filesystemsnodev aufs

After executing the above command, if the output contains aufs, it means that the current operating system supports AUFS. AUFS is recommended for use in Ubuntu or Debian operating systems. If you want to use AUFS in operating systems such as CentOS, you need to install the AUFS module separately (AUFS is not recommended in CentOS in the production environment). After installation, use the above command to output aufs.

After confirming that the operating system supports AUFS, you can configure the startup parameters of Docker.

First create a new daemon.json file under / etc/docker and write the following:

{"storage-driver": "aufs"}

Then restart Docker using the following command:

$sudo systemctl restart docker

After Docker restart, you can use the docker info command to check whether the configuration takes effect:

$sudo docker infoClient: Debug Mode: falseServer: Containers: 0 Running: 0 Paused: 0 Stopped: 0 Images: 1 Server Version: 19.03.12 Storage Driver: aufs Root Dir: / var/lib/docker/aufs Backing Filesystem: extfs Dirs: 1 Dirperm1 Supported: true

You can see that Storage Driver has become aufs, proving that the configuration has taken effect, and after the configuration has taken effect, you can use AUFS to provide a federated file system for Docker.

3. How AUFS works 3.1 how AUFS stores files

AUFS is a federated file system, which means it uses multi-tier directory storage on the host. Each directory is called branch in AUFS and layer in Docker, but the final presentation to the user is an ordinary single-tier file system. The process in which multiple layers are presented in a single-tier manner is called federated mount.

As shown in figure 1, each mirror layer and container layer is a subdirectory under / var/lib/docker, the image layer and container layer are under the aufs/diff directory, the directory name of each layer is the ID value of the image or container, the joint mount point is under the aufs/mnt directory, and the mnt directory is the real container working directory.

Let's describe in detail the changes before and after the creation of the container according to the directory structure under the aufs folder.

When an image does not generate a container, the storage structure of AUFS is as follows.

Diff folder: stores mirrored content, and each layer is stored in a subfolder named after the mirror layer ID.

Layers folder: stores metadata for mirror layer relationships, where each mirror layer under the diif folder has a file whose content is the ID of the parent mirror of that layer mirror.

Mnt folder: the joint mount point directory, which is empty when no container is generated.

When an image has been generated into a container, the AUFS storage structure changes as follows.

Diff folder: when the container is running, the container layer is generated under the diff directory.

Layers folder: add metadata related to the container layer.

Mnt folder: the joint mount point of the container, which is consistent with the contents of the files seen in the container.

3.2 how does AUFS work

1. Read a file

When we read the file in the container, there may be the following scenarios.

When the file exists in the container layer: when the file exists in the container layer, it is read directly from the container layer.

When the file does not exist in the container layer: when the container is running, you need to read a file, if it does not exist in the container layer, look for the file from the mirror layer, and then read the contents of the file.

The file exists in both the mirror layer and the container layer: when the file we read exists in both the mirror layer and the container layer, it will be read from the container layer.

two。 Modify a file or directory

The modification of files by AUFS adopts the working mechanism of copying when writing, which can save storage space to the maximum extent.

The specific file operation mechanism is as follows.

Modify the file for the first time:

When we modify a file in the container for the first time, AUFS triggers a write-time copy operation. AUFS first copies the file from the mirror layer to the container layer, and then performs the corresponding modification operation.

The operation of copying when AUFS writes will copy the entire file, and if the file is too large, it will greatly reduce the performance of the file system, so when we have a large number of files to be modified, AUFS may have significant delays. Fortunately, the copy-on-write operation is triggered only when the file is modified for the first time and does not have much impact on daily use.

Delete a file or directory:

When a file or directory is deleted, AUFS does not actually delete it from the mirror, because the mirror layer is read-only, and AUFS creates a special file or folder that blocks access to the container.

4. AUFS demo 4.1 prepare demo directories and files

First, we create the aufs directory under the / tmp directory:

$cd / tmp/tmp$ mkdir aufs

Prepare the mount point directory:

/ tmp$ cd aufs/tmp/aufs$ mkdir mnt

Next prepare the contents of the container layer:

# # create an image layer directory / tmp/aufs$ mkdir container1## prepare a file / tmp/aufs$ echo Hello, Container layer! > container1/container1.txt under the image layer directory

Finally, prepare the mirror layer content:

# # create two mirror layer directories / tmp/aufs$ mkdir image1 & & mkdir image2## to write data / tmp/aufs$ echo Hello, Image layer1! > image1/image1.txt/tmp/aufs$ echo Hello, Image layer2! > image2/image2.txt respectively

The prepared directories and file structures are as follows:

/ tmp/aufs$ tree.. |-- container1 | `--container1.txt |-- image1 |`-- image1.txt |-- image2 | `--image2.txt`-- mnt4 directories, 3 files4.2 to create an AUFS federated file system

Use the mount command to create a file system of type AUFS, as follows:

/ tmp/aufs$ sudo mount-t aufs-o dirs=./container1:./image2:./image1 none. / mnt

When the mount command creates an AUFS-type file system, it should be noted here that the first colon of the dirs parameter defaults to read-write permission, and the subsequent directories are read-only permission, which is consistent with the mode in which the Docker container uses AUFS.

After executing the above command, mnt becomes the federated mount directory of AUFS. We can use the mount command to view the AUFS file system that has been created:

/ tmp/aufs$ mount-t aufsnone on / tmp/aufs/mnt type aufs (rw,relatime,si=4174b83d649ffb7c)

Every time we create an AUFS file system, AUFS will generate an ID for us. The ID will create a corresponding directory in / sys/fs/aufs/, and you can view the file mount permissions in this ID directory.

Tmp/aufs$ cat / sys/fs/aufs/si_4174b83d649ffb7c/*/tmp/aufs/container1=rw/tmp/aufs/image2=ro/tmp/aufs/image1=ro646566

You can see that the permission of the container1 directory is rw (for read-write), and the permission for image1 and image2 is ro (for read-only).

To verify that you can see everything in the container1, image1, and image2 directories under the mnt directory, let's look at the mnt directory using the ls command:

/ tmp/aufs$ ls-l mnt/total 12 RWMurray RWMI-1 ubuntu ubuntu 24 Sep 9 16:55 container1.txt-rw-rw-r-- 1 ubuntu ubuntu 21 Sep 9 16:59 image1.txt-rw-rw-r-- 1 ubuntu ubuntu 21 Sep 9 16:59 image2.txt

You can see that all the mirror layer and container layer files we prepared have appeared in the mnt directory. Let's verify the write-time replication of AUFS.

4.3 verify write-time replication of AUFS

Write-time copy of AUFS means that only when a file needs to be modified will the file be copied from the mirror layer to the container layer. Let's verify this process by modifying the contents of the joint mount directory mnt.

We use the following command to modify the image1.txt file in the mnt directory:

/ tmp/aufs$ echo Hello, Image layer1 changed! > mnt/image1.txt

Then let's look at the contents of the image1/image1.txt file:

/ tmp/aufs$ cat image1/image1.txtHello, Image layer1!

It is found that the image1.txt file of the Mirror layer has not been modified.

Then let's look at the contents of the image1.txt file corresponding to the "container layer":

/ tmp/aufs$ ls-l container1/total 8 ubuntu ubuntu RW Image layer1 changed-1 ubuntu ubuntu 24 Sep 9 16:55 container1.txt-rw-rw-r-- 1 ubuntu ubuntu 29 Sep 9 17:21 image1.txt## view file contents / tmp/aufs$ cat container1/image1.txtHello, Image layer1 changed!

It is found that AUFS automatically creates the image1.txt file in the "container layer", and the content is what we just wrote.

These are all the contents of the article "how to configure the federated file system of Docker core components". Thank you for reading! I believe you will gain a lot after reading this article. The editor will update different knowledge for you every day. If you want to learn more knowledge, please pay attention to the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.