Shulou(Shulou.com)05/31 Report--

This article focuses on "how to understand the Linux virtual file system", interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how to understand the Linux virtual file system.

Background

Sometimes there is a situation where the disk space shows that it is full, but when looking at the specific file occupancy of the disk, it is found that the disk still has a lot of free space.

1. Execute the df command to check the disk usage and find that the disk is full.

-bash-4.2$ df-Th Filesystem Type Size Used Avail Use% Mounted on / dev/vda1 ext4 30G 0100% / devtmpfs devtmpfs 489M 0 489M 0% / devtmpfs tmpfs 497M 0497M 0% / dev/shm tmpfs tmpfs 497M 50M 447M 11% / run tmpfs tmpfs 497M 0497M 0% / sys/fs/cgroup

two。 Execute the du command to check the disk occupancy of each directory, add up the size of each directory file, and find that it does not fill the disk, and more than 10 G of space is missing.

-bash-4.2$ du-h-- max-depth=1 / home16M / home/logs11G / home/serverdog11G / home

3. Why did this happen?

Because although the files have been deleted, some processes still open them, so the disk space they occupy has not been released. Execute the lsof command to open the deleted file. Restart (or empty) the problematic process and disk space will be freed.

-bash-4.2# lsof | grep deletemysqld 2470 mysql 4U REG 253 1 523577 / var/tmp/ibfTeQFn (deleted) mysqld 2470 mysql 5u REG 253 1 0 523579 / var/tmp/ibaHcIdW (deleted) mysqld 2470 mysql 6u REG 253 1 0 523581 / var/tmp/ibLjiALu (deleted) mysqld 2470 mysql 7u REG 253 1 0 523585 / var/tmp/ibCFnzTB (deleted) mysqld 2470 mysql 11u REG 253

So, why on earth is Linux's file system designed this way? To understand this, it is not easy to figure it out. Here are some basic concepts to sort out step by step:

What is a virtual file system (VFS:virtual filesystem)?

What is a general file model?

Super Block object (superblock object)

Inode object (inode object)

File object (file object)

Catalog item object (dentry object)

The concept of documents

The expression of a document

Memory expression

Disk representation

Construction of directory tree

Soft links vs hard links

File-disk management

Inode status

File & process management

Action:

Open & delete

Virtual file system (virtual filesystem)

The following figure shows the basic components responsible for file management in the Linux operating system. The upper half is in user mode and the lower half is in kernel mode. The application uses the standard library libc to access files, which maps requests to system calls in order to enter kernel mode.

The entry for all file-related operations is a virtual file system (VFS), not a specific quota file system (such as Ext3, ReiserFS, and NFS). VFS provides an interface between system libraries and specific file systems. Therefore, VFS not only acts as an abstraction layer, but it actually provides a basic implementation of the file system that can be used and extended by different implementations. Therefore, to understand how the file system works, you need to understand VFS.

General file model

The main idea of VFS is to introduce a general file model (common file model). The common file model consists of the following object types:

Super Block object (superblock object)

Memory: created during file system installation to store information about the file system

Disk: corresponds to the file system control block (filesystem control block) stored on disk

Inode object (inode object)

Memory: created on access, storing general information about specific files (inode structure)

Disk: corresponding to the file control block (file control block) stored on disk, each Inode object has an index node number that uniquely identifies the file system.

File object (file object)-memory: created when opening a file, storing information about the interaction between the open file and the process (file structure) Open file information exists in kernel memory only when the process accesses the file.

Catalog item object (dentry object)

Memory: once a directory item is read into memory, VFS converts it into a directory entry object with a dentry structure

Disk: a specific file system is stored on disk in a specific way

Store information about the link between the directory entry (that is, the file name) and the corresponding file

Directory tree

Overall, the root file system (system's root filessystem) of Linux is the first file system on which the kernel boots mount. The kernel code image file is saved in the root file system, and the system boot launcher loads some basic initialization scripts and services into memory to run after the root file system is mounted (the file system and the kernel are two completely separate parts). For other file systems, it is subsequently mounted as a child file system through scripts or commands on the directory where the file system has been mounted, resulting in the entire directory tree.

Start_kernel vfs_caches_init mnt_init init_rootfs / / Register rootfs file system init_mount_tree / mount rootfs file system... Rest_init kernel_thread (kernel_init, NULL, CLONE_FS)

As far as a single file system is concerned, a super block object is created when the file system is installed; when looking for files along the tree, always first find the matching directory entry from the initial directory in order to get the corresponding index node, then read the directory file of the index node, convert it to dentry object, check the matching directory entry, and repeat the above process until the corresponding file's index node is found, and the index node object is created.

Soft links vs hard links

A soft link is an ordinary file in which the pathname of another file is stored. Hard links point to the same index node, and the number of hard links is recorded in the i_nlink field of the index node object. When the i_nlink field is 00:00, there is no hard link to the file.

File & process management

The following figure is a simple example of how a process interacts with a file. Three different processes open the same file, each with its own file object, two of which use the same hard link (each hard link corresponds to a directory object), and both directory entry objects point to the same Inode object.

The data of the index node consists of two parts: memory data and disk data. Linux uses Write back as the data consistency policy for index nodes. For the data of the index node, the index node is loaded into memory only when the file is opened; when it is no longer used by the process, it is kicked out of memory; if there is an update in the middle, the data needs to be written back to disk.

* "in_use"-valid inode, i_count > 0, i_nlink > 0 * "dirty"-as "in_use" but also dirty * "unused"-valid inode, i_count = 0

Whether the index node is still in use or not, the file object is created and destroyed through the open () and close () operations, and the file object updates the i_count field of the index node through the iget and iput provided by the index node to complete the usage count. The open operation makes the i_count plus one, and the close operation makes the i_count minus one. Determine whether the Inode is freed during the close operation. If i_count = 0, it means that there are no more process references and will be freed from memory.

File-disk management

The operations that are most closely related to disk management are touch and rm operations, and the latter is the most critical. Through strace (or dtruss), view the actual system calls of rm

# dtruss rm tmp... Geteuid (0x0, 0x0, 0x0) = 00 ioctl (0x0, 0x4004667A, 0x7FFEE06F09C4) = 00 lstat64 ("tmp\ 0", 0x7FFEE06F0968, 0x0) = 00 access ("tmp\ 0", 0x2, 0x0) = 00 unlink ("tmp\ 0", 0x0, 0x0) = 00

You can see that rm is actually done through unlink. Unlink stands for deleting catalog entries and reducing the count of its inodes. According to the general file model, the parent directory itself is also a file, which means that the directory entry is part of its file data. Deleting a directory entry is equivalent to deleting data from a file in the parent directory, which means opening the file in the parent directory first. Then, the delete operation can be understood as:

Delete command (a process) use the open operation to get the parent directory file object

Increase the Inode object count of directory files through iget

Read directory file data

Convert catalog file data to catalog item objects

Since the directory entry contains the file's Inode, similarly, you need to increase the Inode object count of the file through iget

Delete a directory entry for a directory

Reduce the hard link count of file Inode objects i_nlink

End the operation on the file Inode object through iput, using count i_count minus one

Determine whether the i_count is zero, and if so, free memory

Then, determine whether the i_nlink is zero, and if so, free up disk space

End the operation on the catalog Inode object through iput.

At this point, I believe you have a deeper understanding of "how to understand the Linux virtual file system". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.