Shulou(Shulou.com)06/01 Report--

This article introduces the relevant knowledge of "what Linux disk cache related knowledge". In the operation of actual cases, many people will encounter such a dilemma. Next, let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Preface

Recently I encountered a disk-related online failure to summarize some knowledge about Linux disk cache that I didn't know much about before.

Generally speaking, there are two reasons for disk cache: the first is that the speed of accessing disk is much slower than that of accessing memory, and the access speed can be improved by caching disk contents in memory; second, according to the locality principle of the program, once the data has been accessed, it is likely to be accessed again in a short time, so caching disk contents in memory can improve the running speed of the program.

Locality principle

Program locality principle: the program presents a local law when it is executed, that is, the execution of the whole program is limited to a certain part of the program within a period of time. Accordingly, the storage space accessed by the execution is also limited to a certain memory area. specifically, locality usually has two forms: time locality and spatial locality.

Time locality: a memory location that has been referenced once will be referenced multiple times in the future.

Spatial locality: if the location of a memory is referenced, the location near it will also be referenced in the future.

Page cache

In the Linux system, in order to reduce the IO operation to the disk, the contents of the open disk will be cached, and the place of the cache is the physical memory, thus converting the access to the disk into the access to memory, which can effectively improve the speed of the program. Linux caches content on disk using physical memory, which is called page cache (page cache).

The page cache is made up of physical pages in memory, whose contents correspond to physical blocks on disk. The size of the page cache is dynamically adjusted according to the memory free size of the system. It can expand the size by occupying memory, or it can shrink itself to relieve the pressure of memory usage.

Before the virtual memory mechanism appeared, the operating system used the block cache series, but after the virtual memory appeared, the operating system managed IO with a larger granularity, so it adopted the page cache mechanism, which is a page-based, file-oriented cache mechanism.

Reading of page cache

When reading a file, the Linux system will first read the file content from the page cache. If the page cache exists, the system will first read the file content from the disk and update it to the page cache, and then read the file content from the page cache and return.

The general process is as follows:

The process calls the library function read to initiate a request to read the file

The kernel checks the list of open files and calls the read interface provided by the file system

Find the inode corresponding to the file, and then calculate the specific page to read

Find the corresponding page cache through inode: 1) if the page cache node is hit, the file content is returned directly; 2) if there is no corresponding page cache, a page fault exception (page fault) will be generated. At this point, the system creates a new empty page cache and reads the file contents from disk, updates the page cache, and then repeats step 4

Read file and return

Therefore, all the reading of the contents of the file, whether initially hit or not, comes directly from the page cache.

Write to page cache

Because of the existence of the page cache, when a process calls write, the update to the file is simply written to the file's page cache, and the corresponding page is marked as dirty, and the whole process ends. The Linux kernel periodically writes dirty pages back to disk and then clears the dirty flag.

Because writes only write changes to the page cache, the process does not block until disk IO occurs, and if the computer crashes, the write changes may not occur on disk. Therefore, for some more stringent write operations, such as data systems, we need to actively call fsync and other operations to synchronize the changes to the disk in time. The read operation is different. Read usually blocks until the process reads the data, but in order to reduce the delay of the read operation, the Linux system still uses the "read-ahead" technology, that is, when reading data from disk, the kernel will read more pages into the page cache.

Write-back thread

The write-back of the page cache is done by a separate thread in the kernel, and the write-back thread writes back in the following three cases:

When the free memory is below the threshold. When there is insufficient free memory, part of the cache needs to be freed, and since only non-dirty pages can be released, dirty pages need to be written back to disk to make them clean pages that can be recycled.

When the processing time of dirty pages in memory exceeds the threshold. This is to ensure that dirty pages do not stay in memory indefinitely, reducing the risk of data loss.

When the user process calls sync and fsync system calls. This is to provide the user process with a method of forced writeback to meet the strict usage scenarios of writeback.

Implementation of write-back thread

The name version indicates that before the bdflush2.6 version, the bdflush kernel thread ran in the background, and there was only one bdflush thread in the system. When the memory consumption fell below a certain threshold, the bdflush thread was awakened. Kupdated runs periodically, writing back to the dirty page. But there is only one bdflush thread in the whole system. When the system write-back task is heavy, the bdflush thread may block on the Ibank O of one disk, so that the write-back operation of other disks can not be executed in time. The number of pdflush threads introduced into the pdflush2.6 version is dynamic, depending on the system's Icano load. It is oriented to the global task of all disks in the system. However, because pdflush is oriented to all disks, it is possible that multiple pdflush threads are blocked on a congested disk, which also causes the write-back of other disks not to be performed in time. The number of flusher threads introduced after version 2.6.32 is not unique, and flusher threads are not oriented to all disks, but each flusher thread corresponds to one disk

Collection of page cache

The replacement logic for the page cache in Linux is a modified LRU implementation, also known as the double chain policy. Unlike before, Linux no longer maintains one LRU linked list, but maintains two linked lists: active linked list and inactive linked list. Pages on active lists are considered "hot" and will not be swapped out, while pages on inactive lists can be swapped out. Pages in the active linked list must be in the inactive linked list when they are accessed. Both linked lists are maintained by pseudo-LRU rules: pages are added from the tail and removed from the header, like a queue. The two linked lists need to be balanced-if the active list becomes too many to exceed the inactive list, the header page of the active list will be moved back to the inactive list and can be recycled again. The double linked list strategy solves the dilemma of only one visit in the traditional LRU algorithm. And it is also easier to implement pseudo-LRU semantics. This double-linked list is also known as LRU/2. More commonly, there are n linked lists, so it is called LRU/n.

This is the end of the content of "what is the knowledge of Linux disk caching". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.