Shulou(Shulou.com)05/31 Report--

This article introduces the relevant knowledge of "how the Linux log file system works". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

One of the key problems to be solved in the file system is how to prevent data corruption caused by power outage or system crash. In such accidents, the root cause of file system corruption is that writing a file is not an atomic operation, because writing a file involves not only user data, but also metadata, including Superblock, inode bitmap, inode, data block bitmap, etc., so the write operation cannot be completed in one step. If any of these steps are interrupted, the data will be inconsistent or corrupted.

To take a simplified example, we write to a file, which involves the following steps:

Allocate a block from data block bitmap

Add a pointer to a data block in inode

Write user data to the data block.

If step 2 is completed and step 3 is not completed, the result is data corruption, because the file thinks the data block is its own, but the data in it is actually junk.

If step 2 is completed and step 1 is not completed, the result is metadata inconsistency because the file already owns the data block. however, the file system still believes that the data block is not allocated, and may then allocate the data block to other files, resulting in data overwriting

If step 1 is completed and step 2 is not completed, the result is that the file system allocates a block of data, but no files use this block, resulting in a waste of space

If step 3 is completed and step 2 is not completed, the result is that the user data is written to the hard disk block, but in vain, because the file does not know that the block is its own.

The journaling file system (Journal File System) was born to solve the above problems.

Its principle is that before the write operation, each step (called transaction) is recorded in advance and saved in a separate space on the file system, which is called journal, also known as write-ahead logging. After the log is saved successfully, the real write operation is carried out, and the metadata and user data of the file system are written to the hard disk (called checkpoint), so that in case the power is lost during the write operation. Just re-execute the saved log before mounting the file system next time (the term is replay), avoiding the aforementioned data corruption scenario.

Some people ask what if there is a power outage in the process of saving the log. The original idea was to write the data of a log to the hard disk at once, which is equivalent to an atomic operation, but this is not feasible, because the hard disk is usually operated in 512 bytes, and once the log data exceeds 512 bytes, it is impossible to write it at once. So this is actually done: set a Terminator for each log, and write the Terminator only after the log has been successfully written. If a log does not have a corresponding Terminator, it will be regarded as invalid and discarded directly. This ensures that the data in the log is complete.

A log is useless after its corresponding write operation is completed, and the hard disk space occupied can be released. The hard disk space on which the log is saved is limited and is recycled, so the log is also called circular log.

At this point, you can summarize the working steps of the journaling file system:

Journal write: write transaction to the log

Journal commit: write Terminator after a log has been saved

Checkpoint: do real write operations, writing metadata (metadata) and user data (user data) to the file system

Free: recycle the hard disk space occupied by the log.

The above method also records user data (user data) in the log, which is called Data Journaling,Linux EXT3 file system, which supports this method, which has efficiency problems:

That is, the metadata (metadata) and user data (user data) involved in each write operation are actually written twice on the hard disk, once in the log and once on the file system. Metadata is fine, because user data is usually large, and it is inefficient to multiply a few GB movie files by 2.

A more efficient way is Metadata Journaling, which does not record user data (user data) in the log. The way to prevent data damage is to write user data (user data) and then log, that is, to write user data before the above-mentioned "Journal write". This ensures that as long as the log is valid, then its corresponding user data is also valid. In the event of a power failure, the worst result is that the last log is not finished. Then the corresponding user data will also be lost, the effect is the same as Data Journaling discarding logs, the important thing is that the consistency and integrity of the file system are guaranteed.

Metadata Journaling, also known as Ordered Journaling, is used in most file systems. File systems like Linux EXT3 can also choose between Data Journaling and Ordered Journaling.

This is the end of how the Linux journaling file system works. Thank you for your reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.