Linux: ensure the security of data storage

Background

In many IO scenarios, we often need to make sure that the data is safely written to disk so that it can be read after the system is down and restarted. But as we all know, the IO path of the linux system is still very complex, which is divided into many layers, each of which may have buffer to speed up IO read and write. At the same time, user-mode applications and library functions may also have their own buffer, which adds some complexity to the IO path. It can be seen that if you want to ensure that the data is safely written to disk, it is not easy to call a write/fwrite.

So how do we do that? Many people will think of many ways, such as: fflush (), fsync (), fdatasync (), sync (), open () using O_DIRECT or O_SYNC flags, and so on. Well, these tools (or some combinations) do ensure the persistence of data security, so what's the difference between them? What's the difference between fflush () and fsync ()? What does O_DIRECT mean? can it guarantee the persistence of data security? What is the difference between O_DIRECT and O_SYNC? What about O_SYNC and fsync ()? Can fsync complete the function of msync? This article will try to understand and explain the functions and differences of these concepts.

Linux IO

As the saying goes, a picture is worth a thousand words. In order to analyze the differences between these concepts, I specially drew a picture. If you look at it carefully, you should be able to see their functions and differences clearly.

Here we focus on O_DIRECT and O_SYNC. First of all, it is clear that O_DIRECT only says that data will not go through page cache (usually used to manage buffer in user mode) but will be submitted directly to the block device layer, but will not synchronously wait for data to be safely written to disk before returning (for example, data may still be queued at the block layer or in the disk's own cache). As for the O_SYNC flag, although the data will still be written to page cache, it will adopt the policy of write through and wait for the data to be safely written to disk before returning. Therefore, if you use O_DIRECT and O_SYNC at the same time, it means that the data will not pass through the page cache and wait for the data to be safely written to disk before returning, of course, the performance of IO will be very low.

Because O_DIRECT can bypass page cache, it is also important to note that data inconsistencies may occur if another process reads the file in the normal way.

To give an auxiliary explanation, I would like to post some information that I have seen in the course of my discussion. The first is to reference the open system call:

Http://man7.org/linux/man-pages/man2/open.2.html

Description of relevant parameters:

And innodb-related documentation:

Https://lwn.net/Articles/457667/

The difference between fsync and fdatasync:

Http://man7.org/linux/man-pages/man2/fsync.2.html

Msync:

Http://man7.org/linux/man-pages/man2/msync.2.html

DAX

In fact, there is another IO mode, which is DAX (Direct Access). Does it look a lot like O_DIRECT? This mode needs to be supported by both filesystem and block driver, and is generally mainly used on non volatile memory. In essence, it is also bypassing page cache to directly operate the device. DAX this article does not go into depth. Later, I will write a ramdisk block device driver that supports DAX mode, then format it into an ext4 file system and mount it in-o dax mode, and then study the IO path of DAX in detail.

Finally, the io path tracking of Linux in common scenarios is attached:

Https://my.oschina.net/fileoptions/blog/3061822