Shulou(Shulou.com)05/31 Report--

This article is about how to conduct Linux equipment IO research and database performance tuning. The editor thinks it is very practical, so I share it with you. I hope you can get something after reading this article. Let's take a look at it.

How to tune the database and equipment IO library in Linux system?

The database system is based on the file system, and its performance is closely related to the reading and writing mechanism of the device. There are three operations of the file Iripple O operation that are closely related to database performance:

Open opens a file

Write writes files

Fdatasync flush operation (brushing the file cache to disk).

1. Open operation

Open ("test.file", O_WRONLY | O_APPDENT | O_SYNC))

The system call Open will give the process a file descriptor FD. O_WRONLY | O_APPDENT | O_SYNC is used to open the file:

1. O_WRONLY means that we open it in "write" mode, telling the kernel that we need to write data to the file

2. O_APPDENT tells the kernel to write files in an "appended" way

3. O_DSYNC tells the kernel that when writing data to a file, the write operation is completed only when the data is written to disk (write returns success).

4. The same kind of file logo as O_DSYNC, as well as Olympus SYNC Magneol Olympus direct.

(1) O_SYNC is more stringent than O_DSYNC, which requires not only that the data has been written to disk, but also that the attributes of the corresponding data file (such as file length, etc.) need to be updated before the write operation is successful. It can be seen that O_SYNC has to do more than O_DSYNC.

(2) O_RSYNC means that when the file is read, the OS cache of the file must have all flush to disk.

(3) if you open a file using O_DIRECT, read / write operations will skip OS cache and read / write directly on device (disk). Because there is no OS cache, O_DIRECT reduces the efficiency of sequential reading and writing of files.

II. Write operation

Write (fd,buf,6)

After using open to open the file to get the file descriptor, we can call the write function to write the data, and write behaves differently depending on the previous open parameter.

III. Flush stage

Fdatasync (fd) = =-1

After the write operation, we also call fdatasync to ensure that the file data flush is on the disk. After the fdatasync returns successfully, you can assume that the data has been written to disk. Flush functions like this also include fsync and sync.

1. The difference between Fsync and fdatasync is the same as that between O_SYNC and O_DSYNC.

2. The Sync function indicates that the data of the file in OS cache is put into the write queue, and does not confirm whether it is really written to disk, so sync cannot be relied on.

Ignoring the process of opening a file, we usually say that there are two stages of "writing a file". One is to call write what we call the write data phase (actually affected by the parameters of open), and to call fsync (or fdatasync) what we call the flush phase. The operations of block devices on Linux can be divided into two categories:

The * class is a function that uses the fopen/fread/fwrite series in the C standard library, and we can call it buffered Icano.

The specific I _ path is as follows:

ApplicationLibrary BufferOperation System CacheFile System/Volume ManagerDevice

Library buffer is the buffer of user space provided by the standard library, which can be resized through setvbuf.

The second category is the open/read/write series of functions that use Linux's system calls, which we can call non-buffered Imax O.

Application Operation System Cache File System/Volume ManagerDevice

In addition, we can implement Direct I O_DIRECT O (or Raw I OS Cache O) by setting the O_DIRECT flag of Direct, that is, bypassing OS Cache and reading Device directly (that's what we can ^ o ^), which is equivalent to replacing OS cache with our own managed cache. However, Linus recommends not to do this in the mailing list, but to use posix_fadvice, madvice. It is shown that the performance of Direct I _ plink O is much higher than that of buffered I _ Candle O.

In MySQL, the parameter Innodb_flush_method (Linux) can be set to: Fdatasync, O_DSYNC, O_DIRECT. Let's take a look at how these three parameters affect the operation of the program MySQL on log and data files:

Open logFlush logOpen datafileFlush dataFdatasync

Fsync ()

Fsync () O_DSYNCO_SYNC

Fsync () O_DIRECT

Fsync () O_DIRECTFsync ()

Fdatasync is considered secure because fsync is always called in MySQL to flush the data. There are some risks in using O_DSYNC, and some OS will ignore the parameter O_SYNC.

We see that O_DIRECT is very similar to fdatasync, but it uses O_DIRECT to open the data file. Data show that O_DIRECT will improve efficiency if it is a large number of random write operations. However, sequential write and read efficiency will be reduced. So you need to be careful when using O_DIRECT.

Mysql innodb corresponds to relevant parameters:

Innodb_flush_method has three values, fdatasync,O_DSYNC and O_DIRECT, where fdatasync is the default.

They control the mode in which InnoDB refreshes logs and data.

Fdatasync:InnoDB uses the fsync () function to update log and data files.

O_DSYNC:InnoDB uses the O_SYNC mode to open and update the log file, and the fsync () function to update the data file.

O_DIRECT:InnoDB uses O_DIRECT mode to open data files and uses the fsync () function to update log and data files.

We see that O_DIRECT is very similar to fdatasync, but it uses O_DIRECT to open the data file. Data show that O_DIRECT will improve efficiency if it is a large number of random write operations. However, sequential write and read efficiency will be reduced. So you need to be careful when using O_DIRECT.

The above is how to carry out Linux equipment IO research and database performance tuning, the editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.