Shulou(Shulou.com)05/31 Report--

MySQL 8.0 source code redo log generation and usage is how, many novices are not very clear about this, in order to help you solve this problem, the following editor will explain in detail for you, people with this need can come to learn, I hope you can gain something.

When I first learned about the mysql implementation, I always heard the keywords redo log, WAL (write-ahead logging), undo log, and learned that redo log is mainly used to achieve transaction persistence. In order to further understand redo log, take a look at the relevant code (source code version: mysql 8.0.12). Here is a brief summary of how redo log is generated, how to drop the disk, and how to finally notify the user.

As the time to learn mysql is not long, the mysql code is also very large, I read only a small part of it, there is an incorrect understanding of the article, welcome to correct guidance, learn together.

The generation of redo log

During the execution of read and write transactions, redo log will be generated continuously. Application data page, modify data page, record undo log, etc., will generate redo log. Mysql divides the user transaction into mtr (mini transaction). Redo log is first recorded in mtr and committed with the commit of mtr, and finally falls on the hard disk.

Submission of redo log

When mtr commits, it writes the redo log in mtr to the log buffer of the system variable log_sys. A new feature of mysql8.0 is the lock-free nature of redo log submissions. Before 8.0, each user thread competed with each other to write log buffer serially, so the order of lsn was guaranteed to grow without interval. At 8.0, user threads can write log buffer concurrently. If a user thread flashes the previous log buffer of lsn written by itself after writing log buffer successfully, it may cause other user threads to be flushed before writing log buffer.

Figure 1

To solve this problem, mysql 8.0 introduces Link_buf as a data structure to avoid log buffer holes. Link_buf is actually a fixed-length array that tracks writes to an interval of log buffer like a sliding window, moving forward as continuous redo log is written in log buffer.

The data structure of Link_buf is shown in the figure:

Figure 2

When the user writes redo log between the start_lsn-end_lsn of log buffer, it marks the corresponding location of Link_buf, that is, m _ link [start _ lsn%m_capacity] is assigned to end_lsn-start_lsn.

The process for redo log to record to log buffer is as follows:

1. First of all, when each user thread writes redo log, it first obtains the start_lsn and end_lsn of the redo log log from the system global atomic variable log_sys.sn according to the length of the redo log. The atomic variable sn can ensure that the start_lsn-end_lsn interval obtained by each thread is continuous and void-free.

Figure 3

two。 After the user thread applies to the start_lsn-end_lsn range, it needs to wait until the Link_buf is pushed to a location where it can be used.

Figure 4.

As shown in the figure, start_lsn0-end_lsn0,start_lsn2-end_lsn2 and start_lsn3-end_lsn3 are the lsn intervals newly requested by three user threads; the corresponding intervals of start_lsn1-end_lsn1 have been marked on link_buf; start_lsn3-end_lsn3 is too far from tail and needs to wait for link_buf to push before it can be used.

3. After writing log buffer, mark the range of start_lsn- > end_lsn to link_buf (note: because link_buf is only marked at the location of start_lsn%capacity, it does not affect even if end_lsn exceeds (m_tail, m_tail+m_capacity))

Figure 5

4. Set the event log_sys.writer_event when the user thread commits the transaction, triggering the log_ writer thread to write the log from redo log buffer to the system cache (the log_writer thread itself polls link_buf to determine whether a new log has been written)

The 5.log_writer thread advances the m_tail and removes the log buffer before the m_tail.

Figure 6

The closing and notice of redo log

Earlier, we briefly described how redo log is submitted. Multiple threads are involved in redo log submission and disk release, and their relationship is as follows:

Figure 7.

When a user thread commits a read-write transaction, it generates some redo log and records it in the redo log buffer as the mtr commits. Then the user thread attempts to set writer_event to trigger the log_writer thread to write the log and listens for its own flush_ events [I].

The log_writer thread advances the Link_buf.m_tail, writes the redo log before the maximum continuous lsn to the system cache, and sets the flusher_event to trigger the log_flusher thread

The log_ Flusher thread brushes the log that has been written to the system cache and sets flush_notifier_event to trigger the log_flush_notifier thread to notify the user

Log_flush_notifier converts the events that need to be triggered according to the lsn that has been brushed, and notifies the user thread.

In the specific implementation, the writing of redo log is followed up through several member variables in log_sys. Where log_sys.recent_writtern.m_tail represents the maximum contiguous range of log buffer; log_sys.write_lsn represents the location where the write is written to the system cache; and log_sys.flushed_to_disk_lsn indicates the location where the disk has been dropped. The advance process of each mark is as follows:

Figure 8.

Notify user thread

When a user commits a transaction, log_wait_for_write or log_wait_for_flush is called based on the innodb_flush_log_at_trx_commit parameter to wait for the redo log to be written to the system cache or brushed to the hard disk. The notification of the user thread is realized through the log_sys.flush_events event array. In order to avoid too many flush_events notifications at one time, the flush_events is divided into different user threads like a bucket: the redo log is divided into log block. Assuming that the length of the log_sys.flush_events array is m, the disk of the nth log block is flushed, and the flush_ events [n% m] event listens. When the first log block to the second log block of the log buffer is flushed, the flush_events to which the log block between the L1-L2 belongs is set, so that the user thread of the redo log between the L1-L2 will be notified.

Figure 9.

Through redo log lock-free, mysql8.0 solves the performance impact of competitive locks when user threads write redo log. At the same time, the redo log file writing and redo log disk brushing are separated from the user thread and separated into a separate thread. The user thread is only responsible for writing the redo log to the log buffer, no longer cares about the details of the redo log disk, and only needs to wait for the notification from the log_ writer thread or the log_flusher thread.

Is it helpful for you to read the above content? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.