Shulou(Shulou.com)05/31 Report--

Today, I will talk to you about how to play back in parallel from the MTS library. Many people may not know much about it. In order to make you understand better, the editor has summarized the following contents for you. I hope you can get something according to this article.

I. Summary

Unlike the playback of a single SQL thread, MTS contains multiple worker threads, and the original SQL thread is transformed into a coordination thread. The SQL orchestration thread also takes on the work of checkpoints. We know that there are two ways of parallel playback, including LOGICAL_CLOCK and DATABASE, which are reflected in the different rules for determining which things can be played back in parallel. Actually, the source code corresponds to two different classes:

Mts_submode_logical_clock

Mts_submode_database

I'm only going to talk about LOGICAL_CLOCK-based concurrency, not the old DATABASE-based approach. Here are the parameters I set:

Slave_parallel_type:LOGICAL_CLOCK

Slave_parallel_workers: 4

Note that slave_parallel_workers sets the number of worker threads and does not include coordination threads, so if you do not want to use MTS, you should set this parameter to 0 before 'stop slave;start slave' will take effect. Because the worker thread has already been initialized when it starts.

Because we know that anonymous GTID Event is included even if GTID is not enabled in 5.7. it carries last commit and seq number, so you can use MTS even if GTID is turned off, but it is not recommended to find out why in section 26 later.

Earlier, we discussed the process of transaction commit in the MySQL layer and parallel replication based on WRITESET, and we mentioned a total of three ways to generate last commit and seq number:

ORDER_COMMIT

WRITESET

WRITESET_SESSION

They control the rules for generating last commit and seq number. As long as the slave library sets the parameter slave_parallel_type to LOGICAL_CLOCK, the basis for its parallelism is last commit and seq number.

The following description is described by a normal 'Delete' statement to delete the Event of a row of data, so the order of this Event is as follows:

Event type GTID_LOG_EVENTQUERY_EVENTMAP_EVENTDELETE_EVENTXID_EVENT

At the same time, before we do that, let's identify three places in MySQL to persist MTS information, because unlike the master and slave of a traditional single SQL thread, MTS needs to store more information. Note that we only discuss situations where master_info_repository and relay_log_info_repository are TABLE, as follows:

Slave_master_ info table: updated by the IO thread, exceeding the sync_master_info setting update, per Event.

Relay_log_info_ repositor table: updates are made when a checkpoint is performed by the SQL orchestration thread.

Slave_worker_ info table: updated by the worker thread each time the transaction is committed.

See section 25 for a more detailed explanation and explain why only master_info_repository and relay_log_info_repository are considered as TABLE.

Second, the distribution mechanism of coordinating threads

The orchestration thread mainly performs the following two tasks in the distribution of the Event:

Determines whether the transaction can be played back in parallel.

Determine which worker thread plays back the transaction.

Different from the process executed by a single SQL thread, it is mainly reflected in the function apply_event_and_update_pos. For a single thread, it will complete the application of Event, while for MTS, it will only complete the distribution of Event, and the specific application will be completed by the worker thread.

Here is about the simplified process, the specific function call reference notes. The following is a flow chart (figure 19-1, the original high-definition picture is included in the original picture at the end of the article):

Third, step analysis

The following parses each step as follows:

(1) if GTID_LOG_EVENT represents the beginning of something, add this thing to the GAQ queue (GAQ is described in detail in the next section). Please refer to the function Log_event::get_slave_worker.

(2) add GTID_LOG_EVENT to the curr_group_da queue for temporary storage. Please refer to the function Log_event::get_slave_worker.

(3) get the last commit and seq number values in GTID_LOG_EVENT. Please refer to the function Mts_submode_logical_clock::schedule_next_event.

(4) get current_ LWM value, this value represents all the GAQ queue has not committed to complete the transaction in the earliest transaction of the previous committed transaction of the seq number, may be the later transaction has been committed, it may sound mouthful but very important, if all commit completed then it is to take the newly committed transaction seq number, the following figure represents this meaning, this diagram is the source code. This value can be obtained by referring to the function Mts_submode_logical_clock::get_lwm_timestamp.

The last time index containg lwm +-+ | LWM | V V VGAQ:x xoooooxxxxxXXXXX...X ^ ^ | | LWM+1 (LWM represents the location where the checkpoint points to) | + new current_lwm (here is current_lwm)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.