Shulou(Shulou.com)05/31 Report--

Editor to share with you MySQL slaveend Retrieved _ Gtid_Set read improved example analysis, I believe that most people do not know much about it, so share this article for your reference, I hope you will learn a lot after reading this article, let's go to know it!

I. the origin of the problem

Today, my friend asked me a question @ K.I.S.S, which is described in the official document:

When using GTIDs, the slave tells the master which transactions it has already received, executed, or both. To compute this set, it reads the global value of gtid_executed and the value of the Retrieved_gtid_set column from SHOW SLAVE STATUS. The GTID of the last transmitted transaction is included in Retrieved_gtid_set only when the full transaction is received. The slave computes the following set:UNION (@ @ global.gtid_executed, Retrieved_gtid_set) Prior to MySQL 5.7.5, the GTID of the last transmitted transaction was included in Retrieved_gtid_set even if the transaction was only partially transmitted, and the last received GTID was subtracted from this set. (Bug # 17943188) Thus, the slave computed the following set: UNION (@ @ global.gtid_executed, Retrieved_gtid_set-last_received_GTID)

Ask why you subtract last_received_GTID.

2. Check the commit of Bug # 17943188

For this problem, I first took a look at what the bug fixed as follows:

BUG#17943188 SHOW SLAVE STATUS/RETRIEVED_GTID_SET MAY HAVE PARTIAL TRX OR MISS COMPLETE TRX Problem: = The SHOW SLAVE STATUS command contains the column RETRIEVED_GTID_SET. This is supposed to contain the set of GTIDs that exist in the relay log. However, the field is updated when the slave receiver thread (O thread) receives a Gtid_log_event, which happens at the beginning of the transaction. If the I/O thread gets disconnected in the middle of a transaction, RETRIEVED_GTID_SET can contain a GTID for a transaction that is only partially received in the relay log. This transaction will subsequently be rolled back, so it is wrong to pretend that the transaction is there. Typical fail-over algorithms use RETRIEVED_GTID_SET to determine which slave has received the most transactions to promote the slave to a master. This is true for e.g. The mysqlfailover utility. When RETRIEVED_GTID_SET can contain partially transmitted transactions, the fail-over utility can choose the wrong slave to promote. This can lead to data corruption later. This means that even if semi-sync is enabled, transactions that have been acknowledged by one slave can be lost. Fix: = = It was implemented a transaction boundaries parser that will give information about transaction boundaries of an event stream based on the event types and their queries (when they are Query_log_event). As events are queued by the I/O thread, it feeds the Master_info transaction boundary parser. The slave I/O recovery also uses the transaction parser to determine if a given GTID can be added to the Retrieved_Gtid_Set or not. When the event parser is in GTID state because a Gtid_log_event was queued, the event's GTID isn't added to the retrieved list yet. It is stored in an auxiliary GTID variable. After flushing an event into the relay log, the IO thread verifies if the transaction parser is not inside a transaction anymore (meaning that the last event of the transaction has been flushed). If transaction parser is outside a transaction, the I/O thread verifies if a GTID was stored in the start of the transaction, adding it to the retrieved list, ensuring that all the transaction has arrived and was flushed to the relay log. Also, before this patch, after the I/O thread flushed a single received event into the relaylog, it was possible to rotate the relaylog if the current relaylog file size exceeded max_binlog_size/max_relaylog_size. After this patch, when GTIDs are enabled we only allow this rotation by size if the transaction parser is not in the middle of a transaction. Note: The current patch removed the changes for BUG#17280176, as it also dealt with similar problem in a different way.

Roughly, that is to say, for some Gtid transactions that are not fully transmitted by the RETRIEVED_GTID_SET O thread, it will cause more serious problems, because some monitoring tools use this to determine whether to switch or not, so we added a transaction boundary analyzer to determine whether the thing is fully transmitted. If the complete transmission is recorded in the RETRIEVED_GTID_SET, it was added after 5.7.5. In general, in order to fix two problems:

1. Whether the last gtid thing is complete.

2. Get the correct relay log set across the binlog of multiple gtid.

Third, what has been modified

In fact, there are a lot of changes, not one by one. If you are interested, you can take a look at commit 9dab9dad975d09b8f37f33bf3c522d36fdf1d0f9. Here are a few places I have seen.

1. Add the following logic to MYSQL_BIN_LOG::init_gtid_sets

/ * If we use GTIDs and have partial transactions on the relay log, must check if it ends on next relay log files. We also need to feed the boundary parser with the rest of the relay log to put it in the correct state before receiving new events from the master in the case of GTID auto positioning be disabled. * / if (is_relay_log) {/ * Suppose the following relaylog: rl-bin.000001 | rl-bin.000002 | rl-bin.000003 | rl-bin-000004-+- PREV_GTIDS | PREV_GTIDS (empty) | (UUID:1) | (UUID:1) | (UUID:1)-+- -- GTID (UUID:1) | QUERY (INSERT) | QUERY (INSERT) | XID-+-QUERY (CREATE | TABLE T1.) |- -+ GTID (UUID:2) |-+ QUERY (BEGIN) |-+ As it is impossible to determine the current Retrieved_Gtid_Set by only looking to the PREVIOUS_GTIDS on the last relay log file And scanning events on it, we tried to find a relay log file that contains at least one GTID event during the backwards search. In the example, we will find a GTID only in rl-bin.000001, as the UUID:2 transaction was spanned across 4 relay log files. The transaction spanning can be caused by "FLUSH RELAY LOGS" commands on slave while it is queuing the transaction. So, in order to correctly add UUID:2 into Retrieved_Gtid_Set, we need to parse the relay log starting on the file we found the last GTID queued to know if the transaction was fully retrieved or not. * Adjust the reverse iterator to point to the relaylog file we need to start parsing, as it was incremented after generating the relaylog file name. * / rit--;// rollback a file because the previous operation exp:1 for rit++

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.