Shulou(Shulou.com)06/01 Report--

Address of the brief book:

Http://www.jianshu.com/p/fc836446cde0

This section is also an important part, and the later failure cases are also related to this section. This section details the initialization of the Gtid module and when to read the two Gtid persistence media we mentioned earlier:

Binlog file mysql.gtid_ execute table

In addition, it will describe how they are read.

At the same time, I will analyze this step and divide it into two situations in the key steps:

The main library opens Gtid and opens binlog. Enable Gtid from the library to enable binlog without enabling the log_slave_updates parameter.

Because these two ways that we usually set up, hereinafter referred to as the master library and the slave library.

Initialize the global variable memory space of the Gtid module

First initialize Gtid several Global memory spaces including Gtid_state\ Sid_map\ gtid_table_persistor

This call is called into gtid_server_init () by mysqld.cc.

If (init_server_components ()) unireg_abort (MYSQLD_ABORT_EXIT)

Init_server_components () initializes many modules. Gtid is just a small one, and Innodb is initialized here.

The fragment of the gtid_server_init () function is as follows:

(! (global_sid_lock= new Checkable_rwlock (# ifdef HAVE_PSI_INTERFACE key_rwlock_global_sid_lock#endif)) | |! (gtid_mode_lock= new Checkable_rwlock (# ifdef HAVE_PSI_INTERFACE) Key_rwlock_gtid_mode_lock#endif)) | (global_sid_map= new Sid_map (global_sid_lock)) | | / / new a memory Sid_map memory space out! (gtid_state= new Gtid_state (global_sid_lock) Global_sid_map)) | | / / new a memory Gtid_state memory space! (gtid_table_persistor= new Gtid_table_persistor () / / new A memory Gtid_table_persistor memory space comes out. 2. Initialize the server_uuid of the database.

This initialization process mentioned earlier is nothing more than getting server_uuid through my.cnf. If not, it will be regenerated. For more information, please refer to the previous description here.

If (init_server_auto_options ()) {sql_print_error ("Initialization of the server's UUID failed because it could"not be read from the auto.cnf file. If this is a new "" server, the initialization failed because it was not "" possible to generate a new UUID. "); unireg_abort (MYSQLD_ABORT_EXIT);} III. Initialize Gtid_state global structure global_sid_lock- > rdlock (); int gtid_ret= gtid_state- > init (); / / add sid (Uuid) and sidno corresponding to server_uuid to Sid_map. Global_sid_lock- > unlock (); if (gtid_ret) unireg_abort (MYSQLD_ABORT_EXIT)

In fact, this step is also completed to join the sidno Sid_map, interested can refer to the int Gtid_state::init () function logic is very simple.

4. Read mysql.gtid_ execute table

This step begins to read our first Gtid persistence media mysql.gtid_executed table, and its final call is Gtid_table_persistor::fetch_gtids (Gtid_set * gtid_set). The principle is to add the contents of the read mysql.gtid_executed table row by row to the Gtid_state.executed_gtids. Let's take a look at the source code:

/ / Initialize executed_gtids from mysql.gtid_executed table. If (gtid_state- > read_gtid_executed_from_table () =-1) unireg_abort (1)

Gtid_state::read_gtid_executed_from_table is just a simple package as follows:

Int Gtid_state::read_gtid_executed_from_table () {return gtid_table_persistor- > fetch_gtids (& executed_gtids);}

Next, take a look at the logic snippet of the Gtid_table_persistor::fetch_gtids (Gtid_set * gtid_set) function.

If ((err= table- > file- > ha_rnd_init (true)) {ret=-1; goto end;} while (! (err= table- > file- > ha_rnd_next (table- > record [0])) / / start reading data {/ * Store the gtid into the gtid_set * / / * * @ todo:-take only global_sid_lock- > rdlock (), and take gtid_state- > sid_lock for each iteration. -Add wrapper around Gtid_set::add_gno_interval and call that instead. * / global_sid_lock- > wrlock (); if (gtid_set- > add_gtid_text (encode_gtid_text (table). C_str ())! / add the read line Gtid interval to the Gtid_state.executed_gtids here. RETURN_STATUS_OK) {global_sid_lock- > unlock (); break;} global_sid_lock- > unlock ();}

After completing this step, Gtid_state.executed_gtids will be set up. The settings of master library and slave library are different.

The main library because mysql.gtid_executed contains only all the Gtid of the previous binlog, it does not contain the Gtid of the latest binlog, so you need to correct this value in step 7. The slave library contains all the Gtid because the mysql.gtid_executed is updated in real time. Define intermediate variables and get pointers

This step is a non-critical step but defines some intermediate variables and defines four pointers to obtain the addresses of the four memory variables of Gtid_ state, which is convenient for operation.

If (opt_bin_log) / / if binlog turns on {/ * Initialize GLOBAL.GTID_EXECUTED and GLOBAL.GTID_PURGED from gtid_executed table and binlog files during server startup. * / Gtid_set * executed_gtids= const_cast (gtid_state- > get_executed_gtids ()); / / get the Gtid_state.executed_gtids pointer Gtid_set * lost_gtids= const_cast (gtid_state- > get_lost_gtids ()); / / get the gtid_state.get_lost_gtids pointer Gtid_set * gtids_only_in_table= const_cast (gtid_state- > get_gtids_only_in_table ()) / get gtid_state.get_lost_gtids pointer Gtid_set * previous_gtids_logged= const_cast (gtid_state- > get_previous_gtids_logged ()); / get gtid_state.previous_gtids_logged pointer Gtid_set purged_gtids_from_binlog (global_sid_map, global_sid_lock); / / define temporary variables to store Gtid things scanned from binlog to discarded objects. Gtid_set gtids_in_binlog (global_sid_map, global_sid_lock); / / defines all Gtid things contained in the intermediate variable binlog, including discarded ones. Gtid_set gtids_in_binlog_not_in_table (global_sid_map, global_sid_lock); / / defines Gtid things that have existed in binlog without intermediate variables in the table, / / obviously the master library contains such a collection because the gtids_in_binlog of the master library is > gtids_only_in_table, and the slave library also does not contain such a collection because all Gtid things of the slave library are in the table. 6. Read binlog files

In this step, we will read the second Gtid persistence media binlog mentioned by first reverse reading to get gtids_in_binlog and then forward reading to get purged_gtids_from_binlog, and here the forward reading purged_gtids_from_binlog will be affected by the binlog_gtid_simple_recovery parameter. At the same time, Previous gtid Event in our previous description of 5.7will also include this event in the binlog that does not have Gtid enabled, which will reflect its value in this book.

If (& gtids_in_binlog, & purged_gtids_from_binlog, opt_master_verify_checksum, true/*true=need lock*/, NULL/*trx_parser*/) NULL/*gtid_partial_trx*/, true/*is_server_starting*/))

We found that he was actually calling the bool MYSQL_BIN_LOG::init_gtid_sets () function. Let's move on to the important code snippet of this function:

List filename_list; / / define a string list to store the file name LOG_INFO linfo; int error; list::iterator it;// define a forward iterator for list list::reverse_iterator rit;// define a reverse iterator for list for (& linfo, NULL, false/*need_lock_index=false*/);! error / / this part actually adds all the file names to the list error= find_next_log (& linfo, false/*need_lock_index=false*/)) {DBUG_PRINT ("info", ("read log filename'% s'", linfo.log_file_name)); filename_list.push_back (string (linfo.log_file_name)) } if (error! = LOG_INFO_EOF) {DBUG_PRINT ("error", ("Error reading% s index", is_relay_log? "relaylog": "binlog"); goto end;} if (all_gtids! = NULL) / / all_gtids will not be NULL in the case of database startup initialization, but NULL {rit= filename_list.rbegin () will be passed if deleting binlog log all_gtid such as purge binary logs command; / / reverse iterator points to list tail bool can_stop_reading= false; reached_first_file= (rit= = filename_list.rend ()) / / if there is only one binlog, true while (! can_stop_reading & &! reached_first_file) / / start reverse loop scan to get the gtids_in_binlog (all_gtids) collection {const char * filename= rit- > c_str (); / / get the file name rit++; reached_first_file= (rit = = filename_list.rend ()) / / if the first file is reached, true means scanning completed switch (read_gtids_from_binlog (filename, all_gtids, reached_first_file? Lost_gtids: NULL, NULL/* first_gtid * /, sid_map, verify_checksum, is_relay_log) / / read the binlog file {case ERROR: {error= 1; goto end) through the function read_gtids_from_binlog } case GOT_GTIDS: / / if the scanned binlog has PREVIOUS GTID EVENT and GTID EVENT, the break jumps out of the loop and sets can_stop_reading= true {can_stop_reading= true; break } case GOT_PREVIOUS_GTIDS:// if there is only PREVIOUS GTID EVENT in this binlog, enter the logical judgment {if (! is_relay_log) / / We only consider that binlog will not be relaylog, then break jumps out of the loop and sets can_stop_reading= true, / / notice that it is not affected by the binlog_gtid_simple_recovery parameter. We know that every binlog after 5.7.5 contains PREVIOUS GTID EVENT. In fact, even if GTID is not turned on, it will jump out of the loop, but only scan the last binlog file, can_stop_reading= true. Break } case NO_GTIDS: / / if PREVIOUS GTID EVENT and GTID EVENT are not found, do the following logic. In fact, this problem cannot occur after 5.7, because it must include PREVIOUS GTID EVENT / / even if GTID is not enabled. So the reverse lookup must jump out of the loop {if (binlog_gtid_simple_recovery & & is_server_starting & &! is_relay_log) / / after scanning the last file, which is affected by the binlog_gtid_simple_recovery parameter, but we know that this branch will not be executed. Unless the database is upgraded and Gtid {DBUG_ASSERT (all_gtids- > is_empty ()) is not enabled; / / asserts that all_gtids still hasn't found DBUG_ASSERT (lost_gtids- > is_empty ()); / / asserts that lost_gtids still hasn't found goto end. / / finish the scan. From here, we find that if mysql is upgraded, we must pay attention to this problem. Setting binlog_gtid_simple_recovery may not get the correct GTID. For upgrading / / it is best to use master-slave to upgrade to avoid this risk. } / * FALLTHROUGH*/} case TRUNCATED: {break The next step is to get purged_gtids_from_binlog (lost_gtids) if (lost_gtids! = NULL & &! reached_first_file) / / if the previous scan has not scanned all the binlog, this is actually affirmative in 5.7. {for (it= filename_list.begin (); it! = filename_list.end (); it++) / / do a forward search {/ * We should pass a first_gtid to read_gtids_from_binlog when binlog_gtid_simple_recovery is disabled, or else it will return right after reading the PREVIOUS_GTIDS event to avoid stall on reading the whole binary log. * / Gtid first_gtid= {0,0}; const char * filename= it- > c_str (); / / get the file name pointer switch (read_gtids_from_binlog (filename, NULL, lost_gtids, binlog_gtid_simple_recovery? NULL: & first_gtid, sid_map, verify_checksum, is_relay_log) {case ERROR: {error= 1 / * FALLTHROUGH*/} case GOT_GTIDS: / / if the scanned binlog has PREVIOUS GTID EVENT and GTID EVENT, jump out of the loop and go straight to end {goto end } case NO_GTIDS: / / if the binlog does not contain GTID EVENT and PREVIOUS GTID EVENT, the processing logic is consistent case GOT_PREVIOUS_GTIDS: {if (binlog_gtid_simple_recovery) / / here is binlog_gtid_simple_recovery. If you set it to ON, you will actually goto end after 5.7, / / PREVIOUS GTID EVENT is a hit and you can get the correct result, but if it is upgraded from / * FALLTHROUGH*/ then binlog does not contain PREVIOUS GTID EVENT, then purged_gtids_from_binlog (lost_gtids) gets null / / if GTID is closed in 5.7, In this case, although PREVIOUS GTID EVENT hits, it still / / will not jump out of the loop goto end and continue to the next file scan. } case TRUNCATED: {break;}

So far, we have analyzed reverse lookup and forward lookup. Our code comments also show the role of binlog_gtid_simple_recovery. With the support of PREVIOUS GTID EVENT, this parameter is set to true by default after 5.7.6. If you set binlog_gtid_simple_recovery to flase when Gtid is turned off, you may need to scan a large number of binlog to determine the purged_gtids_from_binlog collection, which may occur in two places:

When Mysql is restarted, as discussed here. As discussed earlier, when the binlog is deleted at the time set by purge binary logs to or the operation parameter expire_logs_days.

This is also the reason for the emergence of the second case I described later.

Normally, our gtids_in_binlog and purged_gtids_from_binlog have been obtained here:

The main library gtids_in_binlog contains all the latest Gtid things (including discarded ones), while purged_gtids_from_binlog contains Gtid things that have been discarded. There is no binlog at all from the slave library, so gtids_in_binlog and purged_gtids_from_binlog are empty collections. VII. Amendments to Gtid_state.executed_gtids and mysql.gtid_ executed tables

As described in step 4, the Gtid_state.executed_gtids obtained by the main database by reading the mysql.gtid_executed table is not up-to-date, so the collation needs to be corrected. The code is as follows:

If (! gtids_in_binlog.is_empty () & & / / if gtids_in_binlog is not empty, the slave library is empty. Here is the main library's amendment to Gtid_state.executed_gtids! gtids_in_binlog.is_subset (executed_gtids)) / / and executed_gtids is a subset of gtids_in_binlog {gtids_in_binlog_not_in_table.add_gtid_set (& slave). If (! executed_gtids- > is_empty ()) gtids_in_binlog_not_in_table.remove_gtid_set (executed_gtids) / / add GTID and gtids_in_binlog-executed_gtids that are not in the table to gtids_in_binlog_not_in_table if (gtid_state- > save (& gtids_in_binlog_not_in_table) = =-1) / / here store the Gtid collection gtids_in_binlog_not_in_table in the mysql.gtid_executed table to complete the correction {global_sid_lock- > unlock (); unireg_abort (MYSQLD_ABORT_EXIT) } executed_gtids- > add_gtid_set (& gtids_in_binlog_not_in_table); / / finally add the gtids_in_binlog_not_in_table to the executed_gtids. The finished executed_gtids is the latest Gtid_set, and the correction of the Gtid_state.executed_gtids is completed.

This step is entirely the logic that will be triggered by the main library:

When the main library completes this step, both the Gtid_state.executed_gtids and mysql.gtid_executed tables will be modified to the latest Gtid collection. The slave library does not do this logic, because the Gtid_state.executed_gtids and mysql.gtid_executed tables of the slave library are already up-to-date.

At this point, the Gtid_state.executed_gtids, that is, our gtid_executed variable initialization has been completed, and the mysql.gtid_ executed table has been modified.

8. Initialize Gtid_state.gtids_only_in_table

Since you have obtained the complete Gtid_state.executed_gtids collection in the previous step, you only need a simple gtids_only_in_table= executed_gtids-gtids_in_binlog subtraction to get the Gtid_state.gtids_only_in_table here.

/ * gtids_only_in_table= executed_gtids-gtids_in_binlog * / if (gtids_only_in_table- > add_gtid_set (executed_gtids)! = / / add executed_gtids to gtids_only_in_table RETURN_STATUS_OK here) {global_sid_lock- > unlock (); unireg_abort (MYSQLD_ABORT_EXIT);} gtids_only_in_table- > remove_gtid_set (& gtids_in_binlog) / / gtids_in_binlog will be removed here

In this step, the master and slave libraries are as follows:

The main library is Gtid_state because Gtid_state.executed_gtids is up-to-date and gtids_in_binlog is up-to-date. Gtids_only_in_table is an empty collection. Slave library because Gtid_state.executed_gtids is up to date but gtids_in_binlog is an empty collection, so Gtid_state. Gtids_only_in_table is equal to Gtid_state.executed_gtids. 9. Initialize Gtid_state.lost_gtids

This step starts to get Gtid_state.lost_gtids, that is, our gtid_purged variable. Here, simply use Gtid_state.gtids_only_in_table + purged_gtids_from_binlog;, which they have already obtained.

/ * lost_gtids = executed_gtids-(gtids_in_binlog-purged_gtids_from_binlog) = gtids_only_in_table + purged_gtids_from_binlog * / if (lost_gtids- > add_gtid_set (gtids_only_in_table)! = RETURN_STATUS_OK | | / add the collection gtids_only_in_table to lost_gtids lost_gtids- > add_gtid_set (& purged_gtids_from_binlog)! = / add purged_gtids_from_binlog to this collection RETURN_STATUS_OK) {global_sid_lock- > unlock () Unireg_abort (MYSQLD_ABORT_EXIT);}

In this step, the master and slave libraries are as follows:

The main library is because Gtid_state.gtids_only_in_table is an empty collection, and purged_gtids_from_binlog is the Gtid of the first binlog Previous gtid Event obtained. So normally Gtid_state.lost_gtids is equal to the Gtid of the binlog Previous gtid Event of the first binlog. Since Gtid_state.gtids_only_in_table and Gtid_state.executed_gtids are equivalent and purged _ gtids_from_binlog is an empty collection, the Gtid_state.lost_gtids of the slave library is normally equal to Gtid_state.executed_gtids. That is, the gtid_purged variable is equal to the gtid_executed variable.

At this point, the gtid_purged variable and gtid_executed variable as well as the mysql.gtid_executed table have been initialized.

10. Initialize Gtid_state.previous_gtids_logged

This value has no variables to see, and it represents all the binlog Gtid contained up to the previous binlog.

/ * Prepare previous_gtids_logged for next binlog * / if (previous_gtids_logged- > add_gtid_set (& gtids_in_binlog)! = / / obviously add this collection of scanned gtids_in_binlog. RETURN_STATUS_OK) {global_sid_lock- > unlock (); unireg_abort (MYSQLD_ABORT_EXIT);}

Obviously, because binlog switches at startup, you can simply add scan to gtids_in_binlog to the collection.

In this step, the master and slave libraries are as follows:

The main library gtids_in_binlog contains all Gtid things. So gtid_state.previous_gtids_logged contains all the Gtid things in binlog. But then the binlog switch will be done. The slave library gtids_in_binlog is empty, and obviously gtid_state.previous_gtids_logged is also empty. Summary of this section

By reading mysql.gtid_executed and binlog, and then after a series of operations, our Gtid module is initialized. All four memory variables and mysql.gtid_executed have been initialized, which are summarized as follows:

The mysql.gtid_executed table master library is read in the fourth step and initialized in the seventh step of the correction, which contains all the existing Gtid things. Read from the library in step 4, because the mysql.gtid_executed from the library is already up to date and does not need to be changed. Gtid_state.executed_gtids and it correspond to the variable gtid_executed main library initializes Gtid_state.executed_gtids through steps 4 and 7, which contains all Gtid things. The library has been fully initialized through step 4, and it contains all the existing Gtid things. The Gtid_state.gtids_only_in_table main library is initialized through the eighth step, and the main library Gtid_state. Gtids_only_in_table is an empty collection. The slave library is initialized through step 8, and the slave library Gtid_state. Gtids_only_in_table is equal to Gtid_state.executed_gtids. Gtid_state.lost_gtids, which corresponds to the variable gtid_purged. The main library is initialized through step 9, and Gtid_state.lost_gtids is the Gtid of the first binlog Previous gtid Event. The main library is initialized through step 9, where Gtid_state.lost_gtids equals Gtid_state.executed_gtidsGtid_state.previous_gtids_logged. The master library is initialized by step 10, and the gtid_state.previous_gtids_logged contains all the Gtid things in binlog. The slave library is initialized by step 10, and the gtid_state.previous_gtids_logged is an empty collection.

Note that step 5 of this section includes the method of reading binlog files and the role of the binlog_gtid_simple_recovery parameter.

At the end of this section, you can at least learn:

1. How and when mysql.gtid_executed is read. 2. When to read Gtid in binlog file. 3. The initialization process and details of the whole Gtid module. 4. The function of binlog_gtid_simple_recovery parameter.

Author Wechat:

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.