Shulou(Shulou.com)05/31 Report--

This article introduces the relevant knowledge of "what are the internal data structures related to Gtid in Mysql 5.7". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

1. Gtid basic format

Single Gtid:

E859a28b-b66d-11e7-8371-000c291f347d:1

The first part is the server_uuid, and the second part is the only sign that executes things, usually self-increasing. Internally, it is represented by a data structure like Gtid, which will be described later.

Interval Gtid:

E859a28b-b66d-11e7-8371-000c291f347d:1-5

The first part is the server_uuid, and the second part is the collection of unique flags that execute things, internally represented by the Interval node corresponding to a Sidno in the Gtid_set, which will be described later.

2. Generation of server_uuid

Now that we're talking about server_uuid, let's talk about the generation of server_uuid. Server_uuid is actually a 32-byte + 1-byte (/ 0) string. When Mysql starts, it calls init_server_auto_options () to read the auto.cnf file. If you don't read it, call generate_server_uuid () to generate a server_id.

In fact, you will see in this function that server_uuid is at least related to the following parts:

1. Mysql startup time

2. Related to thread Lwp

3. Related to a random memory address

Look at the code snippet:

Const time_t save_server_start_time= server_start_time; / / get Mysql startup time server_start_time+= ((ulonglong) current_pid status_var.bytes_sent= (ulonglong) thd;// this is a memory pointer lex_start (thd); func_uuid= new (thd- > mem_root) Item_func_uuid (); func_uuid- > fixed= 1; func_uuid- > val_str (& uuid); / / there is a specific operation process in this function

After obtaining this information, you will go to Item_func_uuid::val_str to do calculation and return. Interested friends can take a closer look, and eventually generate a server_uuid and copy it to the actual server_uuid as follows:

Strncpy (server_uuid, uuid.c_ptr (), UUID_LENGTH)

Call stack frame:

# 0 init_server_auto_options () at / root/mysql5.7.14/percona-server-5.7.14-7/sql/mysqld.cc:3810#1 0x0000000000ec625e in mysqld_main (argc=97, argv=0x2e9af08) at / root/mysql5.7.14/percona-server-5.7.14-7/sql/mysqld.cc:4962#2 0x0000000000ebd604 in main (argc=10 Argv=0x7fffffffe458) at / root/mysql5.7.14/percona-server-5.7.14-7/sql/main.cc:253, internal representation of server_uuid binary_log::Uuid

Binary_log::Uuid is the internal representation of server_uuid. In fact, the core is a 16-byte memory space, as follows:

/ * The number of bytes in the data of a Uuid. * / static const size_t BYTE_LENGTH= 16; / * * The data for this Uuid. * / unsigned char Bytes [byte _ LENGTH]

Server_uuid and binary_log::Uuid can be converted to each other, and the server_uuid represented by binary_log::Uuid in Sid_map is actually its sid.

4. Class structure Gtid

This structure is an internal representation of a single Gtid whose core elements include:

/ SIDNO of this Gtid. Rpl_sidno sidno; / GNO of this Gtid. Rpl_gno gno

Gno is the only sign of what we are talking about, and sidno is actually the value in a lookup table obtained by binary_log::Uuid (sid) through the hash algorithm, which is the internal representation of server_uuid. Reference function Sid_map::add_sid this function returns a sidno based on binary_log::Uuid (sid).

5. Class structure Sid_map

Now that you're talking about the hash algorithm, you need an internal structure to store the entire hash lookup table, and use Sid_map in Mysql as such a structure that contains a variable array and an hash lookup table whose function is already given in the comments. There is only one Sid_map globally. Memory is allocated when the Gtid module is initialized. The core elements of Sid_map are as follows:

/ Read-write lock that protects updates to the number of SIDNOs. Mutable Checkable_rwlock * sid_lock; / * * Array that maps SIDNO to SID; the element at index N points to a Node with SIDNO Nmuri 1. * / Prealloced_array_sidno_to_sid; / / because sidno is a continuous number, then it is more sidno to find sid. Just do a simple / / array search to store the node pointer in / * * Hash that maps SID to SIDNO. The keys in this array are of type rpl_sid. * / HASH _ sid_to_sidno / / because sid is a data structure whose core bytes key values store 16 bytes of irregular memory space converted according to server_uuid / /, you need to use the hash lookup table to quickly locate / * * Array that maps numbers in the interval [0, get_max_sidno ()-1] to SIDNOs, in order of increasing SID. @ see Sid_map::get_sorted_sidno. * / Prealloced_array _ sorted;// an additional array about sidno, the exact role of which is unknown

Here, take a look at the type of pointer element Node in the variable array:

Struct Node {rpl_sidno sidno; / / sid hash no rpl_sid sid; / / sid}

In fact, he is a correspondence between sidno and sid.

6. Class structure Gtid_set

This structure is a general collection about a certain type of Gtid, such as the execute_gtid set and the purge_gtid set that we are familiar with. We know that the Gtid of multiple databases may exist in an execute_gtid collection, that is, multiple sidno may exist at the same time, and the occurrence interval of the Gtid of a database may exist as follows:

| | gtid_executed | 3558703b-de63-11e7-91c3-5254008768e3:1-6:20-30 da267088-9c22-11e7-ab56-5254008768e3:1-34 |

Here there are several intervals for the Gno of 3558703b-de63-11e7-91c3-5254008768e3:

1-6

20-30

In this case, the internal representation should be an array plus an interval linked list, which is, of course, implemented within Mysql. Let's look at the core elements:

/ * * Array where the N'th element contains the head pointer to the intervals of SIDNO naming 1. * / Prealloced_array massively intervalscontrol / each sidno contains an Interval single-item linked list, which is connected by a next pointer, which completes the problem such as splitting GTID / Linked list of free intervals. Interval * free_intervals; / / the free interval is connected to this linked list / Linked list of chunks. Interval_chunk * chunks; / / A global Interval linked list all Interval spaces are connected to the above 7. Gtid_set 's associated class structure Interval

It's exactly what I said earlier. The Gtid range is as follows:

Da267088-9c22-11e7-ab56-5254008768e3:1-34

His inner class is the Interval class structure. Let's take a look at the core elements:

/ The first GNO of this interval. Rpl_gno start; / The first GNO after this interval. Rpl_gno end; / Pointer to next interval in list. Interval * next

It is very simple to start Gno with a next pointer to mark an interval.

8. Class structure Gtid_state

This structure is also initialized with Sid_map when the database is started, and is also a global variable.

Several parameters that we are familiar with are as follows:

Gtid_executed

Gtid_owned

Gtid_purged

All come from this time, of course, in addition to the above, we also contain some other core elements. Let's take a look at the details:

/ The Sid_map used by this Gtid_state. Mutable Sid_map * sid_map; / / use sid_map / * * The set of GTIDs that existed in some previously purged binary log. This is always a subset of executed_gtids. * / Gtid_set lost_gtids; / / corresponds to the gtid_purged parameter, which is usually automatically maintained by Mysql unless the gtid_purged parameter / * The set of GTIDs that has been executed and stored into gtid_executed table is manually set. * / Gtid_set executed_gtids; / / corresponds to the gtid_executed parameter, which is usually actively maintained by Mysql / * The set of GTIDs that exists only in gtid_executed table, not in binlog files. * / Gtid_set gtids_only_in_table;// is always empty for the master database normally because it is impossible for the master database to have a gtid that is only in the mysql.gtid_executed table and not in the binlog, but the slave library must enable log_slave_updates and binlog to achieve this effect. / / otherwise, the Gtid of the binlog that does not contain relay can only be included in the mysql.gtid_ execute table, so Gtid_set gtids_only_in_table always exists. It will be explained later. / * The previous GTIDs in the last binlog. * / Gtid_set previous_gtids_logged;// contains all the Gtid / The set of GTIDs that are owned by some thread in binlog that have been executed by the previous binlog. All Gtid collections / The SIDNO for this server that are currently owned by all threads in Owned_gtids owned_gtids;//. Rpl_sidno server_sidno;// is the sidno9 corresponding to the sid hash of the server server_uuid and the class structure Owned_gtids.

This structure contains all the Gtid collections that are being held by the current thread. Assigning a Gtid to a transaction first adds the Gtid and thread number to the Owned_gtids and then points the thread's thd- > owned_gtid to the Gtid.

Referring to the function Gtid_state::acquire_ownership, in the final phase of commit, the Gtid removes the reference function Owned_gtids::remove_gtid from Owned_gtids and adds it to Gtid_state::executed_gtids.

This process will be described later. Its core elements include:

/ Growable array of hashes. Prealloced_array sidno_to_hash

So every sidno will have a hash structure with the content of its hash:

Struct Node {/ GNO of the group. Rpl_gno gno; / Owner of the group. My_thread_id owner;}

Such a structure, which we see contains gno and thread ID.

10. Class structure Gtid_table_persistor

This structure mainly contains some function interfaces that operate the mysql.gtid_ execute table.

It mainly includes:

Insert the gtid into table.

Int save (THD * thd, const Gtid * gtid)

Insert the gtid set into table.

Int save (const Gtid_set * gtid_set)

Delete all rows from the table.

Int reset (THD * thd)

Fetch gtids from gtid_executed table and store them into gtid_executed set.

Int fetch_gtids (Gtid_set * gtid_set)

Compress the gtid_executed table completely by employing one or more transactions.

Int compress (THD * thd)

Write a gtid interval into the gtid_executed table.

Int write_row (TABLE * table, const char * sid,rpl_gno gno_start, rpl_gno gno_end)

Update a gtid interval in the gtid_executed table.

Int update_row (TABLE * table, const char * sid,rpl_gno gno_start, rpl_gno new_gno_end)

Delete all rows in the gtid_executed table.

Int delete_all (TABLE * table)

These methods are also breakpoints for determining when to read / write mysql.gtid_executed.

10. Diagram of internal structure

In order to be able to explain the relationship between these class structures in a graphical way, I modify the mysql.gtid_ executed table and auto.cnf to construct this interval Gtid case, and add the print code at the source code to output the startup get_executed_gtids/get_lost_gtids/get_gtids_only_in_table/get_previous_gtids_logged to the log. But it is hard to see this kind of interval Gtid online.

Suppose that at some point after our database is started, the various Gtid are as follows ():

2017-12-12T04:10:42.768153Z 0 [Note] gtid_state- > get_executed_gtids:

Gtid have:3558703b-de63-11e7-91c3-5254008768e3:1-35

Da267088-9c22-11e7-ab56-5254008768e3:1-34

2017-12-12T04:10:42.768191Z 0 [Note] gtid_state- > get_lost_gtids:

Gtid have:3558703b-de63-11e7-91c3-5254008768e3:1-6:20-30

Da267088-9c22-11e7-ab56-5254008768e3:1-34

2017-12-12T04:10:42.768226Z 0 [Note] gtid_state- > get_gtids_only_in_table:

Gtid have:3558703b-de63-11e7-91c3-5254008768e3:1-6:20-30

Da267088-9c22-11e7-ab56-5254008768e3:1-34

2017-12-12T04:10:42.768260Z 0 [Note] gtid_state- > get_previous_gtids_logged:

Gtid have:3558703b-de63-11e7-91c3-5254008768e3:7-19:31-35

Immediately after startup, we execute a thing that is in the flush phase of ordered_commit and Gtid_state::acquire_ownership gets a Gtid, so it is in Owned_gtids, so the figure at this time is as follows:

This is the end of the introduction of "what are the internal data structures related to Gtid in Mysql 5.7". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.