Shulou(Shulou.com)06/01 Report--

This article mainly explains "how to design the architecture of MySQL Binlog storage system". The content of the explanation is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "how to design the architecture of MySQL Binlog storage system".

1. Brief introduction to kingbus

1.1 what is kingbus?

Kingbus is a distributed MySQL binlog storage system based on raft strong consistency protocol. It can act as a MySQL Slave to synchronize binlog from a real Master and store it in a distributed cluster. At the same time, it acts as a MySQL Master to synchronize the binlog in the cluster to other Slave. Kingbus has the following characteristics:

Compatible with MySQL replication protocol, synchronize binlog on Master by Gtid, and support slave to pull binlog from kingbus by Gtid.

Cross-region data replication. Kingbus supports cross-region data replication through raft protocol. The binlog data written to the cluster is strongly consistent across multiple nodes and the binlog order is exactly the same as on the master.

High availability, because kingbus is built on the Raft strong consistent protocol, which can achieve the survival of more than half of the nodes in the cluster, the entire binlog pull and push service is highly available.

1.2 what problem can kingbus solve?

Kingbus can reduce the network traffic of Master. In an one-master-multi-slave replication topology, Master needs to send binlog to each slave. If there is too much slave, the network traffic is likely to reach the upper limit of the network card of Master. For example, performing operations such as delete big table or online DDL in Master may cause a large number of binlog event to be generated instantly. If 10 slave are hung under the master, the traffic of the network card on the master will be magnified 10 times. If master uses a gigabit network card, it is possible to fill its network card if it generates traffic above 10MB/S. Through the way that kingbus is connected to master, slave can be distributed to multiple machines to balance the traffic.

To simplify the Master Failover process, simply upgrade a Slave connected to the kingbus to Master, and repoint the kingbus to the new Master, while the other slave is still connected to the kingbus, and the replication topology remains the same.

Save space for Master to store binlog files. Generally speaking, there are more expensive SSD on the MySQL. If the binlog file takes up more space, the data stored in MySQL has to be reduced. You can reduce the amount of storage of binlog files on Master by storing all the binlog in kingbus

Heterogeneous replication is supported. Through Alibaba's open source canal connection to kingbus,kingbus, the binlog is continuously pushed to the canal,canal to receive the binlog and then push to the kafka message queue, and finally stored in the HBase. The business unit realizes the real-time business analysis by writing the SQL directly by Hive.

Overall architecture of 2.kingbus

The overall architecture of kingbus is shown in the following figure:

Storage is responsible for storing raft log entry and Metadata. In kingbus, raft log and mysql binlog are fused together. By distinguishing different header information, the data part of raft log is binlog event, so there is no need to store the two types of log separately and save storage space. Because kingbus needs to store some meta-information, such as raft node voting information, the specific content of some special binlog event (FORMAT_DESCRIPTION_EVENT).

Raft replicates the Lead election, log replication and other functions of the kingbus cluster, using etcd raft library.

Binlog syncer, which runs only on the Lead node of the Raft cluster, has only one syncer for the entire cluster. Syncer disguises as a slave and establishes a master-slave replication connection to the Master. Master will filter the binlog event that the syncer has accepted according to the executed_gtid_set sent by the syncer, and only send the binlog event that the syncer has not received. This set of replication protocol is fully compatible with the MySQL master-slave replication mechanism. After syncer receives the binlog event, it does some processing according to the binlog event type, and then encapsulates the binlog event into a message and submits it to the raft cluster. Through the raft algorithm, the binlog event can be stored in multiple nodes and achieve strong consistency.

Binlog server is a Master that implements the replication protocol. The real slave can be connected to the port where binlog server listens. Binlog server will send binlog event to slave. The whole process of sending binlog event is implemented according to MySQL replication protocol. When no binlog event is sent to slave, binlog server sends heartbeat event to slave periodically to keep the replication connection alive.

Api server, which is responsible for the management of the entire kingbus cluster, including the following:

Raft cluster membership operation, check cluster status, add a node, remove a node, update node information, etc.

Binlog syncer related operations, start a binlog syncer, stop the binlog syncer, and check the binlog syncer status.

Binlog server related operations, start a binlog server, stop the binlog server, and check the binlog server status. All kinds of exceptions in the server layer do not affect the raft layer, and server can be understood as a plug-in that starts and stops on demand. When you extend kingbus later, you only need to implement the server of the relevant logic. For example, if you implement a server of the kafka protocol, you can consume messages in kingbus through kafka client.

3.kingbus core implementation

3.1Core implementation of storage

There are two log forms in storage, one is the raft log (hereinafter called raft log), which is generated and used by the raft algorithm, and the other is the user form of Log (that is, mysql binlog event). In the design of Storage, two Log forms are combined into a Log Entry. It's just distinguished by different head information. Storage consists of a data file and an index file, as shown in the following figure:

Segment is of a fixed size (1GB) and can only be appended. The name is first_raft_index-last_raft_index, which indicates the raft index range of the segment.

Only the last segment is writable, its file name is first_raft_index-inprogress, and the other segment is read-only.

Both the read-only segment and the corresponding index file are written and read through mmap.

The index contents of the last segment are stored on both disk and memory. Reading the index only needs to be read in memory.

3.2 use of the etcd raft library

Etcd raft library is single-threaded when dealing with Apply logs, committed entries, and so on. Refer to the link for the specific function. The processing time of this function should be as short as possible. If the processing time exceeds the raft election time, it will cause the cluster to be re-elected. This requires special attention.

Core implementation of 3. 3 binlog syncer

The main tasks of binlog syncer are as follows:

Pull binlog event

Parse and process binlog event

Submit the binlog event to the raft cluster. It is obvious that the processing speed of the whole process can be raised through the pipeline mechanism, and the kingbus of each stage is handled by a separate goroutine, and the different stages are connected through pipes. Because binlog syncer is received one by one according to binlog event, syncer does not guarantee transaction integrity, so you may need to reconnect Master after syncer is dead. At this time, the last transaction may be incomplete, and binlog syncer needs to have the ability to discover transaction integrity. Kingbus implements the function of transaction integrity parsing, with full reference to MySQL source code.

3.4 Core implementation of binlog server

Binlog server implements the function of a master. When slave establishes a replication connection with binlog server, slave sends relevant commands, and binlog server needs to respond to these commands. Finally send binlog event to slave. For each slave,binlog server, a goroutine is started to read the raft log continuously, and the relevant header information is removed, which becomes binlog event, and then sent to the slave.

Thank you for your reading, the above is the content of "how to design the architecture of MySQL Binlog storage system". After the study of this article, I believe you have a deeper understanding of how to design the architecture of MySQL Binlog storage system. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.