Shulou(Shulou.com)05/31 Report--

How to analyze the core technology and implementation of RadonDB distributed database, I believe that many inexperienced people do not know what to do about it. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.

Abstract: with the gradual expansion of the scale of data, the cost of storage and operation and maintenance is gradually increasing, and the requirements of enterprises for database are also gradually increasing. Some people think that the traditional database represented by MySQL is outdated and can not meet the requirements of enterprises in the era of data explosion; the new database represented by NoSQL still has limitations and does not have ACID characteristics, so it is difficult to meet the stringent needs of enterprises for core business data. The shared RadonDB is a new generation of distributed relational database that combines distributed consistency protocol Raft with MySQL. The most important feature of RadonDB is to combine the advantages of the two types of databases to make DBA seamlessly transition from traditional, stand-alone MySQL to cloud-native, infinitely extended distributed relational databases.

RadonDB architecture diagram

RadonDB positioning is a new generation of distributed relational database. In today's era of rapid development of cloud computing, we prefer to call it a cloud native database. The natural properties of cloud native database are distributed, scalable, highly available and strong consistent. The following MyNewSQL is because we combine MySQL and NewSQL to meet the above features.

After one of the requests arrives, the SQL layer parses the request into a distributed execution plan, and then executes the SQL in parallel on the back-end storage layer. And it will also do a secondary operation, that is, Orderby/limit/groupby/aggration/join. . Wait. It is also a decentralized design, easy to deploy and easy to expand, which is a design of the entire SQL layer.

Storage Nodes

Looking at the storage layer, the storage layer is a relatively novel design in RadonDB, and it is also one of the reasons why we are positioned as a new generation of distributed databases.

The first challenge, how to choose the master quickly?

The second challenge is how to play back the data quickly after selecting a new master.

The third challenge is, after data playback, how to ensure that the data will not be lost? How did RadonDB solve it?

For the first challenge, we used the Raft protocol, you know, the Raft protocol mainly does two things, the first is to choose the host, and the second is data synchronization. In RadonDB, only Raft is used to select the master. When the master is dead, we use Raft to select the new master, and then synchronize the data. We are based on MySQL's native FGTID mechanism and parallel replication for fast playback.

To ensure that the data is not lost, we are still based on MySQL, and use MySQL's Semi-Sync mechanism. When the user writes to the master copy, it first reaches a slave copy, and then feeds back to the client after receiving the slave copy. This ensures that the data of a slave copy is completely consistent with that of the master copy, and the slave node is selected into the new master node to ensure that the data is not lost.

After being elected to the new master node, the Log parallel replication of RadonDB is still based on the mechanism of MySQL, parallel replication and rapid playback, which is equivalent to the combination of Raft master selection and Log parallel replication. In the original ecological Raft protocol, it is difficult to play back these two in parallel with Raft Log, but we have completed them well with MySQL. Moreover, these three copies are a decentralized design, and its deployment is more flexible as long as we can reach it.

Expand capacity

The underlying layer of RadonDB is all based on MySQL, so it is convenient to expand the capacity, because MySQL has a set of tools and mechanisms.

As you can see in the figure above, the MySQL Xa mechanism actually has a total of five steps, but in RadonDB, we have abstracted it, that is, encapsulated it. We implemented the snapshot isolation level and the Snapshot Isolation transaction isolation level.

Binlog

In addition, RadonDB supports Binlog, you may think that a distributed database needs to put some huge amounts of data in it, but once the data enters your distributed database, how can it come out? It is troublesome, so RadonDB provides a Binlog mechanism that allows data to be processed synchronously quickly.

OLTP + OLAP

For example, we have a cluster of OLAP, and the Binlog,Binlog that can be set to RadonDB is updated in real time, which completes an isomerization process and data outflow, and is real-time.

As you can see, in the architecture diagram just now, there is a computing node in the lower right corner. In fact, our computing node synchronizes with RadonDB data through this mechanism. In this way, OLTP and OLAP are combined. When a more complex query arrives at RadonDB, RadonDB will be sent to TP node or AP node according to the mode of SQL. Front-end users are not aware of it, so some resources are isolated. Of course, there is a drawback. There may be two copies of data, but at present, we do it through isomerization, column storage and high compression.

Audit log

In addition, RadonDB also provides some audit log functions, in order to facilitate people to conduct some audits of the business, and the audit mechanism can also locate some slow query SQL, because the distributed database has more nodes, so locating a SQL will be more complex. With audit logs, you can locate some slow SQL.

Backup and recovery

RadonDB provides a complete set of backup and recovery tools that allow users to quickly stream data in and out much faster than native ones.

Performance

Mysql > show processlist

The first life is commonly used by MySQL, which indicates the status of the user's link to RadonDB.

Mysql > show txnz

The second command is that you can clearly see at which stage the distributed transaction executes and how much time it takes.

Mysql > show queryz

The last command, which clauses fall to which nodes, and even how much time it takes, for example, some problems with a node, can be reflected from this, which is more flexible.

After reading the above, have you mastered how to parse the core technology and implementation of RadonDB distributed database? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.