Shulou(Shulou.com)06/02 Report--

Preface

What is DDM? This is the distributed database middleware launched by Huawei Cloud Paas. DDM (Distributed Database Middleware) is a server that implements the Mysql protocol stack. Front-end users can regard it as a database agent, which can be accessed by Mysql client tools and command line, while the DDM backend connects to one or more Mysql Server. Therefore, the DDM itself does not store data, which is stored on the back-end connected Mysql Server. To put it simply, DDM is the best companion for Mysql, giving Mysql the ability to compete with Oracle to some extent. In this way, it sounds like DDM is the auxiliary god of Mysql. Here, it is necessary to understand the background of DDM development under the gossip and understand the past life history of DDM.

The dispute between NoSQL and SQL

At present, there are often scenarios of high concurrency and storage of massive data on the Internet. for data storage, there are relational databases based on SQL (Structured Query Language) and non-relational databases based on NoSQL (Not Only SQL). SQL data is stored in a table with a specific structure; NoSQL is more flexible and extensible and can be stored in JSON documents, hash tables, or other ways, such as key-value storage (redis,ROMA,Memcached) database applications in ranking updates, session saving, document-oriented databases (mongoDB, couchDB) applications in logging, column-oriented databases (Cassandra, HBase) in blog applications. And the speed is extremely high.

Although in the era of cloud computing, the traditional database has congenital disadvantages, but the NoSQL database can only be used as a supplement to the traditional data and can not be replaced.

Performance bottleneck of relational database

At present, most databases do not have the ability of automatic expansion and automatic migration, and there is an obvious bottleneck for the performance of a single database and a single table. when the amount of data of a single table exceeds 1000W, the performance will decline obviously. In order to solve this problem, the common way is to divide the database and table to avoid it.

Vertical sub-database: the data can be divided into different databases according to business modules.

Horizontal split library: we found that the data volume of some tables has decreased significantly over 1000W QPS. Here, this table can be further split horizontally to make multiple shard tables. Most e-commerce enterprises will slice according to the date, but some problems arise. There is a great pressure on one shard in a fixed period of time, and the other shards are idle at this time. In the scenario of high concurrency of e-commerce enterprises, a large amount of data will be generated, which will lead to the bottleneck of a single database QPS in a short time.

Optimization of splitting rules: data can be evenly distributed to multiple slices according to the way of splitting fields and taking models. It can ensure that the QPS of the whole system can be improved, but it also gives rise to some problems: the problem that the number of fragments is not enough, and the consistency of cross-fragment transactions cannot be guaranteed.

Database middleware

In order to reduce the intrusiveness to the application, and to put the pressure on the middleware, the middleware is responsible for dealing with the upper application, which can be represented as an independent database, while shielding the underlying complex system details. In addition to the basic functions of sub-table and sub-database, distributed database middleware can also be enriched, such as the integration of read-write separation or horizontal expansion functions, or for example, read-write separation itself can also be used as an independent middleware. (DDM,Cobar,MyCAT, TDDL, etc.)

The problems of the expansion of insufficient number of shards and the consistency of distributed transactions mentioned above have been solved in Huawei Cloud's DDM, and the functions of middleware have been continuously enriched, which has become an absolute weapon for high concurrency applications.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.