Shulou(Shulou.com)06/01 Report--

Mycat Learning practice-first acquaintance of Mycat

Learning mycat mysql

Mycat Learning practice-first acquaintance of Mycat

1. What is Mycat?

2. The difference between Mycat and other middleware

2.1 Mango

2.2 Cobar

2.3 Heisenberg

2.4 Atlas

2.5 Ameoba

3. Problems that Mycat can solve

4. Core concepts of Mycat

4.2.1 sharding table

4.2.2 non-fragmented table

4.2.3 ER table

4.2.4 Global tabl

4.1 logical Library (schema)

4.2 logical tables (table)

4.3 sharding Node (dataNode)

4.4 Node Host (dataHost)

5. Introduction of Mycat folders and files

1. What is Mycat?

Mycat is a thoroughly open source large database cluster for enterprise application development, which supports transactions and ACID. It is an enhanced database that can replace MySQL. Mycat is regarded as the enterprise database of MySQL cluster, which is used to replace the expensive Oracle cluster. It is a new type of SQL Server which integrates memory cache technology, NoSQL technology and HDFS big data. It is a new generation of enterprise database product that combines traditional database and new distributed data warehouse. It is also an excellent database middleware.

2. The difference between Mycat and other middleware

At present, there are many database middleware.

2.1 Mango

Mango, whose Chinese name is "Mango", is a lightweight high-speed data-layer access framework that has been used by more than a dozen large online projects. It is said that a payment system uses Mango to carry 120000 payment requests per second, and its ultra-high performance and ultra-fast response speed is almost equivalent to using JDBC directly. It defines DAO in the form of interfaces and annotations, perfectly combines db and cache operations, supports dynamic SQL and can construct arbitrarily complex SQL statements, supports multiple data sources, sub-tables, sub-libraries, transactions, and embedded "function call" function, which can map any complex objects to the tables of the database. But on the whole, Mango is a Java Dao Framework and a jar package, its operation depends on the project code and server of the application system, adopts the idea of JDBC Shard, and is the same product as TDDL.

2.2 Cobar

Cobar is an upgraded version of Amoeba, a distributed processing system for relational data developed by Alibaba. This product successfully replaces the original data storage scheme based on Oracle. At present, it has taken over the scheme of more than 3000 MySQL databases, handling an average of nearly 5 billion SQL execution requests per day.

2.3 Heisenberg

Heisenberg originates from Cobar and combines the advantages of Cobar and TDDL to change its slicing strategy into a database and table strategy, saving a lot of connections. The advantage is that the sub-database sub-table is separated from the application, and the sub-database sub-table is the same as using a single database table, which reduces the pressure on the number of db connections, adopts hot restart configuration, can be horizontally expanded, follows the native MySQL protocol, adopts read-write separation, and has no language restrictions. MySQL Client, C, Java, and so on can all use the Heisenberg server to view and adjust the number of connections, thread pools, nodes, and so on through administrative commands. Heisenberg uses velocity's sub-database and sub-table script to customize the sub-database and sub-table, which is quite flexible.

2.4 Atlas

Atlas is a database middle-tier project based on MySQL protocol developed and maintained by Qihoo 360's Web platform infrastructure team. It is optimized on the basis of MySQL-proxy version 0.8.2, adding some new features and features. Qihoo 360 uses Atlas to run the MySQL business, which carries billions of read and write requests every day. Atlas is located between the application and the MySQL, which implements the client-server protocol of MySQL, which communicates with the application as the server, and communicates with MySQL as the client. It shields the details of db from the application and maintains connection pooling in order to reduce the burden on MySQL.

2.5 Ameoba

Ameoba is a Proxy which takes MySQL as the underlying data storage and provides MySQL protocol interface to the application. It focuses on responding to application requests and sends SQL requests to a specific database for execution according to the rules set by users in advance. Based on this, it can achieve load balancing, read-write separation, high availability and other requirements. Compared to MySQL's official MySQL Proxy, Ameoba's authors emphasize the ease of Amoeba configuration (XML-based configuration files, writing rules in SQLJEP syntax, simpler than Lua script-based MySQL Proxy).

The comparison between Mycat and the above middleware is shown in the table below.

Compare project MycatMangoCobaHeisenbergAtlasAmoeba data slicing support read-write separation support support downtime automatic switching support semi-support Impact write does not support MySQL protocol front-end support JDBC front-end support JDBC-supported databases MySQL, Oracle, MongoDB, PostgreSQLMySQLMySQLMySQLMySQLMySQL, MongoDB community activity high activity stagnation low medium stagnation documentation is extremely rich and complete lack of medium lack whether open source support transaction weak XA support single database strong consistency, distributed weak transaction single library strong consistency, Multi-database weak transactions single database strong consistency, distributed weak transactions do not support 3. Problems that can be solved by Mycat

4. Mycat core concepts 4.1 logical library (schema)

Business developers usually do not need to know the existence of middleware in practical applications, but only need to pay attention to the database, so database middleware can be regarded as a logical library composed of one or more database clusters.

In the era of cloud computing, database middleware can provide services for one or more applications in the form of multi-tenancy, and each application may access an independent or shared physical library.

4.2 logical tables (table)

4.2.1 sharding table

The slicing table is to split the table with a large amount of index data into multiple database instances, and all the fragments are combined to form a complete table.

4.2.2 non-fragmented table

Not all tables need to be sliced when there is a large amount of data, and some tables do not need to be sliced. A non-sharded table is a table that does not require data slicing relative to a sharded table.

4.2.3 ER table

Relational database is based on entity relation model (Entity Relationship Model), which is the source of ER table in Mycat. Based on this idea, Mycat proposes a data slicing strategy based on Emurr relation, in which the records of the child table and the records of the parent table are stored on the same data slice, that is, the child table depends on the parent table, and the table grouping (Table Group) ensures that the data association query will not operate across databases.

Table grouping (Table Group) is not only a good idea to solve the cross-fragment data association query, but also an important rule of data segmentation.

4.2.4 Global tabl

In a real business scenario, there are often a large number of similar dictionary tables, the data in these dictionary tables do not change frequently, and the scale of the data is small, rarely have more than hundreds of thousands of records.

When the business table is sliced because of its scale, the association query between the affiliated dictionary tables becomes a thorny problem, so the association query of this kind of table is solved through data redundancy in Mycat, that is, all the fragments copy a piece of data, and the table of these redundant data is defined as the global table.

Data redundancy is not only a good idea to solve the cross-fragment data association query, but also another important rule of data segmentation planning.

4.3 sharding Node (dataNode)

After slicing the data, how big one is divided into different sharding data libraries, and the database where each table shard is located is the sharding node.

4.4 Node Host (dataHost)

After sharding the data, each sharding node may not necessarily own one machine, and there can be multiple sharding databases on the same machine, so that the machine in which one or more sharding nodes are located is the node host. In order to avoid the limit of the number of concurrent hosts in a single node, the shard nodes with high read and write pressure are placed uniformly on different node hosts as far as possible.

5. Introduction of Mycat folders and files

Reference:

[1] http://mycat.io/

[2] "distributed database architecture and enterprise practice-- based on Mycat middleware"

[3] long GE official course Courseware

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.