Shulou(Shulou.com)05/31 Report--

In this article, the editor introduces "TiDB case Test and Analysis" in detail, with detailed content, clear steps and proper handling of details. I hope this "TiDB case Test and Analysis" article can help you solve your doubts.

First of all, for me, I think it is really a very cool thing to be able to develop a database and have a deep technical complex. I appreciate the geek spirit, satisfy the business at the same time, and also reflect the technical value, this model is worthy of reference for many open source projects.

First of all, what interests me is not the NewSQL role of TiDB, but the development process of TiDB. The architecture evolution of TiDB is very helpful for understanding TiDB technology, and it also has some reference for our work and practice. If I were to sum up, I think there were several milestones that triggered me a lot.

① designs MySQL distributed storage engine.

The whole project began in April 2015, and the initial stage is to write a MySQL distributed storage engine to meet the basic distributed requirements, but the performance is not satisfactory. At the same time, the storage engine layer's support for the optimizer layer and the transaction model layer is very limited, so the initial architecture design does not meet the expectations.

② is compatible with MySQL protocol and is implemented from top to bottom.

The later architecture design is based on the standard MySQL protocol, which is rewritten from top to bottom, which is fully compatible with MySQL protocol and realizes the basic requirements of Server layer.

The architecture of TiDB version 0.5 is as follows:

③ Storage engine introduces HBase

The initial TiDB does not have a storage engine, and the data is all at the memory level. Accessing HBase is also a strategic choice, mainly to verify whether the implementation of SQL layer is stable or not.

④ uses Rust to rewrite Raft in Etcd

The KV storage layer is implemented using Rust. The main difficulty is to completely rewrite the Raft implementation of Etcd using Rust. I think this is the most difficult thing to do. You can imagine the difficulty, but if you do it, you will find it full of achievements.

⑤ connects to RocksDB

RocksDB is a stand-alone key-value engine, the predecessor is actually LevelDB, is Google around 2011 open source key-value storage engine. The data structure of RocksDB is that LSM Tree is a very write-friendly data structure with good read performance when the machine memory is large.

At the technical architecture level, the RAC in TiDB and Oracle is actually very similar (components and functions). Of course, the biggest difference is that one is distributed and flexible, and the other is integrated and shared.

I tested using the following deployment architecture.

In the process of testing, some basic tests and performance tests have been done for TP,AP services, and some basic coverage tests have also been done for high availability, flexible expansion, rolling upgrade, backup recovery.

The advantages are obvious, and you can feel from the deployment and installation that many new technologies are being used on a large scale.

The highlights are as follows:

① supports multiple deployment methods (offline deployment, online deployment, docker deployment)

Integration of ② monitoring and deployment

Rapid deployment of ③

④ backup and recovery, customized mainstream tool mydumper,myloader

⑤ incremental copy syncer

Features of ⑥ real-time backup and recovery TiDB's binlog scheme, docking with kafka

⑦ undertakes the business of AP, based on spark

Elastic expansion and reduction of ⑧

⑨ rolling upgrade

⑩ read-write mix, not limited to intensive writes alone

11 Tidb is redeployed, the original data will not be deleted, if it is reused at a discount

12 automatic fault recovery

13 the product has a strong customization ability, customizing nearly 30 parameters to meet the needs of TiDB

There are also some minor mistakes or problems in details, follow-up with friends and centralized feedback.

As far as I understand it, the current business entry point of TiDB can be used as a supplement to the existing MySQL solution, or even a transparent cluster solution. Whether you use PXC,MHA or MGR, the whole process can be connected through cascading.

Another entry point should be big data. At present, from my tests, TiDB is an optimistic lock, and the demand for support for AP business is actually greater, so it is a good point to be able to dock to big data platform and achieve some basic data flow and even data sinking to big data.

After reading this, the article "TiDB instance Test and Analysis" has been introduced. If you want to master the knowledge points of this article, you still need to practice and use it yourself. If you want to know more about related articles, welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.