Shulou(Shulou.com)06/01 Report--

HTAP is a popular concept in the database field at present. It can support both OLTP (online transaction processing) and OLAP (online Analytical processing). It can cover the needs of most enterprise applications and solve their problems in one stop. This time, the editor had the honor to interview Mr. Ma Xiaoyu, the person in charge of PingCAP analytical products, to share the HTAP road of TiDB.

Ma Xiaoyu, head of Analytical products at ▲ PingCAP

TiDB= X% TP+Y% AP = HTAP

TiDB is a distributed NewSQL database, which supports horizontal elastic extension, ACID transactions, standard SQL, MySQL syntax and MySQL protocol, and has the high availability feature of strong data consistency. It is a hybrid database suitable for not only OLTP scenarios but also OLAP scenarios.

To trace the development history of TiDB, we should start from its "ancient times". "at first, inspired by Google Spanner/F1, PingCAP began to develop TiDB," Ma said. " At that time, a series of emerging database manufacturers such as CockroachDB (cockroach database) had appeared abroad to solve the TP problem, but few domestic companies developed such databases, and the emergence of TiDB filled this gap in the market.

At first, TiDB only solved the problem of TP, but in the process of practical application, users put forward higher requirements for the query function of TiDB database. To this end, TiDB adjusted its own database executor to increase the functional expansion of AP. In this way, TiDB has both TP and AP functions.

Ma Xiaoyu revealed, "customers have always been skeptical about 80% of TiDB applicable to 100%TP and 80%AP scenarios. Why 80%, not 75%, and not 85%? so we later switched to a more accurate (fashionable) statement: TiDB is a HTAP database."

Architectural advantages of TiDB database

The overall architecture of TiDB is divided into two layers: TiDB and TiKV. TiDB is a layer of stateless SQL Layer that exposes MySQL network protocols. It is responsible for parsing users' SQL statements, generating distributed Query Plan, translating it to the underlying Key Value operation and sending it to TiKV. "generally speaking, TiDB is the XXXXXXXXXXXL version of MySQL." Ma Xiaoyu said.

The original schema of the ▲ TiDB database

The key foothold of TiDB is that it is an extensible relational database. Ma Xiaoyu added, "in addition to replacing MySQL for TP business, TiDB is also very suitable for mid-Taiwan scenarios, such as easy synchronization of MySQL production database, transparent and barrier-free cross-shard query, real-time landing of data, mass storage (TiKV) allows multi-source aggregation, and so on."

Not only that, TiDB's code is all open source. Ma Xiaoyu said, "We are an open source company, and we will also participate in some other open source communities. Many of our employees are experienced participants and contributors to the open source community. We hope to win-win with other communities. More small partners can participate in our open source."

▲ TiDB 1.0 vs 2.0

It is reported that the upgrading speed of TiDB is very fast. TiDB was founded by GitHub in May 2015, followed by a Beta version in June 2016 and a RC2 version in March 2017. On May 10, 2019, the TiDB 3.0.0-rc.1 version was officially launched, which makes a lot of improvements to system stability, ease of use, functionality, optimizer, statistics and execution engine compared to 3.0.0 Beta.1.

The TiFlash project is in progress

What big moves will PingCAP do after 2019? Ma Xiaoyu said, "in addition to improving the performance of the product and improving more functions, we will also carry out TiFlash projects." It is reported that TiFlash is a relatively independent and complete analytical database product, which synchronizes a set of storage independently through Raft Learner, and Label carries out physical isolation, so that AP/TP jobs do not affect each other, thus solving the problem of resource isolation of HTAP system.

Under the extremely low consumption of the TP side, Raft Learner access combined with MVCC transaction model can provide consistent data reading while taking into account resource isolation. When the data is read, TiFlash sends a Raft index proofreading request. TiFlash provides reads only when the data is synchronized to the point in time when the read request is sent. Ma Xiaoyu revealed, "the GA version will be released in 2019, including TiSpark and TiDB dual portals."

Ma Xiaoyu said, "in the beginning, in the industry, HTAP is actually a marketing slogan. If a database can perfectly integrate TP and AP, then in terms of design, it will be a very clean and beautiful approach." He also said, "there will be more scenarios to be developed in HTAP in the future, which are worth exploring."

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.