Shulou(Shulou.com)06/01 Report--

With the continuous development and gradual popularization of the Internet, various industries have chosen to go to the cloud one after another. Tencent Cloud Database is committed to using leading technologies to help enterprises go to the cloud. Tencent Cloud's self-developed financial-level distributed database TDSQL is a database product with strong consistency and high availability, global deployment architecture, distributed scale, high performance, enterprise security and other characteristics. The column "Picture TDSQL" will take you to understand all aspects of TDSQL in the form of a combination of pictures and text.

This article will give you a detailed interpretation of the core features and application scenarios of TDSQL.

Nowadays, most of the databases of financial business scenarios use foreign commercial databases, because they have high requirements for the maturity of the database, which can only be trusted after long-term research and testing. At present, these foreign centralized databases have been faced with some problems, such as the cost is too high, massive business scenarios can not be flexibly scaled and so on.

At the same time, domestic database products have made great progress. Tencent Cloud's self-developed TDSQL is such a financial-grade domestic distributed database product that supports automatic horizontal split and SharedNothing architecture.

The past Life and present Life of TDSQL

In fact, TDSQL is nothing new to Tencent, dating back to the early 21st century, when Tencent's billing products were still in their infancy, and the technical team chose to build a database system entirely based on open source MySQL. With the rise of the open strategy, and in order to ensure the high availability of company-level sensitive businesses such as billing, zero loss of core data and zero error accounts of core transactions, Tencent database team began to develop a database product in 2007, which is also the predecessor of TDSQL. Later, with the development and expansion of Tencent opening and cooperation, there are more and more industry scenarios, starting in 2012. Based on the strategy of "open source customization + self-research", Tencent builds a more general financial-level distributed database product, officially named TDSQL, with the goal of ensuring high availability, data consistency and horizontal scalability of financial-level application scenarios. With constant updates and iterations up to now, that's what you see.

As you can see, the predecessor of TDSQL was to solve internal billing problems, so up to now, 90% of Tencent's financial billing business systems have been built on TDSQL, including familiar product businesses such as WeChat Pay and Tencent recharge. Today, TDSQL has supported more than 500 government, enterprise and financial institutions, with customers covering banking, insurance, securities, Internet finance, billing, third-party payment, Internet of things and Internet + and other related fields.

Six core features of TDSQL

TDSQL can develop into what it is now, ensuring the smooth operation of more than 90% of the financial billing business systems in various financial institutions and Tencent, which naturally has its own unique features. Here are the six core features of TDSQL.

1. Strong consistency of data

TDSQL achieves multi-replica strong consistent replication in a real sense through kernel-level deep optimization. The strong synchronous performance of TDSQL across data centers is equivalent to asynchronous replication. Through the active and standby deployment mode, RPO=0, that is, zero data loss can be achieved, which is the most important and basic requirement for financial scenarios. At the same time, TDSQL also realizes the automatic strong consistent handover between master and standby, which can complete the whole process of master / standby handover within 30 seconds, and realize the second recovery of fault RTO.

two。 High financial level available

Strong synchronous replication (Raft) independently developed by TDSQL ensures disaster recovery and data reliability across racks, IDC and cities, and achieves zero data loss. Under the guarantee of automatic disaster recovery monitoring and second handover, 99.999% high availability of financial level can be achieved. After optimization, the strong synchronization performance of TDSQL TPS is similar to that of asynchronous performance under cross-IDC network latency, without any loss. In addition, TDSQL also supports flexible global deployment architecture: in addition to supporting the banking standard "two places, three centers" deployment architecture, it also supports "two centers in the same city", "two places and four centers" and other architectures, which can achieve different modes of high availability disaster recovery according to different computer room construction.

3. High performance and low cost

The maximum performance of a single shard of TDSQL can reach more than 550000 QPS, and the performance of the whole instance increases linearly with the increase of the number of shards, and there is no performance bottleneck in the middleware + database scheme, that is, SQL Engine can also do linear expansion, and the strong synchronization performance is equivalent to that of asynchronous synchronization, which can ensure high performance without data loss.

4. Linear horizontal extension

TDSQL has super flexible expansion ability, uses self-developed automatic rebalancing technology to ensure automatic capacity expansion and stability, and supports online real-time capacity expansion. The whole expansion process is completely transparent to the business without business downtime. When the capacity is expanded, only part of the fragments will be read-only or interrupted in seconds, and the whole cluster will not be affected.

5. Enterprise-class security

The security of TDSQL has been verified by large-scale products of various core businesses of Tencent for more than 10 years, including social networking, e-commerce, payment, audio and video and other products. In terms of disaster preparedness, TDSQL provides comprehensive data backup, disaster recovery, one-click upgrade, rapid recovery, database firewall, transparent encryption and other functions, while establishing a sound monitoring and alarm system, most of the failures are handled and recovered through automated programs.

6. Intelligent operation and maintenance

The "Red Rabbit" and "Bian Que" systems provided by Tencent Cloud provide rich tools for automated database operation and maintenance, and greatly solve the problem of the efficiency of traditional centralized database operation and maintenance.

The "Bian Que" system is a collection of intelligent tools provided by TDSQL, including data acquisition, real-time detection, automatic processing, performance testing and health evaluation, SQL performance analysis, business diagnosis, and so on. It uses module plug-in to seamlessly connect various databases, automatically crawls SQL with performance problems, and provides index optimization suggestions for intelligent analysis, so as to nip the database performance problems in the bud in time. With the help of Bian Que, DBA can be freed from the daily complicated database operation and maintenance work. "Red Rabbit" platform provides all the operation and maintenance functions of TDSQL and the display of hundreds of database status monitoring indicators from the administrator's point of view, so that more than 90% of the daily operations of database administrators can be completed through the interface, and it is more convenient to locate and troubleshoot problems.

Three Application scenarios of TDSQL

Currently, TDSQL is mainly suitable for the business of OLTP scenarios. Here are three common TDSQL application scenarios.

1. Ultra-high concurrency real-time transaction scenario

There are common problems in e-commerce, finance, O2O, social applications, retail, SaaS service providers and so on, such as large user base (millions or more), frequent marketing activities, and slow database response of the core transaction system, which restrict the development of the business. TDSQL can provide linear horizontal scalability, improve database processing capacity and access efficiency in real time, and easily deal with real-time transaction scenarios with high concurrency. WeChat Pay, Tenpay, Tencent recharge and other TDSQL database architectures are all used.

two。 Financial innovation business scenario

The financial industry has a very high demand for transaction processing, money transfer and deduction are all using transactions. TDSQL implements strong distributed transactions through the classic XA two-phase commit plus two-phase blocking protocol to support the needs of financial scenarios for transaction management. At the same time, it adopts the strategy of automatic sub-database and sub-table, taking full account of the characteristics and needs of banking business, so it is a professional database for "financial transaction" scenario.

3. PB-level data storage access scenario

In the Internet of things scenarios such as industrial monitoring and remote control, smart city, smart home and vehicle Internet, there are many sensor monitoring devices, high sampling rate and large data scale, and usually the data stored for one year can reach PB level or even EB. However, the traditional solutions based on x86 server architecture and open source database simply cannot store and use such a large amount of data. The capacity scale provided by TDSQL and the compression capability of storage engines such as Rocksdb can effectively help users to store large amounts of data at low cost (compared to shared storage solutions).

TDSQL customer case

Starting from 2014, WeBank abandoned the traditional IT architecture and completely adopted the Internet distributed architecture to build the core transaction database, deploying more than 1500 instances to host all OLTP services of the bank. With the financial business support platform provided by Tencent Cloud, WeBank has not only achieved independent control of technology, but also saved a lot of data center investment costs, management costs, and payment loss rate. For details, click on the actual combat process of the financial-level database on the scale of 100 million customers and PB-level data.

On September 12, 2019, TDSQL was officially put into production at Zhangjiagang Rural Commercial Bank, and Zhangjiagang industry became the first bank in China to use domestic databases in traditional core business scenarios. The hardware cost of Zhangjiagang Bank, which adopts TDSQL, has been reduced to 1/5 or even lower than that of the traditional architecture. While the cost has been greatly reduced, the performance of the database has also been greatly improved, which is expected to meet the business development in the next decade. For details, click on the figure: TDSQL | Zhangjiagang Bank, I am your "wall breaker".

In addition, Tencent Cloud TDSQL also supports the Internet core production systems of many banks and securities insurance companies, such as Bank of China and Futu Securities. At present, more than 500 financial institutions have chosen TDSQL to carry business data.

After more than ten years of development, TDSQL technology continues to break through, and has gained a lot of trust and recognition from the industry. I believe that 2020 will ride the wind and waves and look forward to the future.

Previous recommendation

The picture shows TDSQL | Zhangjiagang Line, I am your "wall breaker"

Special experience of cloud database

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.