Shulou(Shulou.com)06/01 Report--

Just now, the United Nations officially announced at its headquarters in New York that Tencent has become a global partner of the United Nations, providing a comprehensive technical solution for the 75th anniversary of the founding of the United Nations. In a series of dialogue activities to mark the 75th anniversary of the founding of the United Nations, Tencent Conference will provide the United Nations with an online meeting and communication platform for the United Nations to flexibly organize online meetings of different sizes for global citizens. Tencent Cloud Redis and TDSQL provide underlying database support for Tencent meetings.

In other words, the goose factory database supports the largest ever United Nations global online dialogue!

With its smooth conference experience and strong internationalization capability, Tencent's daily active user accounts exceeded 10 million within two months of its launch, making it not only the most widely used video conferencing application in China, but also the international version of VooV Meeting, providing conference capabilities to more than 100 countries and regions around the world.

Tencent Cloud Redis: lossless expansion of 10 million-level QPS&

In the case of the sharp increase in the number of Tencent conference requests, the Redis database, which is used as the cache of meeting list and the storage of meeting information, has also been expanded rapidly. Because of the particularity of the business, Tencent conference service has always maintained large-scale online operation in the whole process of resource expansion, and a large number of users can still conduct meetings smoothly and smoothly in high definition. At present, the Redis databases of most cloud vendors are disconnected and read-only within 1 minute during cross-machine expansion. As a result, enterprises using Redis services provided by self-built and other cloud vendors can only rely on the business layer to achieve smooth expansion, or stop service to maintain the expansion. But this problem has been easily solved in Tencent Cloud.

The Redis service of Tencent Cloud cluster architecture can only provide the maximum 4TB storage capacity and concurrent access performance of 100000-10 million levels, and guarantee the 1ms response delay within 99.99% of the water mark. Tencent meeting operators only need to click a button on the console to efficiently complete dozens of scale expansion within half an hour, and the background processing time of the expansion process of a single cluster is no more than 30 minutes. While maintaining 100% system availability, Tencent Cloud Redis is the only Redis database product in China with lossless capacity expansion.

Tencent Cloud Redis single cluster peak requests exceed 1.7 billion / min

To achieve lossless expansion, Tencent Cloud Redis has overcome two core problems. First of all, the problem of community migration tools migrating large Key blocking system access and even downtime is solved. Through the self-developed data migration tool, Tencent Cloud team has not only solved the stutter problem, but also doubled the speed of data migration. Secondly, it has solved the problem of correctly marking the data status during data handling and rerouting after relocation. Tencent Cloud Redis has solved the data routing problem through self-developed Proxy regular refresh and response to route redirection commands.

Tencent Cloud Redis Lossless expansion Diagram

At the same time, Tencent Cloud Redis supports multi-region deployment around the world. To support the overseas version of Tencent conference, Tencent Cloud Redis has quickly deployed access points and business logic layers in Europe and the United States, Japan and South Korea, Southeast Asia and the Middle East to synchronize data globally. There are currently 32 availability zones and are continuing to increase.

Tencent Cloud TDSQL: intelligent and fine operation and maintenance at nanny level

With the Tencent conference going international, round-the-clock and irregular requests from around the world require operators to pay attention to the health of the system and deal with various system problems quickly, so as to achieve fine operation of the business. it can also meet the rapid support for the operation and maintenance of a large number of databases. Tencent Cloud TDSQL protects Tencent meetings by providing intelligent operation and automated operation and maintenance systems, automatic smooth switching of failures and flexible horizontal expansion services.

I. Intelligent operation and automatic operation and maintenance system

In the face of massive global user growth, as one of the database solutions at the back-end of Tencent conference, TDSQL is also facing an explosive reading and writing test and needs to be expanded. Expansion means receiving a large number of service requests, which is also essential for the database to provide high-performance services.

A SQL with performance problems may not cause any problems at first, but as business requests grow, these slow SQL snowball, eroding the performance of the database. At the same time, as the business grows, finding these performance SQL from hundreds of millions of SQL is like looking for a needle in a haystack.

The TDSQL intelligent DBA diagnosis system "Bian Que" solves this problem very well, it can automatically grab the SQL with performance problems, and provide index optimization suggestions through intelligent analysis, so as to nip the database performance problems in the bud in time. After optimization, 99% of SQL eliminates performance bottlenecks.

The "Bian Que" system is a collection of intelligent tools provided by TDSQL, including data acquisition, real-time detection, automatic processing, performance testing and health evaluation, SQL performance analysis, business diagnosis, and so on. With the help of Bian Que, DBA can be freed from the daily complicated database operation and maintenance work.

The "Red Rabbit" platform provides all the operation and maintenance functions of TDSQL and the display of hundreds of database status monitoring indicators from the perspective of administrators, so that more than 90% of the daily operations of database administrators can be completed through the interface, and it is more convenient to locate and troubleshoot problems. " The combination of "Red Rabbit" and "Bian Que" can not only meet the fine operation and maintenance of high-star business, but also easily deal with a large number of ordinary database operation and maintenance needs, and better help users to reduce operation and maintenance costs.

Second, the fault is automatically and smoothly switched, and the business is not perceived.

The huge amount of data makes the scale of TDSQL cluster larger and larger, and the impact of node failure on business is more and more sensitive, which puts forward a higher and higher test to the disaster recovery capability of TDSQL. The consistent handover of TDSQL ensures that the handover can be completed smoothly in several cluster node failures, minimizing the impact on the business.

Most business systems usually need to use high availability solutions to ensure the continuous operation of the system. Database, as the bottom of the software stack, provides persistence and access services to data. If the high availability is done well enough, the high availability design of the business layer can be done more lightweight and simple.

The core of the data layer of TDSQL high availability solution is based on automatic live detection logic and strong synchronous replication of raft developed by Tencent, together with automatic resource scheduling to achieve automatic disaster recovery monitoring and second switching, to ensure that the system does not interrupt service 24 hours a day, zero data loss and high data consistency.

Each shard of TDSQL supports highly available solutions based on strong synchronization and consistency, and provides 7X24 round-the-clock monitoring of databases and underlying physical devices. In the event of a failure, TDSQL will automatically restart the database and related processes, and if the node crashes and cannot be recovered, it will automatically rebuild the node through backup files.

Third, the horizontal expansion of elasticity

As an important system foundation of Tencent Conference, with the continuous surge of traffic, TDSQL has carried out a round of rapid horizontal expansion of database machines after optimization. Based on distributed architecture and multi-tenant scheme, TDSQL is inherently flexible and scalable. This means that the concurrent performance, processing power, and storage capacity of database instances can increase linearly.

In the expansion practice of Tencent Conference, through the rich read-write separation technology of TDSQL strategy, the database level quickly responds to the growing capacity and performance requirements.

In order to separate read requests as much as possible and further reduce the impact on the master node, TDSQL separates the pure read-only business by separating read-write accounts and disaster recovery read-only instances, further reducing the pressure on the master node and improving the overall throughput. Finally, 25% of the complex queries are sent to read-only instances according to the read-write separation strategy, which can quickly reduce the load on the master node.

The read-write separation technology that supports multiple strategies is one of the elastic expansion features of TDSQL's self-developed evolution. TDSQL products support read-write separation by default, and each slave in the architecture can support read-only capability. If multiple slaves are configured, the SQL Engine cluster (SQL Engine) will be automatically assigned to low-load slaves to support read traffic for large applications. The advantage of TDSQL read-write separation technology is that it provides a variety of read-write separation schemes, and users do not need to pay attention to the complete survival of several slaves, because the system can be automatically scheduled according to the policy.

In addition to the read-write separation technology, TDSQL elastic horizontal expansion includes a variety of features to cope with different scenarios.

Finally, TDSQL has a robust ability to distribute transactions, as well as continuous performance optimization. As a coordination node, SQLEngine is stateless and can be expanded horizontally almost without restriction. Each SET of the database, as a data node, not only stores normal business data, but also uses hash routing to store all global transaction logs. As a whole, each module can be expanded horizontally and can meet the almost unlimited horizontal storage requirements of the business layer.

In addition, in order to solve the thorny problem of distributed locks in distributed transactions, TDSQL adopts real-time lock diagnosis view measures, and also has global deadlock detection to eliminate various problems of business layer locks.

From providing meeting list cache and meeting information storage to back-end intelligent operation and maintenance, automatic failover and rapid business unaware expansion, with the support of Tencent Cloud database, Tencent Conference can easily cope with the continuous growth of service requests. continue to provide users with clear and smooth conference and live broadcast services, and achieve fine business system operation.

United Nations, here we come!

Related reading

Tencent conference users are skyrocketing, how to achieve seamless expansion of Redis cluster?

Special experience of cloud database

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.