Shulou(Shulou.com)05/31 Report--

This article introduces the relevant knowledge of "what are the advantages of MySQL Taurus technology". In the operation of practical cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Technology trend of Cloud Native Database

The traditional database architecture has been designed and optimized for local deployment, local storage, and stand-alone database instances for more than 30 years, and has not been optimized for public cloud environments for a long time since then. In a public cloud environment, computing and storage are separated over the network to support multi-tenant database instances and scale out scale to both ends; high reliability of a single node is not required.

For example, suppose that each database instance of a local database has local storage. To support high availability, it is common to deploy a standby instance and use the database internal replication mechanism to send database logs (MySQL binlog) to the standby instance to rebuild the database replica. At present, most cloud vendors adopt similar mechanisms to support high availability of cloud databases. However, this internal replication mechanism does not take advantage of the reliability that cloud storage has already provided through storage tier replication, resulting in a large amount of storage and bandwidth waste. In fact, the slave database instance can share data with the primary instance through cloud storage, which can not only save disk space and bandwidth, but also provide fast RTO for cloud database services.

It is clear that when migrating traditional databases to the cloud, the database system needs to be redesigned and optimized. Over the past few years, major cloud providers, such as AWS and Aliyun, have done some work. It is expected that in the coming years, more cloud and database vendors will follow this trend and build some form of cloud native database technology. In addition, Gartner predicts that 90% of cloud database architectures will support the separation of computing and storage by 2019.

It is worth noting that in addition to the redesign of the database architecture based on the cloud environment, there are two major trends in this area:

Leverage new hardware such as new hardware and RDMA in cloud native databases. These are all active research areas, but apart from acting as a cache and a quick link between computer and storage, we haven't seen any other breakthroughs so far.

Based on real-time and historical data, ML (machine learning) and statistical techniques are used to help manage and self-adjust the cloud database, including but not limited to database instance automatic scaling and parameter setting and automatic adjustment, anomaly detection, etc.

The deficiency of MySQL on Cloud

Currently, for RDS for MySQL, all major cloud vendors basically adopt the high availability mode of master / slave, and Binlog replication is used between master and slave. Under this architecture, full-stack resources are required for each slave library added, which leads to some problems:

1. Waste of resources

The standby database often does not bear the traffic; the replication thread consumes the computing resources of the master / slave database, resulting in a waste of computing resources; under the 3 copy redundancy mechanism of EVS, N standby libraries need a maximum of 3 * (Number1) copies, resulting in a waste of storage resources and a variety of data need to be brushed through the storage network, resulting in a waste of network resources.

2. Poor scalability

When adding read-only data, you need to copy the data and replay binlog, which is very slow in the case of a large amount of data, especially the local disk scheme. The delay of master-slave replication will affect master / slave switching, so it is difficult to guarantee RTO. In addition, when using physical backup and logical backup, the backup recovery speed is very slow, and the amount of data above 1T is counted in hours, which limits the expansibility of the database.

Evolution of Huawei Cloud RDS Architecture

Huawei Cloud Taurus and Amazon Aurora database evolve in the same direction, but their evolution paths are different, which is determined by the different implementation ways of their respective database cloud services. There are several versions of Huawei Cloud RDS MySQL. These products meet different user business scenarios, have different characteristics, and can complement each other.

MySQL stand-alone version

MySQL standalone version uses the separation of database computing nodes and storage nodes, taking advantage of the reliability and multi-copy characteristics of cloud disk data, as well as ECS CVM virtualization to improve the efficiency of standardized deployment, version and operation and maintenance management, which can meet the business scenarios where low-end users pay less attention to high-availability services.

MySQL master and backup

MySQL High availability Edition is a highly available database version for enterprise users, providing 99.95% SLA guarantee. The Active-Standby high availability architecture is adopted, and the data Replication is carried out between the primary node and the standby node through MySQL Binlog. When the primary node fails, the standby node takes over the service. At the same time, it also supports up to 5 read-only nodes and supports data read-write separation access mode of load balancing.

MySQL Financial Edition

MySQL Financial Edition is a highly available and highly reliable cloud service product designed for high-end users such as the financial industry. The one-master, two-backup, three-node architecture can be deployed across AZ, and the outage of any server does not affect business availability. Based on Paxos protocol, it achieves strong data consistency in distributed environment and has better failure recovery time, which meets the needs of business scenarios such as data disaster recovery and backup.

Evolution of Taurus

Taurus adopts the technical architecture of computing and storage decoupling and can support up to 15 read-only nodes at the same time. Between the master node and the read-only node is the Failover mode of Active-Active, and the computing node resources are fully utilized. Due to the use of shared storage, the user cost is further reduced.

There are several major innovations in Taurus's design ideas:

Offload some computing power to the storage node

Deep optimization of software and hardware to make full use of hardware advantages such as RDMA network and NVME SSD

Log is data. Compared with traditional RDS for MySQL, page is no longer needed, all update operations are logged, and double is no longer needed.

Write, which reduces network communication.

In addition, in DB Server design, it fully embraces the open source ecology and is fully compatible with MySQL 8.0.

A common design philosophy of Huawei Cloud Taurus and Amazon Aurora is to launch a write-and-read version first, and a multi-write version according to market demand, to meet the vast majority of OLTP application scenarios and performance requirements.

Taurus architecture design principles:

Use Huawei next Generation Cloud Storage (DFV) as fast, scalable, reliable and shared database storage. Do not replicate existing functions in the storage tier, such as data replication, cross-AZ reliability, and data cleanup.

A single database cluster should only need a sufficiently reliable set of database replicas. All read-only copies are shared and stored in cloud storage, even across

AZ, there is no logical replication in the database tier. Write and read many times, and there is no independent backup instance. When the primary node fails over, the read-only copy can be switched to take over the primary server.

Record as a database. Write only sequentially to minimize SSD wear (avoid write magnification)

Only database logs are written to the DFV storage tier from the database computer node over the network. No page checkpoints and refreshes across the network, no double writes. Based on DFV

The database log in the storage layer rebuilds the data surface to avoid heavy network traffic.

Partition the database based on the slicing strategy across DFV storage nodes to support large database volumes. Single DFV

The storage node manages multiple shards from different database cluster instances to achieve unlimited expansion of storage capacity and processing capacity.

Analysis of key Technical points of Taurus Database

Taurus architecture

As shown in the figure, Taurus is a distributed cluster architecture design, which adopts the design concept of separation of computing and storage to meet the rigid needs of user business elastic expansion in the public cloud computing environment. The database computing node and the storage node are interconnected by high-speed network, and the data is transmitted through RDMA protocol, so that the performance of Icano is no longer a bottleneck.

The database node is designed to be fully compatible with MySQL. The Failover mode of Active-Active is adopted between the master node and the read-only node, which provides the highly available service of DB. At the same time, only Redo log-related metadata information needs to be synchronized between DB Server.

SQL Nodes

Manage client connections, parse SQL requests, analyze execution plans, and execute and manage transaction isolation levels

Read more than one master

Main library and read-only loose coupling

Less communication between the main library and read-only

Fast Failover

Introduce some features in the kernel

Query result cache

Query plan cache

Online DDL

SQL Replication

Read-only nodes maintain multiple versions of the data page in the local buffer pool

Reduce reading of data pages from the remote end of storage

Main library interaction

MVCC: synchronizes the list of current active transactions in the main library

Page failure: receive the main library page failure list

RDMA Protocol in High Speed Network

Deploy RoCE RDMA to reduce network latency and increase bandwidth. In a single AZ, Taurus can use high-speed network (RDMA) to obtain low-latency storage for updates and reads, which is very useful for low-client connection schemes.

Storage abstraction layer (SAL)

The storage abstraction layer (SAL) is a logical layer that isolates data storage from SQL front ends, transactions, query execution, and so on. It consists of a group of sharding nodes whose capacity and processing capacity are expanded on demand. SAL manages and scales the data pages needed by all SQL nodes according to shards. As the size of the database grows, the available resources (storage, memory) grow proportionally with sharding creation. Data processing is localized, and data-intensive operations are performed as sharding services on the storage node.

Taurus Storage

Taurus storage is based on Huawei cloud storage DFV persistence layer. The DFV persistence layer cluster includes multiple storage nodes. Each storage node contains multiple SSD devices and append storage service processes adapted to SSD media. The DFV persistence layer provides a storage interface for SQL nodes.

Taurus deals with SQL node data in a journalized way, that is, "log is data". The data organization adopts the Append mode for SSD redesign, which significantly reduces the system write magnification and storage delay. At the same time, in Append mode, data rollback and rollback become very fast.

Lower cost of Taurus core competence

Compared with traditional RDS for MySQL, there is only one copy of storage. When you add a read-only node, you only need to add a compute node and no additional storage is required. The more read-only nodes, the more storage costs are saved.

Higher scalability and reliability

Scale out faster. Taurus uses shared storage, only needs to add computing nodes, no matter how much data, only takes about 5 minutes, and supports 15 read-only.

Vertical expansion is faster. Containerized deployment, specification change faster.

Master-slave switching time is faster. Replication latency is eliminated and RTO is guaranteed.

Database Crash recovery is faster.

Better performanc

Combination of software and hardware, based on hardware: Hi1822 uninstall, NVMe SSD,RDMA, while kernel optimization: get rid of double write,Query Cache/Plan Cache optimization, Innodb Lock Management optimization, Audit Plugin efficiency optimization, performance up to 7 times that of native MySQL.

Backup recovery is faster

Distributed storage system customized for Taurus engine to greatly improve data backup and recovery performance

Strong data snapshot processing ability. AppendOnly vs. WriteInPlace, data is naturally stored in multiple time points and multiple copies, snapshots are generated in seconds, and mass snapshots are supported.

Quick rollback at any point in time. Based on the multi-time point characteristics of the underlying storage system, it does not need incremental log playback, and can be rolled back directly according to the point in time.

Parallel high-speed backup and recovery. Backup and recovery logic sinks to each storage node, accesses data locally and interacts directly with third-party storage systems, with high concurrency and high performance.

Fast instance recovery. Through the asynchronous data copy + on-demand real-time data loading mechanism, Taurus instances can achieve full functionality in minutes.

This is the end of the content of "what are the advantages of MySQL Taurus technology". Thank you for your reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.