Shulou(Shulou.com)06/02 Report--

Cloud computing brings great advantages in business elasticity. Alibaba Cloud Database senior product expert Shi Man introduces POLARDB in detail from the aspects of application architecture changes, customer actual cases, business analysis, etc., and how to use POLARDB to design the database architecture of Internet innovative applications.

cdn.com/fbbf0025947aef2ac0443c289a77f68255345722.jpeg">

Application Architecture Evolution-Why Do We Need Super MySQL?

POLARDB is 100% compatible with MySQL, with many times more performance than MySQL, and a large storage space of up to 100TB in a single instance, which can be understood as Super MySQL developed by Ali. So why did we create such a super MySQL? We understand that this is an inevitable consequence of the distributed evolution of the Internet in application architecture. First we need to review the history of application architecture evolution, from the earliest CS architecture to BS architecture, from J2EE to Spring/Struts/Hibernate, and then to the current microservices architecture, through many generations of architectural transformation. From the traditional application business architecture to the Internet distributed application architecture, changes have taken place in all aspects. From the resource layer, to the data layer, middleware, application release packaging and application framework, the perspective of development and operation and maintenance of the Internet distributed changes have taken place.

Resource layer: Traditional applications use X86, small machines, and storage devices; Internet distributed applications use public clouds, private clouds, hybrid clouds, etc.

Data layer: Traditional applications use centralized commercial databases such as Oracle and DB2, while Internet applications use distributed databases such as MySQL, Redis and HBase, which do not require centralized storage devices.

Middleware: Traditional applications will use Logic, WebSphere, etc., Internet applications will usually use Swarm, K8S, Mesos in the transition to microservices architecture.

Application Release Package: Traditional applications are developed using JAVA and released as war/ear file packages, which are then released to middleware. Microservices architectures typically publish applications as mirrors of containers.

Application framework: Traditional applications are usually developed using Spring, Struts, hibernate, while currently Internet distributed applications are more likely to use microservice architectures such as SpringCloud, Double, EDAS, etc.

Development and operation: Traditional applications use controlled release, conservative operation and maintenance, and it takes weeks or even months for new features to go live; Internet distributed architecture uses DevOps continuous integration, agile and rapid iteration.

We understand that these architectural changes to Internet distributed applications are aimed at making the business more agile and resilient, and able to withstand the high concurrency pressures from the Internet. Under the innovative architecture, business applications can be horizontally expanded at any time through microservices, but the pressure will not be handled, and the load will be directly transmitted to the data level, solving the problem of application elasticity, but creating greater challenges to the database. The distributed architecture of the Internet requires databases to be more agile, have better resiliency, and have lower costs. (In traditional applications, an application may only need one database to carry, but in Internet distributed applications, after microservice transformation, a business system may need dozens or even hundreds of databases to carry, so the cost is also required.)

Actual Combat-Alibaba Cloud Database prepares for business architecture changes

At present, Alibaba Cloud's database form has covered 99% of the business scenarios in the Internet. Relational databases include MySQL, SQL Server, PG, and POLARDB. The NoSQL family of products includes Redis, MongoDB, HBase, etc. It also has hybrid analytics data warehouse, distributed database DRDS, and database service tools (DTS, DBS, CloudDBA, DMS, etc.).

evolution route

Alibaba Cloud provides so many database products, how to choose in practical applications? We are ready for rapid business growth and renewal iterations. This is our recommended evolutionary path for application architecture: MySQL is recommended for building business applications quickly in the early stages of a business. When it grows up and independent MySQL cannot bear greater business pressure, it can be separated from reading and writing based on MySQL without any modification to the application. When we enter the rapid growth period and the read-write separation cannot support the business requirements, we can seamlessly migrate to POLARDB without any changes to the business system. Moreover, the read-write separation of POLARDB eliminates replication delay through shared storage, which is more suitable for scenarios with higher requirements for data consistency. As the business grows further, vertical splits can also be done on POLARDB. Vertical splitting refers to vertically splitting business modules into different database instances and into multiple independent databases, such as user libraries, order libraries, and warehouses, so that more independent databases can be combined to cope with the pressure of business load. When the business develops to such a large scale and volume as Taobao, DRDS needs to be adopted for distributed transformation, multi-activity across computer rooms, and unitized transformation according to business splitting, which is the evolution path that Alibaba Tao application has gone through and is effective.

Application link optimization-automatic read/write separation, short link optimization

We use a database proxy for link access layer optimization. The standard pattern for accessing databases is direct access to primary and read-only instances. In this pattern, a logical split of read and write separation needs to be done at the business level. We provide proxy patterns that completely decouple the business layer from the database layer. When accessing the database, it is not necessary to connect directly to the database instance, but to connect to the Proxy that is completely transparent to the service. After receiving the SQL request, it will automatically perform read-write separation, route all write operations to the master instance, and route the read operation Load Balancer to the read-only instance, thus realizing automatic read-write separation transparent to the service. In addition to the separation of reading and writing, proxy mode can also perform transparent switching of failed databases. Whether it is standard mode or proxy mode, when the primary instance fails, it can automatically switch to the backup instance to ensure database availability. However, in the standard mode, the service needs database reconnection after handover, but through Proxy, the service application does not need reconnection and does not feel the high availability handover. The proxy pattern also provides short-connect optimization. For example, if the business is developed using PHP, it connects to the database using short links, and each connection generates a connection when accessing the database, making the database overwhelmed by processing the connection pool. Proxy can transform short links into long links and maintain its own connection pool. At the same time, proxy mode also provides anti-brute force attack function. For example, Proxy can detect that an IP keeps trying to re-enter the password and actively block it.

Real-time Analysis Data Warehouse--PolarMPP, PolarDB Best Partner

Data processing can be divided into database ecology and big data ecology. The database ecosystem is suitable for scenarios with strong data consistency requirements such as transaction orders, but it will not be particularly large in processing capacity and processing magnitude. For example, when the order volume is 1TB or 2TB, it can still be used, but once the quantity increases to 3TB~5TB, the performance of a single database will have a very large bottleneck. At this time, complex analysis queries will make the database overwhelmed. The common practice is to adopt a big data ecosystem, and copy the data generated by online transaction processing to the Hadoop ecosystem for real-time data analysis through ETL or data replication. In the Hadoop ecosystem, the standard way is to use MapReduce or Spark for data analysis, but developers are not used to MR or Spark, nor do they like to use Scala, they are still used to SQL. Therefore, in this mode, developers are often prepared with SQL components such as Hive and Impla, so that developers can still use SQL to process data. The problem with this approach is that there is a delay between online transaction processing and offline data warehousing, ranging from seconds to minutes or even hours. And there are actually two copies of the data, which is not economical.

For this situation, we provide POLAR MPP and HybridDB to solve, it can handle data writing very well, provide millions of TPS, very suitable for storing user behavior, tags, logs and so on. This pattern can respond to tens of billions of large tables in milliseconds, do complex aggregations for multi-table associations, do multi-valued sub-columns, and do full-text retrieval. The most important thing is that it can share one piece of data with POLARDB, which greatly alleviates the need to store two pieces of data in the database ecosystem and the big data ecosystem, and there is a delay in reading and writing.

Business Scenario Analysis-Internet Innovative Application Scenario Practice

With cloud-native databases as weapons, how should innovative Internet business scenarios be designed? Before talking about innovative business, let's take a look at the traditional MySQL master N slave architecture, how to build data warehouse driven BI reports to achieve business intelligence. The problem with this architecture is that it needs to store N pieces of data and do synchronous replication of the data. Data replication is required between master and slave of MySQL, and data replication is also required from business database to analysis database.

So how should a system architecture with cloud-native POLARDB be designed? In the meantime, POLARDB and read-only analysis library constitute cloud-native data clusters, and POLAR Store uniformly stores data. Business applications will write online services to POLARDB. When POLARDB's one-master-one-slave mode is insufficient, it can be quickly expanded to one master, two slaves or even N slaves. This extension is different from MySQL in that it provides agility and business resilience. If the data volume is large, MySQL read-only library generation may take several hours. Regardless of the amount of data, it takes minutes to create a read-only repository in the POLARDB ecosystem. And only one piece of data is needed to drive business reports via POLARMPP.

Cloud-native architecture delivers the following business benefits:

1. Business compatibility, no application change: as long as it is a business system developed using MySQL, it can be 1. Seamless migration to POLARDB.

2. Read-write separation: With POLARDB, one piece of data can achieve read-write separation of multiple nodes, and support minute-level expansion. If you use MySQL to achieve read-write separation, you need to generate multiple read-only libraries through data replication, wasting time and space.

3. Real-time analysis, data sharing: In data warehouse and BI analysis business, only one piece of data is needed, and data replication is not required.

4. Read-only instances share a copy of data: Because only one copy of storage is required, it brings better cost performance. Taking the architecture of one master and five slaves as an example, the price of POLARDB is 44% lower than MySQL. It provides a higher price/performance ratio in addition to providing more powerful performance.

5. Millisecond Latency: Since master and slave share a single copy of data, there is only millisecond latency in between. Zero data loss in handoff is guaranteed when a primary node fails.

6. Session-level read/write separation data consistency: In business scenarios with high consistency requirements such as finance, the requirements for read consistency are very high, and it is difficult to tolerate second-level or even millisecond data delays. PolarDB can be used to achieve consistent reading of data within a session.

7. Pay-on-demand, second-level backup: When using MySQL, if we expect to use 500GB of capacity, we need to buy 500 GB of storage space, but the actual data may only occupy less than 100GB, but we still need to pay for 500GB of reserved capacity. But POLARDB doesn't need to make space reservations, storage pays on demand. At the same time, POLARDB can realize backup and recovery in seconds through data snapshot, which is more conducive to our database security operation and maintenance, bringing more value.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.