Shulou(Shulou.com)06/01 Report--

Copyright notice: this article is originally created by the blogger and may not be reproduced without the permission of the blogger.

What is high availability?

High availability HA (High Availability) is one of the factors that must be considered in the design of distributed system architecture. it usually refers to reducing the time when the system can not provide services through design.

Assuming that the system has been able to provide services, we say that the availability of the system is 100%.

If the system runs 100 time units, 1 time unit will not be able to provide services, we say that the availability of the system is 99%.

Many companies have a high availability target of four 9s, or 99.99%, which means that the annual downtime of the system is 8.76 hours.

How to ensure the high availability of the system

As we all know, single point is the enemy of high availability of system, and single point is often the greatest risk and enemy of high availability of system, so we should try to avoid single point in the process of system design. Methodologically, the principle of high availability guarantee is "clustering", or "redundancy": if there is only one single point, the hanging service will be affected; if there is a redundant backup, there are other backup can be supported.

To ensure the high availability of the system, the core principle of architecture design is: redundancy.

With redundancy, it is not enough, each failure requires manual recovery is bound to increase the unserviceable practice of the system. Therefore, it is often through "automatic failover" to achieve high availability of the system.

Next, let's take a look at the typical Internet architecture, how to ensure the high availability of the system through redundancy and automatic failover.

Common layered architecture of Internet

Common Internet distributed architectures are as follows:

(1) client layer: typical callers are browser browser or mobile application APP

(2) reverse proxy layer: system entry, reverse proxy

(3) site application layer: implement core application logic and return html or json

(4) Service layer: if service is implemented, there is this layer.

(5) data-cache layer: cache accelerates access to storage

(6) data-database layer: database solidified data storage

The high availability of the whole system is realized by redundancy and automatic failover of each layer.

Hierarchical High availability Architecture practice

High availability of [client layer-> reverse proxy layer]

The high availability from the client layer to the reverse proxy layer is achieved through the redundancy of the reverse proxy layer. Take nginx as an example: there are two nginx, one providing service online and the other redundant to ensure high availability. The common practice is keepalived survival detection, and the same virtual IP provides services.

Automatic failover: when nginx dies, keepalived can detect and automatically fail over and automatically migrate traffic to shadow-nginx. Because the same virtual IP is used, the switching process is transparent to the caller.

High availability of [reverse proxy layer-> site layer]

The high availability from the reverse proxy layer to the site layer is achieved through the redundancy of the site layer. Suppose that the reverse proxy layer can configure multiple web backends in nginx,nginx.conf, and nginx can detect the viability of multiple backends.

Automatic failover: when the web-server dies, the nginx can detect and automatically fail over, and the traffic will be automatically migrated to other web-server. The whole process is done automatically by nginx and is transparent to the caller.

High availability of [site layer-> Service layer]

The high availability from the site layer to the service layer is achieved through the redundancy of the service layer. The Service connection Pool establishes multiple connections to downstream services, and connections are "randomly" selected for each request to access downstream services.

Automatic failover: when the service dies, the service-connection-pool can detect and automatically fail over and automatically migrate the traffic to other service. The whole process is done automatically by the connection pool and is transparent to the caller (so the service connection pool in RPC-client is a very important basic component).

High availability of [service layer > cache layer]

The high availability from the service layer to the cache layer is achieved through the redundancy of cached data.

There are several ways of data redundancy in the cache layer: the first is to use client encapsulation, service to double read or write to cache.

The cache layer can also solve the problem of high availability of the cache layer through a cache cluster that supports master-slave synchronization.

Take redis as an example. Redis naturally supports master-slave synchronization, and redis officials also have sentinel sentinel mechanism to test the viability of redis.

Automatic failover: when the redis master dies, sentinel can detect and notify the caller to access the new redis. The whole process is completed by the cooperation of sentinel and redis cluster, which is transparent to the caller.

After talking about the high availability of the cache, I would like to say that the business does not necessarily have a "high availability" requirement for the cache. More scenarios for the use of the cache are used to "accelerate data access": put part of the data into the cache. If the cache is hung or the cache is not hit, you can go to the back-end database to retrieve the data.

For this type of business scenario that allows "cache miss", the recommendations for the cache architecture are:

The kv cache is encapsulated into a service cluster, and a proxy is set upstream (the proxy can ensure high availability by means of cluster redundancy). The back end of the proxy is divided into several instances according to the key level of cache access, and the access of each instance is not highly available.

The cache instance is masked: when the instance with horizontal sharding dies, the proxy layer returns cache miss directly, and the cache hang is also transparent to the caller. The number of key horizontal sharding instances is reduced, and re-hash is not recommended, which can easily lead to inconsistencies in cached data.

High availability of [service layer > database layer]

In most Internet technologies, the database layer uses the "master-slave synchronization, read-write separation" architecture, so the high availability of the database layer can be divided into two categories: "high availability of read libraries" and "high availability of write libraries".

High availability of [service layer > database layer "read]

The high availability of reading from the service layer to the database is achieved by reading the redundancy of the library.

Now that the read library is redundant, there are generally at least 2 slave libraries. The "database connection pool" establishes multiple connections to the read library, and each request is routed to these read libraries.

Automatic failover: when the read library dies, db-connection-pool can detect and automatically fail over and automatically transfer traffic to other read libraries. The whole process is automatically completed by connection pooling and is transparent to callers (so database connection pooling in DAO is a very important basic component).

High availability of [service layer > database layer "write]

The high availability of writing from the service layer to the database is achieved through the redundancy of the write library.

Take mysql as an example, you can set up two mysql dual masters to synchronize, one providing services online and the other redundant to ensure high availability. The common practice is keepalived survival detection, and the same virtual IP provides services.

Automatic failover: when the write library dies, keepalived can detect and automatically fail over and automatically migrate traffic to shadow-db-master. Because the same virtual IP is used, the switching process is transparent to the caller.

Summary

High availability HA (High Availability) is one of the factors that must be considered in the design of distributed system architecture. it usually refers to reducing the time when the system can not provide services through design.

Methodologically, high availability is achieved through redundancy and automatic failover.

The high availability of the whole Internet hierarchical system architecture is realized through redundancy and automatic failover of each layer.

(1) the high availability from [client layer] to [reverse proxy layer] is realized through the redundancy of the reverse proxy layer. The common practice is keepalived + virtual IP automatic failover.

(2) the high availability from [reverse proxy layer] to [site layer] is realized through the redundancy of the site layer. The common practice is survivability detection and automatic failover between nginx and web-server.

(3) the high availability from [site layer] to [service layer] is realized through the redundancy of the service layer. The common practice is to ensure automatic failover through service-connection-pool.

(4) the high availability from [service layer] to [cache layer] is achieved through the redundancy of cached data. The common practice is to cache client-side double read and write, or to use master-slave data synchronization of cache cluster and sentinel to keep alive and fail over automatically. In more business scenarios, cache servicealization can be used to mask the underlying complexity to callers.

(5) the high availability from [service layer] to [database "read"] is realized by reading the redundancy of the database, and the common practice is to ensure automatic failover through db-connection-pool.

(6) the high availability from [service layer] to [database "write] is realized through the redundancy of writing database. The common practice is keepalived + virtual IP automatic failover.

Open source: × × / technology QQ 2042849237

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.