Shulou(Shulou.com)06/02 Report--

In the last article, we talked about the "service isolation" pattern in architecture design, and today we continue to explore another common design in distributed system architecture: service flow restriction.

So, what is "service restriction"? Before explaining "service restriction", let's take a look at a joke that was very popular on the Internet some time ago, about an engineer on Sina Weibo who was holding a wedding at home when he suddenly received a call from the company to deal with the surge in online traffic. that day, it should be a popular star who suddenly announced his relationship on Weibo, and Weibo traffic suddenly increased several times, resulting in unstable system function and poor user access. Then the engineer had to hang out the bride and open the notebook to debug the code in a suit at the wedding site.

At that time, the engineer must have been devastated and must be thinking: why did you announce your relationship today! Wait until I finish the expansion of the system and the service flow-limiting mechanism.

Haha, after reading the joke, basically the role of service flow restriction will understand: "service flow restriction" actually refers to when the system resources are insufficient to cope with a large number of requests, that is, when there is a contradiction between system resources and access volume, in order to ensure that the limited resources can serve normally, so we carry out traffic restrictions or functional restrictions on the system in accordance with the preset rules.

First, why do you want to do the service flow limit design?

Let me cite another example in our life: some popular tourist attractions often have strict restrictions on the number of daily visitors, such as Gulangyu in Xiamen and the Imperial Palace in Beijing, which only sell a fixed number of tickets every day. if you go late, the tickets may have been sold out that day, and you won't be able to go in and play on that day.

Why are tourist attractions subject to such restrictions? Wouldn't it be better to sell more tickets and make more money?

In fact, for tourist attractions, they are also very helpless, because the service resources of scenic spots are limited, and the number of people who can serve every day is limited. Once the restrictions are released, the staff of the scenic spots will not be enough, and the sanitary conditions will not be guaranteed. Safety also has hidden dangers, and the super-dense crowd will also seriously affect the experience of tourists. However, due to the fame of the scenic spot, there is an endless stream of tourists to play, far beyond the carrying capacity of the scenic spot, so the scenic spot has to make measures to limit the daily flow of people.

Similarly, in the IT software industry, system services are the same.

If your system theory is that 100W users can be served in a time unit, but today, 300W users suddenly come. Because of the randomness of user traffic, if there is no current restriction, it is very likely that these 300W users will overwhelm the system at once. As a result, no one can get the service.

Therefore, in order to ensure that the system can provide normal services for at least 100W users, we need to design the current limit of the system.

Some people may wonder, since there will be 300W users to access, why not simply design a cluster that can support such a large number of users?

That's a good question. If the system is accessed by 300W users for a long time, we must do the above upgrade, but the situation we often face is that the daily visit volume of the system is 100W, but occasionally there is a short-term surge of traffic caused by some unpredictable specific reasons. at this time, companies often do not expand our system to its maximum size for an unusual spike for the sake of cost savings.

Second, what should be done to limit the flow of services?

To limit the flow of system services, there are generally the following modes:

Circuit breaker: this mode requires that the circuit breaker should be taken into account at the beginning of the design of the system. When there is a problem with the system, if it can not be repaired in a short time, the system should automatically make a judgment, turn on the fuse switch, deny traffic access, and avoid large traffic overload requests to the back end. The system should also be able to dynamically monitor the repair of the back-end program, and when the program has returned to stability, it can turn off the circuit breaker and resume normal service.

Service degradation: all the functional services of the system are graded. When there is something wrong with the system and urgent current restriction is needed, the less important functions can be degraded and the service can be stopped. In this way, more resources can be released for core functions. For example, in the e-commerce platform, if there is a sudden surge in traffic, non-core functions such as commodity reviews and points can be temporarily downgraded, these services can be temporarily downgraded, and resources such as machines and CPU can be released to ensure that users place orders normally, while these degraded functional services can wait until the whole system returns to normal, and then start for replenishment / compensation processing. In addition to functional degradation, we can also adopt the way of not directly operating the database, but all read cache and write cache as a temporary degradation scheme.

Delayed processing: this mode requires setting up a traffic buffer pool at the front end of the system to buffer all requests into this pool without immediate processing. Then the real business handler at the back end takes requests out of the pool and processes them in turn, which can be implemented in queue mode. This is equivalent to reducing the processing pressure on the back end in an asynchronous way, but when the traffic is large, the processing capacity of the back end is limited, and the requests in the buffer pool may not be processed in time and will be delayed to a certain extent.

Privilege processing: this mode needs to classify users, through the preset classification, let the system give priority to the user groups that need high security, and the requests of other user groups will be delayed or not processed directly.

In a real project, the following technical methods can be used to restrict access traffic:

The technology of circuit breaker can mainly refer to the practice of hystrix, an open source component of Netflix, which mainly has three modules: circuit breaker request judgment algorithm, circuit breaker recovery mechanism and circuit breaker alarm.

The counter method system maintains a counter that adds 1 to a request, subtracts 1 from the completion of request processing, and rejects new requests when the counter is greater than the specified threshold. Based on this simple method, some advanced functions can be extended, such as the threshold can not be fixed, but can be adjusted dynamically. In addition, there can be multiple sets of counters to manage different services to ensure that they do not affect each other, and so on.

The queuing method is based on the FIFO queue, where all requests are entered into the queue, and the back-end program takes the requests to be processed from the queue and processes them in turn. The queue-based approach can also extend more games, such as setting multiple queues to configure different priorities.

First of all, the token bucket method is based on a queue and the request is placed in the queue. But in addition to the queue, you have to set up a token bucket, and there is another script that puts tokens into the token bucket at a constant speed. The back-end processor must take a token out of the bucket every time it processes a request. If the token is finished, then you can't process the request. We can control the speed of the script to put tokens to control the back-end processing speed, in order to achieve dynamic flow control.

Matters needing attention in service flow restriction

When we do service flow restriction, there are still some principles and things to pay attention to:

Real-time monitoring: the system must do a good job of real-time monitoring of the whole link in order to ensure the timely detection and processing of current limitation.

Manual switch: in addition to the automatic current limit of the system, there needs to be a switch that can be manually controlled to ensure manual intervention at any time.

Current-limiting performance: in theory, the current-limiting function will affect the normal performance of the business to a certain extent, so it is necessary to achieve current-limiting performance optimization and control.

System failures are often unpredictable and unavoidable, so as system designers, we must presuppose various measures in advance to deal with system risks that may occur at any time.

This article is transferred from the official account of Wechat and not only thinking, author: Kuige

Original link: https://mp.weixin.qq.com/s/G3Pt-rpQ9feUU6h_usa8Fw

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.