Shulou(Shulou.com)06/02 Report--

This article mainly introduces the relevant knowledge of "High concurrency case Analysis of Server". The editor shows you the operation process through the actual case, and the operation method is simple, fast and practical. I hope this article "High concurrency case Analysis of Server" can help you solve the problem.

As we all know, high concurrency systems have three axes: cache, circuit breaker and current limit. But there is also an axe, often forgotten in the corner, depressed, that is preheating.

Examples of phenomena

Let's start with two phenomena. These phenomena can only occur in systems with high concurrency.

Well, it has caused a number of failures.

1. DB died instantly after restart

In a DB with high concurrency, the process dies and then restarts. As the business is at its peak, the upstream load balancing strategy has been redistributed. DB, which had just started, instantly accepted the traffic of 1max 3, and then load soared until there was no response.

The reason is: the newly launched DB, all kinds of Cache are not ready, and the system state is completely different from that of normal operation. Maybe the usual amount of 1 prime 10 can bring it to death.

2. Access exception after service restart

Another common problem is that there is a problem with one of my servers. Due to load balancing, the remaining machines immediately carry these requests and run well. When the service rejoins the cluster, there are a large number of time-consuming requests, even a large number of failures in the case of a high number of requests.

The causes can probably be attributed to:

1. After the service starts, jvm is not fully prepared, JIT is not compiled, and so on.

2. The various resources used by the application are not ready.

3. Rebalance has occurred in load balancer.

Both of these problems are not well warmed up.

Warm Up, that is, cold start / preheat mode. When the system is in a low water level for a long time, when the flow increases suddenly, directly pulling the system to a high water level may crush the system instantly. Through the "cold start", let the flow through slowly increase, gradually increase to the upper limit of the threshold within a certain period of time, give the cooling system a preheating time to prevent the cold system from being crushed.

I want a curve like this.

Not like this.

The facts are much more complicated.

Traffic is unpredictable, which is different from natural growth, or man-made attacks-a process that starts from scratch. Even some self-proclaimed ultra-high-speed components, such as lmax's disruptor, will collapse under this sudden flood.

The most suitable entry level for warmup is the gateway. As shown in the figure: node4 is a newly started node. The load balancer component integrated in the gateway will be able to identify the newly added instance and then gradually measure it to the machine until it can really withstand high-speed traffic.

If all the requests went through the gateway, everything would be much easier, and there would be components like Sentinel to cut in. But the reality often can not meet the conditions. For example:

1. Your application directly obtains the information of the registry, and then distributes the traffic in the client component.

2. Your application has passed some complex middleware and routing rules, and finally located on a certain DB.

3. Your terminal may be connected to the MQTT server directly through the MQTT protocol.

Let's do some abstraction, and we can see that all of this traffic allocation logic, including gateways, can be called clients. That is, all warmup logic is placed on the client side, and they are closely coupled with load balancing.

Solution method

Interface volume

According to the above analysis, the problem can be solved by controlling all client calls by means of coding.

A simple polling method

1. I want to be able to get the collection of all the resources to be called, as well as the startup time, cold start configuration and so on.

2. Assign some weights to these resources, for example, the maximum weight is 100, and the cold start is successful after 100 seconds of configuration. If it is 15 seconds now, the total weight is 100 * (nMel 1) + 15.

3. According to the calculated weight, the traffic is allocated, and the traffic will gradually increase according to the passage of time until it is equal to other nodes.

4. In an extreme case, there is only one instance on my backend, which cannot be started at all.

In the case of SpringCloud, we need to change the behavior of these components.

1. Ribbon's load balancing strategy.

2. Load balancing strategy of gateway.

Fortunately, they are all basic components, so you don't have to copy the code back and forth.

to glance over things hurriedly

As the name implies, it means to visit all the interfaces in advance to allow the system to prepare resources in advance.

For example, iterate through all http connections and then send a request.

This approach is partially effective, and some lazy resources will be loaded in at this stage, but not all of them.

Some enhancements such as JIT may make the preheating process very long, and the fleeting approach can only work to a certain extent.

For example, some DB, after startup, will execute some very characteristic sql to load the most needed hot data into the PageCache.

State retention

The system takes a snapshot at the time of death and then returns it intact at startup.

This process is more magical, because the general abnormal shutdown, the system simply does not have the opportunity to make last words, so can only be timed, in the running system to make a snapshot.

When the node starts, it loads the snapshot into memory. This is widely used in some memory-based components.

By comparison, we find that the most reliable way is to code and integrate warmup logic on the client side. The job may be painful and long, but the outcome is happy.

Of course, you can also use the way of "remove nginx- > modify weight-> reload nginx". Sometimes it works but not always. It's usually reassuring but not always reassuring.

This is the end of the content of "High concurrency instance Analysis of Server". Thank you for reading. If you want to know more about the industry, you can follow the industry information channel. The editor will update different knowledge points for you every day.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.