Shulou(Shulou.com)06/02 Report--

This article will explain in detail how to carry out the actual combat and analyze the adaptive current limit of the Sentinel system. The content of the article is of high quality, so the editor shares it for you as a reference. I hope you will have a certain understanding of the relevant knowledge after reading this article.

1. Overview of Sentinel system adaptation

From the official understanding that Sentienl system adaptive current limit is a global concept, unified control of application ingress traffic, combined with the application's machine load, CPU utilization, the overall average response time, ingress QPS and the number of concurrent lines and other dimensions of monitoring indicators to determine whether to call for current limiting operation. In order to have an intuitive feeling, we can take a look at the adaptive current-limiting interface of the system from the official operation and maintenance platform:

RT, number of threads and entry QPS can be calculated statistically by collecting call information, so how to obtain the utilization rate of system LOAD and CPU? You can bring this question into the study of this article.

Before analyzing the principle of system adaptive implementation in detail, let's consider the purpose of introducing this mechanism into Sentinel.

The official document has been elaborated on this issue, let's first take a look at the official document on it.

The main purpose of introducing system adaptive current limit is as follows:

Ensure that the system will not be dragged down.

Ensure the throughput of the system on the premise of system stability.

At present, the current protection ideas we come into contact with are to set an index (wide value). For example, when the load load of the system exceeds a certain wide value, it will prevent or reduce the continued entry of traffic, and resume the entry of traffic when the system load drops to a certain level. It is usually passive, and its actual effect depends on whether the wide value setting is reasonable, but it is often not an easy thing to set it reasonably.

The system adaptation provided by Sentinel can use the set rules as a protection factor, and the traffic allowed to pass through is determined by the ability to process the request, that is, according to the response time of the request and the request rate being processed by the current system.

So how does Sentinel work? Then the principle of its implementation is revealed by means of source code.

2. The adaptive current limiting principle of the system.

The core entry class of current limit in Sentinel execution system is SystemSlot, which is easy to implement, as shown in the following figure:

SystemRuleManager#checkSystem can see that the key to the implementation is SystemRuleManager. Here, we directly call checkSystem to trigger its current limit, so let's focus on tracking the implementation of this method. 2.1 Adaptive current limit detection process

The adaptive current limit detection of the system is realized by SystemRuleManager's checkSystem method, and then its implementation details are analyzed in detail.

Step1: verify related resources, including three layers of verification:

If the resource name is empty, it is skipped directly, which is a fault-tolerant mechanism.

If the system adaptive switch is turned on, it is released directly. The switch is initialized as false, and when loaded into a system adaptive configuration rule, the state is set to true, as specified in loadSystemConf.

If the type of resource is not ingress traffic (EntryType.IN), it is released directly.

Step2: verify whether it needs to be restricted from the dimension of QPS. The key points are as follows:

For the current qps, 0 is returned if ENTRY_NODE is empty, otherwise the successful qps of the statistical node is returned, then the ENTRY_NODE statistical node is "who is it". It turns out that Sentinel specifically defines a resource whose name is total_inbound_traffic, which is used to collect the information of all ingress calls. When the resource enters type ENTRY_TYPE_IN, it automatically collects information. The specific statistical information is called in the entry method of StatisticSlot. The screenshot is as follows:

If the QPS currently called is greater than the set QPS, that is, the current limit is triggered, when is the qps set? It is also set when loading system current limiting rules. If multiple rules are set for the same current limiting point (LOAD, QPS), the minimum value takes effect.

Step3: the thread count and response time throttling mode is similar to QPS, so I won't repeat it.

Step4: if the load of the current system exceeds the set wide processing logic, this is the core of self-adaptation, which is not limited by the current load, but needs to be judged comprehensively according to the request processing capacity of the current system. The specific logic is implemented in the checkBbr method. How to obtain the system load and the checkBbr method will be described in more detail later.

Step5: if the current CPU load exceeds the set width value, and the current limit is triggered, how do you get the CPU utilization in JAVA? It will be described in more detail later.

2.2 detailed explanation of adaptive algorithm according to system load

As in step 4 above, the TCP BBR algorithm will be used to evaluate whether the current is limited according to the system Load. The specific implementation code is as follows:

SystemRuleManager#checkSystem

In Sentinel, the capacity of the system is estimated as a metric length of 1 s, which is expressed by the product of the maximum qps and the minimum response time passed in that second. The specific calculation details:

MaxSuccessQps's calculation multiplies the maximum value of the current sampling window by the number of sliding windows within 1 second, which is not very accurate.

The minimum response time for minRt is taken from the minimum response time in the current sampling window.

Therefore, the above calculation formula is obtained, divided by 1000 because the time unit of minRt is milliseconds, which is unified into seconds. It can be seen from here that the final judgment is based on the number of threads to limit the current according to the system load.

2.3 how to obtain operating system load from JAVA

The class that gets the load of the operating system in Sentinel: SystemStatusListener, collected once per second.

SystemStatusListener#run can get information about the operating system through com.sun.management.OperatingSystemMXBean in JDK.

Warm Tip: the above is only valid for Linux/Unix operating system, not for windows.

3. Practical thinking.

After the above analysis, the system adaptation in Sentinel actually refers to the adaptation according to the operating system load of the machine where the application resides, combined with the request processing capacity of the application itself. The load of the operating system can be output through the top command. Examples are as follows:

Although the configuration types of Sentienl system rules are divided into LOAD, CPU, RT, thread count, ingress QPS and other dimensions to limit current, the self-adaptation is mainly aimed at the case of LOAD.

The current limit rules at the Sentinel system level are not for a particular resource, but for all the resources that define EntryType.IN. You need to pay special attention when using them, especially if multiple applications are deployed on a machine, which may result in low load of the application itself. However, the current limit is triggered on the machine where the resource load is high due to other applications. On how to carry out the actual combat and analysis of the Sentinel system adaptive current limit is shared here, I hope the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.