Shulou(Shulou.com)06/02 Report--

Today, I would like to share with you the relevant knowledge of what the flow control method is under the high concurrency of Java. The content is detailed and the logic is clear. I believe most people still know too much about this knowledge, so share this article for your reference. I hope you can get something after reading this article.

At this time, if no protection measures are taken, the server will be subjected to great processing pressure, the number of requests is very high, and the server load is also very high, and when the request exceeds the server load limit, the system will crash, making it inaccessible to everyone.

In order to ensure the high availability of application services, a common method is to limit the flow of requests with large traffic (second kill / rush purchase), intercept most of the requests, and only allow some of the requests to enter the back-end server. This can prevent a large number of requests from crashing the system caused by excessive system pressure, so as to ensure the normal availability of the service.

Token bucket (Token Bucket), leaky bucket (leaky bucket) and counter algorithm are the three most commonly used current-limiting algorithms.

Current limiting algorithm

Counter

Counter current-limiting algorithm is also commonly used, which is mainly used to limit the total number of concurrency. For example, if the current limit qps is 100, the implementation idea of the algorithm is to start timing from the first request, and within the next 1 s, the count will be added to 1 for each request. If the cumulative number reaches 100, then all subsequent requests will be rejected. Wait until the end of 1s, restore the count to 0 and start counting again.

This implementation has a drawback: if I have passed 100 requests in the previous 10ms within 1 second per unit time, then the later 990ms can only refuse the request, which is called spurs.

Leaky bucket

In order to eliminate the spike phenomenon, the funnel algorithm can be used to limit the current. The name of the funnel algorithm is very vivid. There is a container inside the algorithm, which is similar to the funnel used in daily life. When the request comes in, it is equivalent to pouring water into the funnel. Then slowly and evenly flow out from the lower mouth. No matter how much traffic is above, the speed of the outflow below remains the same.

No matter how unstable the service caller is, the current is limited by the leaky bucket algorithm and the request is processed every 10 milliseconds. Because the speed of processing is fixed and the speed of incoming requests is unknown, many requests may come in suddenly, and requests that have not been processed in time are put in the bucket first. Since it is a bucket, there must be a capacity limit. If the bucket is full, new requests will be discarded.

In the aspect of algorithm implementation, a queue can be prepared to save requests, and requests can be obtained from the queue and executed periodically through a thread pool, and multiple concurrent executions can be obtained at once.

After using this algorithm, it also has some disadvantages: it can not cope with the sudden traffic for a short time, and its advantage is that it can smooth the burst traffic on the network, and the request can be integrated to form a stable traffic.

Token bucket

In a sense, token bucket algorithm is an improvement of leaky bucket algorithm, which can limit the rate of request calls, while token bucket algorithm can not only limit the average rate of calls, but also allow a certain degree of burst calls.

In the token bucket algorithm, there is a bucket that holds a fixed number of tokens. There is a mechanism in the algorithm to put tokens into the bucket at a certain rate. You need to obtain a token for each request call, and only if you get a token will you have a chance to continue execution, otherwise choose to wait for an available token or simply reject it.

The action of releasing tokens is carried out continuously. If the number of tokens in the bucket reaches the upper limit, the tokens will be discarded, so this is the case. There are always a large number of available tokens in the bucket, and the incoming request can be directly executed with tokens. For example, if the qps is set to 100, then one second after the initialization of the current limiter is completed, there are already 100 tokens in the bucket. At this time, the service has not been fully started. When the service is provided after startup, the current limiter can withstand 100 instantaneous requests. Therefore, the request waits only when there is no token in the bucket, which is equivalent to executing at a certain rate.

Implementation idea: you can prepare a queue to save tokens, and periodically generate tokens through a thread pool and put them in the queue. For each request, you will get a token from the queue and continue to execute.

Leaky bucket VS token bucket: the main difference between the two is that the "leaky bucket algorithm" can forcibly limit the data transmission rate, while the "token bucket algorithm" can not only limit the average data transmission rate, but also allow a certain degree of burst transmission. In the token bucket algorithm, as long as there is a token in the token bucket, it is allowed to transmit data abruptly until the user-configured threshold is reached, so it is suitable for traffic with burst characteristics.

Cluster current limit

Redis request window

Using the timing and counting method of redis, the maximum number of requests allowed to pass during the specified time window

For example, in order to limit the number of times a resource can be accessed by each user or merchant, it can only be accessed twice in 5s, or only 1000 times a day. This requirement cannot be achieved by standalone traffic restriction, so it needs to be implemented through cluster flow restriction.

How to achieve it? In order to control the number of visits, you must need a counter, and this counter can only be stored in a third-party service, such as redis.

General idea: every time there is a related operation, send an incr command to the redis server. For example, you need to limit the number of times a user can access the / index interface. You only need to concatenate the user id and the interface name to generate the key of redis. Each time the user accesses this interface, you only need to execute the incr command on this key, and the frequency of access at the specified time can be achieved with the expiration time on this key.

Nginx current limit

The Nginx speed limit module uses the leaky bucket algorithm, that is, it can forcibly ensure that the real-time processing speed of the request will not exceed the set threshold.

The official version of Nginx limits the connection and concurrency of IP. There are two modules:-limit_req_zone is used to limit the number of requests per unit time, that is, rate limit, and the leaky bucket algorithm "leaky bucket" is used. -limit_req_conn is used to limit the number of connections at a time, that is, the concurrency limit.

These are all the contents of the article "what is the flow control method under Java high concurrency". Thank you for reading! I believe you will gain a lot after reading this article. The editor will update different knowledge for you every day. If you want to learn more knowledge, please pay attention to the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.