Shulou(Shulou.com)05/31 Report--

This article mainly explains "which load balancing algorithms are used in nginx". The content of the explanation is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "which load balancing algorithms are used in nginx".

What is load balancing?

Load balancing, known as Load Balance in English, means to balance and distribute the load (work tasks) to multiple operation units, such as FTP server, Web server, enterprise core application server and other major task servers, so as to complete the work task together.

Load balancing usually serves two purposes: sharing stress and providing redundancy (which can also be understood as backup).

Life case list

If you still don't understand it, let's continue to use the list of life cases:

At the exit of the highway, if there is only one exit, suddenly a large number of vehicles (assuming that no one has handled the ETC) suddenly appear at the exit of the highway. For example, hundreds of taels of this exit will have to get off the highway, but you have to pay a toll to get off the highway, and each car will be delayed for at least a few minutes, hundreds of cars! It means that the next one may have to wait for several hours. What if there are multiple exits? Then there's no need to wait that long.

If an exit is added, at this time, the two exits can share the vehicle off the highway equally, and score the speed of the toll collector. Vehicle 3 should be faster when it sees car 1, and then follow car 1.

If you add another n, you can imagine the effect. But too many, it also seems to cause a waste of resources, many exits do not have a few cars in and out of a day, if you do too much, it will not be wasted, so we generally see that most of them are two, which is understandable for emergency use.

"We understand the driver as a load balancer, and we can judge which exit to take according to the road conditions ahead. The method of discrimination can be understood as the load balancing algorithm. "

In the terminology of our technical field, it is called redundancy. The speed of the toll collector can be understood as the performance of a service in our system.

Technical field

Here is a diagram to describe the load balancing in our technology field:

It is more sour to understand it by combining the scenes in life with those in the technical field.

Note: cluster refers to the deployment of multiple nodes in the same App application service, and the main purpose of the cluster is to share the pressure. A load balancer (system) can be understood as a commander. Make a request, and the commander gives the request to a service in the cluster according to a certain method. The commander can then assign requests to a service in the cluster in a variety of ways. The methods of random giving, queuing giving, quick response and so on, that is, the load balancing algorithm is formed.

The above analogy is only a personal understanding.

Types of load balancing

DNS

(Domain Name System Domain name system) as a distributed database that maps domain names and IP addresses to each other, it makes it more convenient for people to access the Internet. DNS uses TCP and UDP port 53. Currently, the limit for the length of each first-level domain name is 63 characters, and the total length of a domain name cannot exceed 253 characters. DNS is the simplest and most common load balancing method, which is generally used to achieve "geographical" load balancing. for example, northerners visit the computer room in Beijing, southerners visit the computer room in Guangzhou, and Westerners visit the computer room in Chengdu. The essence of DNS load balancer is that DNS can return different IP addresses when parsing the same domain name. For example, when using https://www.sina.com.cn/ in the north, it will be resolved as 10.210.1.12 (Beijing server room) and returned as 14.213.164.27 (Guangzhou server room) when used in the south.

Simple schematic diagram of DNS

Advantages

Simple configuration, no cost

The work of load balancing is handed over to the DNS server, saving the trouble of management.

Shortcoming

The addition and modification of records take some time to take effect (because DNS caches A records). Once a server goes down, it will take a long time to make it effective even if the A record is modified. During this period, DNS will still resolve the domain name to the offline server, resulting in user access failure.

The load cannot be distributed according to demand, and DNS does not know the real load of each server, so the load effect is not very good.

The actual situation: in the actual project deployment, we usually use DNS resolution for some servers and use domain name resolution as the first-level load balancing. Then use nginx load balancing as the second-level load balancing in the server.

Hardware load balancing

Hardware load balancing is a function of load balancing through separate devices, which are similar to router switches, or can be understood as a basic network device for load balancing. At present, there are mainly two kinds of hardware load balancing in the industry: F5 and A10. This kind of equipment has good performance and powerful function, but the price can be described as expensive. Generally, only large and rich enterprises such as banks and state-owned enterprises hold meetings to consider using this kind of equipment. I have only seen F5 in the bank. As for the A10, it will not withdraw without contact.

Advantages

Powerful: fully support all levels of load balancing, support a variety of load balancing algorithms, support global load balancing.

Good performance: it is good for general software load balancer to support 10w + concurrency, but hardware load balancer can support more than 100w + concurrency.

High stability: because it is a commercial product, it has been well tested and used on a large scale, so the stability is very high.

High security: hardware load balancing devices can not only handle load balancing, but also have firewalls, anti-DDOS attacks and other effects.

Shortcoming

The price is expensive: I remember that the bank spent millions on F5, and it is said that there are more expensive ones, so the price can be imagined.

Poor scalability: hardware devices can be configured according to the business, but cannot be extended and customized.

Software load balancing

Software load balancing realizes the function of load balancing through load balancing software. Common load balancing software are LVS and Nginx. Among them, LVS is the four-layer load balancing of the Linux kernel, and the difference between layer four and layer seven lies in their protocol and flexibility. Nginx is a layer 7 load balancer and supports the HTTP,E-mail protocol, while LVS is a layer 4 load balancer, so it has nothing to do with the protocol, and basically all applications can do it, such as chat, database and so on.

The following is a simple schematic diagram of Nginx's load balancing:

Advantages

Nginx is written by C, the same web server, takes up low resources and memory and high performance.

When the nginx server is started, a master process is generated, and the master process fork multiple worker processes, and the worker thread processes the client's request.

Nginx supports high concurrency, and each worker child process is independent and equal. When there is a client request, the worker process competes fairly, and the grabbed worker process will submit the request to the back-end server. When the back-end server does not respond in time, the worker process will continue to receive the next request. When the last request has a response, it will trigger an event, and the worker process will continue its previous execution until the response ends. One request will not be executed by two worker processes.

Nginx supports reverse proxy (user-aware access is called forward proxy such as youtube, and user unaware access is called reverse proxy such as load balancer), and supports layer-7 load balancer (extending the benefits of load balancer).

Nginx is asynchronous non-blocking processing request (the third point confirms), using the epollandqueue mode, apache is blocking processing request.

Nginx is fast to process static files (reason:

Nginx is highly modular and easy to configure.

Nginx is single-process multithreading).

Shortcoming

Compared with apache is unstable, because it is a single-process multi-threaded, process death will affect many users.

What is the use of load balancing?

The "traffic distribution" load balancer can distribute the traffic of multiple hosts, improve the business processing capacity of the user system and improve the service availability.

Session persistence during the session cycle, session persistence allows requests from the same IP or network segment to be distributed to the same back-end server.

"Health check" supports custom health check method and frequency, regularly checks the running status of back-end hosts, provides failover, and achieves high availability.

"load balancing" solves concurrency pressure and improves application processing performance (increase throughput, enhance network processing capacity)

Improve scalability by adding or reducing the number of servers to provide website scalability (scalability)

Improve security protection, do some filtering on load balancer, black and white list, hotlink protection, etc.

Common load balancing algorithms

Rotational training

After receiving the request, the load balancing system distributes the request to the server in a certain order. Rotation training is a simple load balancing algorithm strategy that does not pay attention to the status of the server.

Advantages: if the server is normal, then rotation training is ideal, because it will result in an equal number of requests for each service, which can be described as "rain and dew".

Disadvantages: the above is a little bit ideal, but the reality is often not like that, the reality is still very bony, online systems often have a variety of problems, such as: when a server is down, the rotation training algorithm will not take care of the server status, that is, it will lead to a large number of requests to a dead server, resulting in the system unavailable, resulting in the loss of users. Another common problem is that some servers respond quickly, some respond slowly (such as 32-core servers and 16-core servers), and the rotation training algorithm does not pay attention to the corresponding speed, so it will lead to slow response time of many service requests. it simply leads to poor user experience and may even drag down other systems due to slow response time.

Weighted rotation training

The load balancing system assigns request tasks to the corresponding server according to the server weight. The weight here is generally statically configured according to the system hardware configuration, and dynamic calculation will be more suitable for business. But the complexity is much higher than that of simple rotation training.

Weighted rotation training is a special way of training rotation, the main purpose is to solve the difference in the processing capacity of servers. For example, some servers in the cluster have 32 cores, while some old systems have 16 cores. In theory, we can configure its weight value, that is, the processing capacity of 32-core servers is twice that of 16 cores, and the weight ratio of load balancing algorithm is adjusted to 2:1, so that more requests are distributed to 32-core servers.

Weighted rotation training solves the problem that the wrong server can be better allocated according to the different tasks configured by the server in the rotation training algorithm. in fact, there will still be the problem that the request task can not be assigned according to the state difference of the server.

Lowest load priority

The load system allocates the request to the server with the lowest load. The load here can be measured by different metrics according to different request types and business processing scenarios. For example, the following scenarios

LVS, a layer 4 network load balancing device, can judge the status of the server by the number of connections. The greater the number of connections to the server, the greater the pressure on the server.

Nginx, a layer-7 network load balancing system, can judge the status of the server by the number of HTTP requests (Nginx's built-in load balancing algorithm does not support this method and needs to be expanded on its own).

If we develop our own load balancing system, we can choose the indicators to measure the system pressure according to the business characteristics. If CPU is intensive, the pressure on the system can be measured by CPU load; if it is IO-intensive, system pressure can be measured by IO load.

The lowest load priority algorithm solves the problem of being unable to perceive the server state in the rotation training algorithm, but the resulting cost is a lot of complexity, such as:

The algorithm with the minimum number of links first requires the load system to count the links of each server's current resume. Its application scenario is limited to any requests received by the load balancer will be forwarded to the server for processing. Otherwise, if there is a fixed connection pool between the load balancing system and the service, it is not suitable to adopt this algorithm. LVS can use this algorithm for load balancing, but a load balancing system that links database Mysql clusters through connection pooling is not suitable for load balancing.

The lowest priority algorithm of CPU load balancer requires the load balancer system to collect the specific load of each server's CPU in some way, and to determine whether it is based on an one-minute load standard or a 10-minute or 15-minute load standard. The optimal time interval of different services is also different. Too short time interval can easily cause frequent fluctuations, and too long time may cause slow response when the peak value comes.

The algorithm substrate with the lowest load priority can perfectly solve the shortcomings of the rotation training algorithm, but also because the load balancing system needs to perceive the current running state of the server after using the lowest load priority algorithm. At this time, the cost also increases a lot. For developers, the rotation training algorithm may be implemented with a short code, but the lowest load priority algorithm requires a lot of code to implement.

The lowest load priority seems to solve the shortcomings in rotation training, and then due to the increase in complexity, the proportion in use is not as good as rotation training or rotation training weighting algorithm.

The best performance

The lowest load priority algorithm allocates requests from the point of view of the server, while the optimal performance algorithm allocates requests from the point of view of the client, giving priority to assigning requests to servers with fast processing speed. in this way, the fastest response is achieved to the client.

In fact, the performance priority is similar to the lowest load priority, which needs to be aware of the state of the server. The difference is that the best performance is to sense the server state externally through the standard of response time. The same implementation complexity is also very high, mainly reflected in the following aspects:

The load balancing system needs to collect the response time of each request. In the case of a large number of request processing scenarios, this collection plus the statistics of response time itself will also consume the performance of the system.

In order to reduce this statistical consumption, we can take sampling statistics, that is, we do not have to count all the request time of all servers completely, but sample the response time of some tasks to estimate the response time of the overall request. Although sampling statistics can reduce the consumption of performance, it increases the complexity of implementation a lot, because in order to determine the appropriate sampling rate, too low sampling rate will lead to the correctness of data, and high sampling rate will also cause performance consumption. the complexity of finding a suitable sampling rate is also imaginable.

No matter all the statistics or sampling statistics, it is necessary to choose the appropriate period, which is the best performance in 30 seconds or the best in 1 minute? At present, there is no standard cycle, all need specific business scenarios to make decisions, do you feel its complexity, especially the online system needs to be constantly debugged, and then find out the relatively appropriate standards.

Hash class

The cloud load balancer system performs hash operations according to some keywords in the request, and the same result is worth distributing to the same server. The purpose of doing this is mainly to meet specific business needs, such as:

Source address Hash: assign requests from the same IP address to the same server for processing, which is suitable for businesses with transactions and sessions. For example, when we log in to an online bank through a browser, a session message is generated. The session is temporary and will expire when the browser is closed. The online banking backend does not need persistent session information, it only needs to keep the session temporarily on a certain server, but you need to ensure that users can access the same server for each request during the session. This business scenario is realized through the source address hash.

ID hash: assign the business represented by an ID to the same server for processing, such as userId session id. In the above example of online banking login, session id hash can be used to achieve the purpose of accessing the same server every time during the same session.

Application of load balancing algorithm

What load balancing algorithms are used in Dubbo?

Random LoadBalance (random algorithm, default)

RoundRobin LoadBalance (weighted rotation training algorithm)

LeastAction LoadBalance (minimum active calls algorithm)

ConsistentHash LoadBalance (consistent Hash method)

Class diagram

What load balancing algorithms are used in nginx?

"round robin (default)": polling method, which allocates requests to each backend server in turn, the default load balancing method. It is suitable for the situation where the performance of background machines is consistent. Dead machines can be automatically removed from the list of services.

"weight": distribute requests to different machines according to weight, and specify the polling probability. Weight is proportional to the access ratio, which is used in the case of uneven performance of back-end servers. For example:

Upstream bakend {server 192.168.0.14 weight=10; server 192.168.0.15 weight=10;}

"IP_hash": sending the request to the background server according to the hash value of the requester ip can ensure that the request from the same ip will be called to the fixed machine, which can solve the session problem. For example:

Upstream bakend {ip_hash; server 192.168.0.14 88; server 192.168.0.15 purl 80;}

"url_hash": divides requests into different machines according to the hash value of the requested url, which is efficient when the background server is cached.

For example: add hash statement to upstream, server statement can not write other parameters such as weight, hash_method is the hash algorithm used.

"fair": distributes requests according to the background response time, and requests with short response time distribute more requests. For example:

Upstream backend {server server1; server server2; fair;} Thank you for your reading. The above is the content of "what load balancing algorithms are used in nginx". After the study of this article, I believe you have a deeper understanding of which load balancing algorithms are used in nginx, and the specific usage needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.