Shulou(Shulou.com)06/02 Report--

This article mainly introduces the relevant knowledge of "what are the knowledge points of load balancing technology". The editor shows you the operation process through an actual case, and the operation method is simple, fast and practical. I hope this article "what are the knowledge points of load balancing technology" can help you solve the problem.

1. Overview

Through the introduction of the previous article, it can not cover all the technologies of the load balancing layer, but it can be used as an introduction to tell readers a way to learn and use load balancing technology. Although we will turn to the introduction of the "business layer" and "business communication" layer later, the introduction of the load balancing layer will not stop. In the following time, we will be interspersed with the release of new articles on the load balancing layer, including the reintroduction of Nginx technology, HaProxy, new usage scenarios of LVS, and so on.

In this article, we summarize the previous knowledge points and intend to expand them so that readers can find new learning ideas.

2. The core idea of load balancing layer.

2-1. Selection of consistent hash and Key

We introduce the consistent hashing algorithm in detail. It is emphasized that consistent Hash algorithm is one of the most critical algorithms in modern system architecture, which is widely used in many fields, such as distributed computing system, distributed storage system, data analysis and so on. For my blog post, there will be it in the load balancing layer, the business communication layer, and the data storage layer.

The core of the consistency algorithm is:

Use one of the properties of the object (this property can be the IP address of the server, the open port can also be a user name, some kind of encrypted string. Any hash property you can think of), calculate an integer and distribute it to the power of 32 from 0 to 2.

Of course, one or some of the attributes of a server can also be calculated by hash, and according to the calculation of a certain point on the ring, that is, the blue dot on the ring in the figure.

When a processing request arrives, the hash is calculated according to one or some of the properties of the request, and is distributed at a certain point on the ring according to the calculation. That is, the yellow dot on the circle above.

We agreed that the requests represented by the yellow dots to the left of a blue dot An and to the right of a blue dot B were processed by the server represented by the blue dot A, thus solving the problem of "who will handle it". Under the premise of the stable existence of the blue dot, requests from the same Hash contract all fall in the same location, which ensures the stability of the service processing mapping.

When a blue dot goes offline for some reason, the yellow point affected by it is also limited. That is, the next request from the client will be processed by the server represented by other blue dots.

2-2, polling and right

Unweighted polling means that the master node (task source) assigns tasks according to the list order of the target node without considering any factors of the target node (such as CPU performance, disk performance, network performance). This is the simplest polling and the least complex polling for the master node. My previous blog posts "Architecture Design: load balance layer Design (2)-Nginx installation" and "Architecture Design: load balance layer Design (4)-LVS principle" all introduced this kind of minimalist polling: for example, the "rr" parameter in LVS.

The word "weight" in weighted polling can be regarded as the meaning of "polling" basis. "weight" can be a number of possibilities, it can be the quantitative value of the performance of the target machine, it can be a fixed number (weighted by a fixed number), and it can be the network speed of the target node. For example, the "lc" parameter in LVS is weighted according to the number of existing "connections" on the target machine: the smaller the number of connections, the greater the chance of getting processing rights for the task.

2-3, lease and health check-up

The main purpose of the lease agreement is to ensure the fact that if the server fails to check the client after the "latest time", then the server will definitely log out the login information of the client. At the same time, the connection information of the server on the client will disappear (and will no longer provide services down). Every time the inspection is successful, this "latest time" will be pushed back.

Like the hash algorithm we mentioned, lease agreement is also the most basic design idea in system architecture design, and it is widely used in various types of systems, and its working principle is what every architect needs to master. For example, zookeeper uses this protocol to ensure that the link between Flow node and Leader node is normal; distributed storage systems use this protocol to ensure that the connection between datanode and namenode is normal

3. Summary of load balancing layer technology

In the previous blog post, I focused on Nginx, LVS, and Keepalived technologies. Due to the limited time, here we summarize several technologies mentioned in the blog post, and then extend the introduction of DNS technology, CDN technology and hardware load technology.

3-1. Nginx technology

In the big chapter of the load balancing layer, I have three articles that directly introduce the principle and use of Nginx. But then a friend told me that he wanted to know more about Nginx, and specifically asked me to do another article about Nginx's dynamic cache. Yes, I plan to introduce Nginx's dynamic caching technology in the coming time, as well as the performance comparison between Nginx and several mainstream reverse proxy software. But it takes time, especially if I don't want to go to the Internet to find some existing performance comparison diagrams, or it is more reliable for me to do such a performance test while doing a performance report.

The following techniques have been highlighted in my blog post, so let's make a summary:

The limit of the number of connections in Nginx

Important configuration items include: worker_processes, worker_connections. But just configuring these properties is not enough, the most important thing is to turn on the "maximum number of files" limit at the operating system level. Use "ulimit-n 65535" to set the "maximum number of files" limit for this session; also use the "vim / etc/security/limits.conf" command to modify the kernel configuration information. Mainly the following two items:

* soft nofile 65535 * hard nofile 65535

Also note that it is used in conjunction with the "worker_rlimit_nofile" attribute in the nginx configuration item:

User root root; worker_processes4; worker_rlimit_nofile65535;#error_log logs/error.log; # error_log logs/error.log notice; # error_log logs/error.log info;#pid logs/nginx.pid; events {use epoll; worker_connections65535;}

Gzip Technology in Nginx

Gzip is a technology used by Nginx for HTTP Body data compression. The following Nginx configuration information is an example of enabling gzip compression:

# to enable gzip compression service, you need to apply for temporary memory space for gzipon;#gzip compression, assuming that the size after compression is less than or equal to that before compression. For example, if the original file size is 10K, it is more than 8K, so the memory allocated is 8 * 2 = 16K; for example, if the original file size is 18K, 16K is obviously not enough, then apply for memory according to the size of 8 * 2 * 2 = 32K. If it is not set, the default value is to request the same amount of memory space as the original data to store the gzip compression results. The minimum size of the original file compressed by gzip_buffers28k;#, that is, if the original file is less than 5K, then the http protocol version on which gzip_min_length6K;#gzip compression is based will not be compressed. The default is HTTP 1.1 gzip_http_version1.1;# gzip compression level 1-9. The higher the compression level, the higher the compression rate, the longer the compression time, the higher the CPU gzip_comp_level5. # the type of Content-Type Header that needs to be compressed by gzip. It is recommended that js, text, css, xml, json should be compressed; pictures are not necessary, gif, jpge files have been compressed very well, even if the pressure, the effect is not good, but also cost cpu. Gzip_typestext/HTMLtext/plainapplication/x-javascripttext/cssapplication/xml

The data returned by http for compression is useful in many scenarios:

A. If the browser is using a 3G/4G network, then the traffic is money to the user.

B, compression can save the external bandwidth of the server room and serve more users. According to the current market price, the bandwidth resources of the computer room are generally in 200RMB/Mbps, and the pressure of the server solution often comes from the computer room bandwidth.

C, not Nginx enabled gzip function, HTTP response data will certainly be compressed, in addition to meet the Nginx setting of the "need to compress http format", the client (browser) also needs to support gzip (otherwise how to decompress it), a good news is that most browsers and API support http compression.

Rewrite (rewriting) Technology in Nginx

The power of Nginx lies in its rewriting (repositioning) of URL requests. The rewrite function of Nginx depends on PCRE Lib, so be sure to install Pcre lib when Nginx compiles and installs.

Here is an example of a rewrite:

# example 1:location ~ * ^ / (. +) / (. +)\. (jpg | gif | png | jpeg) ${rewrite ^ / orderinfo/ (. +)\. (jpg | gif | png | jpeg) $/ img/$1.$2break; root / cephclient;} # location uses regular expressions to match $url without case sensitivity. Rewrite relocation is performed when the match is successful. The rule for # rewrite to rewrite url is that the content in the first parenthesis of the regex expression corresponds to $1 in the second parenthesis of the regex expression corresponds to $2, and so on. The meaning of this relocation is clear: relocate the file name in any directory to the corresponding file name in the img directory, # and immediately perform the rewritten URL location in this location (note that it is Nginx, not the client). # example 2:server {. . Location ~ * ^ / orderinfo/ (. +)\. (jpg | gif | png | jpeg) ${rewrite ^ / orderinfo/ (. +)\. (. +) $/ img/$1.$2last;} location / {root / cephclient }} # in server, there are two location locations. When url needs to access an image in the orderinfo directory, rewrite will rewrite the url,# and re-bring the url to server for execution, so that the "location /" location will be executed and find the directory where the image is stored.

Picture processing module of Nginx

Http_image_filter_module is the picture processing module of nginx, and it is the key reference technology that uses nginx to manage static resources and dynamic resources separately. Through this module, static resources can be scaled, rotated and verified.

It should be noted that the scaling images processed by the http_image_filter_module module are not saved, are calculated entirely using the node's CPU performance, and are temporarily stored using the node's memory. So if you want to use http_image_filter_module for image processing, be sure to adjust the nginx node according to the request scale of the client. And when the PV of the site reaches a certain scale, we must use CDN technology to accelerate access and plan the access processing means of pictures.

Since we did not explain the image processing module of Nginx in detail in the previous article on Nginx, we just said that we would like to introduce it, so here I give a more detailed installation and configuration example:

Nginx's http_image_filter_module module is supported by GD library, so to use this image processing module, you must install a third-party dependency package:

Yuminstallgd-devel

Then, the Nginx is recompiled:

Configure--with-http_image_filter_modulemake&&make install

An example of a configuration using the image processing module:

Location ~ * / (. +) _ (\ d +) _ (\ d +)\. (jpg | gif | png | ioc | jpeg) ${set$h$3;set$w$2; rewrite / (. +) _ (\ d +) _ (\ d +)\. (jpg | gif | png | ioc | jpeg) $/ $1.room4break; image_filter resize$w$h; image_filter_buffer2M;}

The syntax of regular expressions and the syntax of rewrite that have been introduced are no longer introduced, but mainly look at the property settings related to http_image_filter_module:

Image_filter test: test the validity of image files

Image_filter rotate: rotate the image, which can only be rotated according to 90 | 180 | 270

Image_filter size: returns the JSON data of an image

Image_filter resize width height: proportional reduction of the picture, note that it can only be scaled down, and the second reduction is proportional.

Image_filter_buffer: limit the maximum read size of images. If there is no setting, it is 1m. It is best to set it to 2M-3M according to different systems.

Image_filter_jpeg_quality: sets the compression ratio of jpeg images (1-99, the higher the better)

Image_filter_transparency: disables the transparency of gif and png images.

Other technologies / software similar to Nginx

At present, there are many software in the industry that solve similar problems with Nginx. They are Apache HTTP Server of Apache Foundation, open source Tengine of Taobao, Haproxy, including IIS running under Windows, and also support reverse proxy.

Here the author once again focuses on Tengine, suggesting that readers can use it when they have time, which is a software that has been deeply redeveloped for Nginx.

3-2. LVS technology

LVS is the abbreviation of Linux Virtual Server, which means Linux virtual server, which is a virtual server cluster system. This project was established by Dr. Zhang Wensong in May 1998.

LVS cluster adopts IP load balancing technology and content-based request distribution technology. The scheduler has a good throughput, it transfers requests to different servers evenly, and the scheduler automatically shields the failure of the server, thus forming a group of servers into a high-performance and highly available virtual server. The structure of the whole server cluster is transparent to customers, and there is no need to modify client-side and server-side programs.

In my series of articles, "Architecture Design: load balancing layer Design (4)-LVS principle", "Architecture Design: load balancing layer Design (5)-LVS single Node installation", "load balancing layer Design (7)-LVS + Keepalived + Nginx installation and configuration" all involve the explanation of LVS.

Here we summarize the three modes of operation in LVS:

3-2-1, NAT mode

NAT is a way in which the Datagram is received by the LVS Master service node and then transferred to the lower Real Server node. When the Real Server processing is completed, the Datagram is sent back to the LVS Master node and then forwarded by the LVS Master node. IPVSADMIN, the manager of LVS, is responsible for binding and forwarding rules and rewriting attributes in IP and TCP data packets.

The advantages of LVS-NAT mode are:

Configuration management is simple. The working mode of LVS-NAT is the easiest to understand, configure and manage among the three working modes of LVS.

To save public network IP resources, the number of IP allocated to users by the data center is limited, especially when the number of racks you purchase is small. LVS-NAT works by encapsulating your system architecture in a local area network. As long as LVS has a public network address or public network address mapping, access can be achieved.

The system architecture is relatively closed. In the intranet environment, our requirements for the setting of the firewall will not be very high, and it is relatively easy to operate and maintain the physical server. You can set the request from the public network to be filtered by the firewall and open to the request from the private network.

In addition, Real Server does not care about the authenticity of the data message that is rewritten and transferred to Real Server, as long as both the TCP check and the IP check can pass, Real Server can process it. So Real Server can be any operating system in LVS-NAT mode, as long as it supports the TCP/IP protocol.

3-2-2, DR mode

LVS's DR working mode is the most commonly used working mode in the current production environment, and there are the most materials on the Internet. Some articles explain the DR working mode more thoroughly:

The advantages of LVS-DR mode are:

The problem of forwarding bottleneck in LVS-NAT working mode is solved, and the larger load balancing scenario can be supported.

Compared with the consumption of off-network IP resources, the external network IP resources of the computer room are limited. If this problem does exist in the formal production environment, you can use the mixed use of LVS-NAT and LVS-DR to alleviate it.

LVS-DR certainly has its drawbacks:

Configuration is a little more troublesome than LVS-NAT mode. You need to at least understand the basic working mode of LVS-DR mode in order to better guide yourself in configuring LVS-DR mode and solving problems in the running process.

Due to the packet rewriting rules of LVS-DR mode, LVS nodes and Real Server nodes must be in the same network segment, because layer 2 switching can not cross subnets. However, for most system architecture scenarios, there is actually no essential limitation to this problem.

3-2-3, TUN mode

The working principles of LVS-DR mode and LVS-TUN mode are completely different, and the working scenarios are completely different. DR is based on Datagram rewriting, and TUN mode is based on IP tunneling, which is the reencapsulation of datagrams:

IPIP tunnel. A complete IP message is encapsulated into the data portion of another new IP message and sent to a designated location through a router. In this process, the router does not care about the contents of the original protocol that is encapsulated. After arriving at the destination, the destination relies on its own computing power and support for the IPIP tunnel protocol to open the encapsulation protocol and obtain the original protocol:

It can be said that LVS-TUN basically has the advantages of LVS-DR. On this basis, cross-subnet penetration is supported.

3-3. CDN technology

CDN technology Content Delivery Network: content distribution network. Why sometimes we are slow to access video resources and picture resources on the Internet, or even fail to access them. One of the important reasons is that the physical location of the resource is too far away from the client, and there may be four layers of NAT devices (equivalent to using Netcom's lines to access resources on the telecom server).

Let's imagine if we put the resource we want to access on a service closest to our client (for example, the resource accessed by the client in Guangzhou is in the computer room in Guangzhou). So does this solve the problem (this point is called "edge node"). This is the problem solved by the CDN network, as shown in the following figure:

At present, we do not need to develop CDN services, there are many companies in the market that provide free / paid CDN services (please type: CDN directly on google or Baidu, there will be a lot of "promotion" information, there will be no advertising in my blog post ^ _ ^). Of course, if you want to build your own CDN network, you can refer to the following technical solutions:

Squid:Squid is a software that caches internet data. It receives download requests from users and automatically processes the downloaded data. At present, the networks of many domestic CDN service providers are based on Squid.

Using Nginx's proxy_cache to build: rewrite technology in Nginx can actually realize URL request rewriting and request forwarding. The proxy_cache component in Nginx can make the source data requested from the remote end be saved locally, thus realizing the construction of a CDN network.

Write by yourself: the CDN network does not have a particularly complex technical threshold, if you have special needs, you can write one yourself. Of course, the CDN network introduced in the figure above belongs to the first-generation CDN network. Adding the second-generation / third-generation P2P technology to the CDN principle can form a second-generation CDN network: as shown below:

The third generation P2P technology, also known as hybrid P2P technology, is mainly to solve the processing pressure of metadata servers and accelerate the localization of resources. With regard to P2P technology, I will make a special introduction to a new topic after talking about "Business system Design" and "Business Communication system Design". In addition, it is mentioned that YouTube's P2P network is made by itself.

3. Summary of load balancing layer technology

3-4. Keepalived technology

Keepalived has never been introduced separately in these articles. This is because Keepalived is to monitor the working status of cluster nodes and complete the switch of slave devices if the service cannot be provided normally for some reason. There are two key points: monitoring the services provided on the node and completing the network handover. Keepalived itself does not provide business services, but only monitors whether the services provided are working properly, so since there are no services that can be monitored, why is it necessary for Keepalived to be used independently?

The following figure shows the working structure of Nginx + Keepalived and LVS + Keepalived:

The way Nginx + Keepalived works

The way LVS + Keepalived + Nginx works

Related technologies include:

Heartbeat is an important project in the Linux-HA project. It is more powerful than Keepalived, and its installation and management are relatively complex. There are a lot of materials on the network about the advantages and disadvantages of Heartbeat and Keepalived and the comparison of their use. But from my own experience, I prefer to use Keepalived for a simple reason: Keepalived is easier to install and configure, and is sufficient. In addition, the Redhat Rhcs suite can also build a similar HA cluster, but to be honest, I have not tried.

3-5, DNS polling and intelligent DNS

/ / TODO DNS technology has not been introduced yet

3-6. Hardware load

In this series of "load balancing layer Design" blog posts, the technologies we mentioned, such as Nginx and LVS, but not Haproxy, Squid and other technologies, are based on software load technology. F5 is a company whose BIG-IP LTM technology is based on hardware load. The hardware load scheme provides the performance space that software load technology cannot provide, and integrates NAT mapping, SSL acceleration, Cookie encryption, cache, attack filtering, packet filtering, dynamic Session retention and other functions that many software loads cannot provide (or functions that require a combination of multiple software to provide).

However, the hardware load scheme also has its disadvantages, mainly that the construction cost is relatively high, unlike the soft load, which can be expanded continuously according to the continuous increase of the system throughput. Of course, according to the throughput requirements of the system, you can use soft load in the early stage and hardware load in the later stage. In addition to the hardware load technology provided by F5 company, there are hardware load scheme of Citrix company and hardware load scheme of A10 company.

4. Combination of common load balancing technologies

Here we review the combination of commonly used load balancing technologies mentioned in this series of blog posts.

4-1. Independent Nginx/Haproxy

In a general WEB system, the first paragraph assumes a Nginx or Haproxy server, which can basically solve many problems, including load distribution.

4-2, Nginx + Keepalived or Haproxy + Keepalived or + Heartbeat

To ensure the stability of Nginx or HaProxy servers, you can use Keepalived or Heartbeat to make a simple hot backup solution.

4-3. LVS + (Keepalived | Heartbeat) + (Nginx | Haproxy)

With the increase of access pressure, we began to adopt a multi-layer load scheme, set up LVS services in the front section of Nginx or Haproxy, and ensure the continuous work of Keepalived through Keepalived or Heartbeat.

4-4, such as DNS polling technology or intelligent DNS routing technology

With the expansion of the technical solution to this stage, the daily tens of millions of PV can be fully supported. The prerequisite is that there is no problem with the program ^ _ ^.

If the traffic on your site is still larger or even several orders of magnitude higher, then congratulations, you must be one of the top 100 Internet companies in the world; but on the other hand, the problems you encounter may only be based on the characteristics of your company's business and find your own solutions. There are many such examples, for example, YouTube found that commercial CDN networks on the market could not meet their needs for video acceleration, so YouTube engineers wrote a CDN acceleration technology specifically for their own business; for example, Taobao found that there was no longer a distributed file system on the market to meet their needs for small file storage, so they wrote a TFS.

5. Other applications of load balancing technology.

In this series of articles, we will all use the client to request server-side processing using the HTTP protocol, where the client can enable the end user or a third-party system. But in fact, load balancing technology in the field of information processing, not only this request response layer is used, but also widely used in other technical fields. In this section, we will sort out these technologies as an extended topic.

5-1. Load balancing of relational database system

When it comes to relational databases, people naturally think of Oracle, MS SQL, DB2 and Mysql. In the field of mobile Internet, many companies are usually on the way to OEI. Here we are not going to discuss whether it is right to go to OEI or how to get to OEI. It is an indisputable fact that many mobile Internet companies are using Mysql databases.

It is true that the processing capacity of a single Mysql database can not catch up with Oracle, or even commercial databases such as MS SQL, but we can cluster Mysql to improve the performance of the entire data service. Mysql has supported "table partitioning" for a single data node since version 5.1.X, but this support is limited to the configuration of each node, improving read and write performance on a single Mysql (in conjunction with the underlying block storage selection, such as DAS). In order to achieve the performance of the entire Mysql cluster, you need to achieve read-write separation from a higher level.

Among them, there is a mature practice of Mysql cluster read-write separation, which is that one write node can be made into a Master node (the stand-alone performance of the Master node can be high, and the back end can use the DAS system); then multiple read nodes become Salve nodes, and accept synchronous logs from the Master node (MySQL Replication technology), and carry out the load of read requests through another LVS, and can cooperate with the "table partitioning" function on a single node. This practice can greatly enhance the overall performance of the database system on any system where more than 80% are read requests, as shown in the following figure:

As you can see from the figure above, the "write" operations from the program are submitted to the Mysql Master through a data source, while all read operations are distributed to the three Mysql Salve through the LVS-DR mode. Here are a few points to illustrate:

The data synchronization of Mysql Master and Mysql Salve is realized through MySQL Replication synchronization technology, which is an asynchronous synchronization based on operation log. Although the response time can not reach the "millisecond" level, it is basically very fast. If it is not the banking system, or the "second kill system" can basically satisfy the facts.

MySQL Replication reduces the performance of Mysql Master nodes by 20%, but diverts all read operations that Mysql Master was responsible for. Of course, when we introduce the "multi-master" approach and use HiveDB horizontal sharding in the future, we will also focus on how to improve the write performance of Mysql.

In fact, in the formal development architecture, we do not give programmers two data sources, which is not conducive to code management, but also increases the difficulty of development. We will use software such as Mysql-Proxy and Amoeba to implement the whole data source.

Later, when I introduce the data storage layer architecture, I will also introduce a variety of mature and reliable Mysql clusters, Mysql read-write separation, Mysql scale-out, and discuss with readers how to implement the operation and management of dozens of Mysql nodes.

5-2. Load balancing of distributed storage system

At present, there are many distributed storage systems, Ceph, Swift, MFS, HDFS. Some of them are based on object storage and some based on fast storage (in the blog post "Architectural layering of Standard Web Systems," I gave a more detailed introduction to block storage, file storage, and object storage, and we will talk about storage systems in more detail later). But they have one or more master nodes (some are called namenode, some are called master, some are called Metadata), and no matter what they call them, they all have some of the same functions:

Calculate the question of "where should the data be stored"

Coordinate and control the problem of "whether data is stored correctly"

Monitor the health status of "data nodes"

Transfer data

Answer the client's question of "where to get the data"

.

In the process of dealing with the problem, these control nodes actually play the role of load distribution. Their basic principle is to analyze the problem of "where the data should be stored" through the "consistent hash algorithm" (the attributes used for hash are different):

5-3. A more generalized load balancing system

Under the same passenger flow, the waiting time of queuing at multiple windows of a bank is certainly shorter than that of one window; for the same traffic flow, the passing rate of eight lanes is definitely higher than that of six lanes; dividing a task into multiple tasks and dealing with some of them by multiple individuals must be shorter than the time it takes for a person to do a big task.

The core idea of load balancing lies in diversion, the key problem lies in how to divert, and the evaluation criterion lies in the throughput after diversion.

This is the end of the content about "what are the knowledge points of load balancing technology". Thank you for your reading. If you want to know more about the industry, you can follow the industry information channel. The editor will update different knowledge points for you every day.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.