Shulou(Shulou.com)06/03 Report--

The following gives you some opinions on load balancing, hoping to give you some help in practical application. Load balancing involves a lot of things, there are not many theories, and there are many books on the Internet. today, we will use the accumulated experience in the industry to do an answer.

What are network load balancers and agents?

Wikipedia defines load balancer as:

In computing, load balancing improves the distribution of workloads across multiple computing resources, such as computers, a computer cluster, network links, central processing units, or disk drives. Load balancing aims to optimize resource use, maximize throughput, minimize response time, and avoid overload of any single resource. Using multiple components with load balancing instead of a single component may increase reliability and availability through redundancy. Load balancing usually involves dedicated software or hardware, such as a multilayer switch or a Domain Name System server process.

Chinese version:

Load balancing (Load balancing) is a computer network technology used to distribute load among multiple computers (computer clusters), network connections, CPU, disk drives, or other resources to optimize resource usage, maximize throughput, minimize response time, and avoid overload. Using multiple CVM components with load balancing instead of a single component can improve reliability through redundancy. Load balancing services are usually performed by dedicated software and hardware.

The above definition includes not only the network, but also all aspects of computing. Operating system, network and container choreographer all have their own load balancing technologies, which are used to schedule tasks using their own resources. This paper only discusses the network load balancing.

1: Overview of network load balancing

This paper gives a high-level summary of network load balancing. Multiple clients initiate resource requests to multiple backends, and the load balancer is between the client and the backend. To put it simply, complete the following tasks:

Service Discovery: what backends are available in the system? The addresses of these backends (that is, how does the load balancer communicate with these backends)?

Health check: which backends are healthy and can be used to receive requests?

Load balancing: what algorithm is used to distribute independent requests to healthy backends?

Reasonable use of load balancing in distributed systems can bring many benefits:

Naming abstraction: each client no longer needs to know every backend (service discovery). The client can find the load balancer through a predefined mechanism, and then let the load balancer complete the naming resolution function. These mechanisms include built-in libraries and well-known DNS/IP/ port addresses, which are discussed in more detail later.

Error isolation: through health checks and other algorithms and techniques, load balancer routing methods can effectively bypass paralyzed or overloaded backends. In this way, operation and maintenance personnel can deal with errors more calmly in the face of system failures.

Costs and performance benefits: the consistency of networks in distributed systems is poor. The system usually spans multiple network areas. In the same area, network resources are usually undersold; in cross-regional cases, overselling is the norm (the discrimination between oversold and undersold is based on the comparison of the bandwidth that can be consumed on the network card and the available bandwidth between routers). Intelligent load balancing ensures that communication takes place in the same area as much as possible, thereby improving performance (reducing latency) and reducing overall system costs (reducing inter-area bandwidth and fiber requirements).

Load balancer vs proxy server

When it comes to network load balancers in the industry, the terms Load Balancer and Proxy are often treated in the same way, and these two terms are treated equally in this article (to show off: not all agents are load balancers, but load balancing is the primary function of mainstream agents).

Some people may ask that some load balancers are done as built-in functions of client libraries, and this load balancer is not a proxy server. This topic is easy to be confused, and this question is even more confusing. The topology of this load balancer is described in detail. This embedded load balancer is only a special case of the agent. The application completes the proxy function through the embedded library, which is different from the typical load balancer only inside and outside the process. Its overall abstraction is consistent.

Layer 4 (connection / session) load balancing

When discussing load balancing technology, the industry is often divided into two categories: L4 and L7. This classification comes from the definition of the four and seven layers of the OSI model. The OSI model can not well describe the complexity of the load balancing scheme. A four-layer load balancer often adds other levels of content in addition to the traditional four-layer protocol tasks such as UDP and TCP. For example, a layer-4 TCP load balancer also supports TLS. Does this count as layer-7 load balancer?

2:TCP terminal layer 4 load balancing

Figure 2 shows a traditional four-tier TCP load balancer. In this example, the client initiates a TCP connection to the load balancer, which terminates the connection (that is, responds directly to SYN), selects a backend, and creates a new TCP connection to the backend (that is, initiates a new SYN). The details in the figure do not need to be paid too much attention to, and the following four-tier load balancing chapters will be discussed in detail.

The key point of this section is that layer-4 load balancer generally operates only on layer-4 TCP/UDP. In general terms, the load balancer is responsible for manipulating these bytes to ensure that bytes in the same session only deal with the same back end. Layer 4 load balancing knows nothing about the application details in communication. The communication content can be HTTP, Redis, MongoDB, or any application protocol.

Seven-tier (application) load balancing

Four-tier load balancing is very simple and is still used in a large area. What is the disadvantage of four-tier load balancing that it requires seven-tier (application) load balancing? For example, the following four-tier cases:

Two gRPC/HTTP2 clients are connected to the back end, so this is done through a four-tier load balancer.

The layer-4 load balancer creates one outgoing TCP connection for each incoming TCP connection, so that there are two access and two outgoing connections.

However, the request frequency of client An is 1 request per minute, while the frequency of client B is 50 requests per second.

In the above scenario, the load level of the backend serving client An is only about 1max 3000 of the backend serving client B, which obviously violates the original intention of the load balancer. This happens in all multiplex, connection-keeping protocols (multiplexing means sending application requests in parallel in a single layer-four connection; keeping a connection means that the connection is not closed in the absence of an active request). For performance reasons (the cost of creating a connection is usually high, especially when using TLS to encrypt the connection), all modern protocols include these two features, so the load imbalance of layer 4 load balancing becomes more obvious over time. Seven-tier load balancing can solve this problem.

Figure 3: HTTP/2 layer 7 terminal load balancing

Figure 3 depicts seven layers of HTTP/2 load balancing. Here the client initiates a single connection to the load balancer. The load balancer is connected to two backends. When the client sends two HTTP/2 streams to the load balancer, the two streams are sent to different backends. In this way, a large number of different requests from the multiplex client will be efficiently distributed to multiple backends. This is why layer-7 load balancing is important to modern protocols (because of the insight into application traffic, layer-7 load balancing has many other benefits, which will be described in more detail later).

Seven-tier load balancing and OSI Model

As mentioned in the previous four-tier load balancing chapter, it is difficult to use the OSI model to describe load balancing. The seventh layer described by OSI covers the stink of many aspects of load balancing. For example, for HTTP, there are several lower levels:

Optional TLS. The network community has always been controversial about which layer of the OSI model TLS should belong to. For the convenience of discussion, we put it on the seventh floor.

Physical HTTP protocol (HTTP/1 or HTTP/2).

Logical HTTP protocols (Header, Body, and Trailer).

Message protocols (gRPC, REST, etc.).

A mature seven-tier load balancer should take care of each of the levels described above. Another seven-tier load balancer may support only a subset of the features in the seven-tier classification. All in all, the scope of layer 7 load balancer includes much more functions than layer 4 (only mentioned in the example that HTTP;Redis, Kafka, MongoDB and so on are also examples of layer 7 application protocols, all of which should benefit from layer 7 load balancer).

two

Load balancer characteristics

Here is a brief summary of the high-level functions of the load balancer. Not all load balancers provide all the features.

Service discovery

Service discovery is the process that the load balancer uses to determine the list of available backends. This function can be implemented in a variety of ways. Here are some examples:

Static configuration file.

DNS .

Zookeeper, Etcd, Consul, etc.

Unified data plane API for Envoy.

Health examination

The load balancer uses the health check function to detect whether the back end is available. There are two ways to achieve health check-up:

Proactive: the load balancer periodically sends ping to the backend (such as a HTTP request sent to the / healthcheck endpoint) to determine the health of the backend.

Passive: the load balancer determines health by analyzing the master data stream. For example, if a layer-4 load balancer finds three connection errors in a row, it will determine that a backend is unavailable; layer-7 load balancer may determine that the backend is unhealthy after three consecutive HTTP 503responses.

Load balancing

Yes, the load balancer must balance the load. With a series of healthy backends, how do you choose a backend to respond to a request or connection? Load balancing algorithm is an active research field, there are simple methods such as Round Robin, and there are complex schemes to judge by delay and back-end load. Power of 2 least request load balancing introduces one of the most popular load balancing algorithms that take into account both performance and simplicity.

Randomly select two backends, and further select the one with a lower load.

Session adhesion

There is an important requirement in some applications: requests for the same session need to reach the same backend. This is necessary for scenarios such as caching, temporary complex states, and so on. There are many forms of definition of "one session", which may include HTTP Cookie, properties of client connections, and other properties. Many seven-tier load balancers have some support for session adhesion. However, I think session adhesion is a naturally fragile situation (the back end of dealing with the session may be paralyzed), so more care should be taken in the design of systems that rely on sessions.

TLS terminal

The topic of TLS, whether it is edge services or inter-service communication, is worth writing about. Therefore, many layer 7 load balancers have done a lot of work in TLS processing, including terminal, certificate checksum binding, using SNI to provide certificates and other functions.

Observability

I often say, "observation, observation, observation." The network is inherently unreliable, and load balancers should provide status, tracking, and logs to help operators identify faults and fix them. There is a big gap in the detection output of the load balancer. The advanced load balancer supply includes a large number of outputs from digital statistics, distributed tracking, and custom logs. I don't think the enhanced observability falls from the sky, and the load balancer needs to do a lot of additional work to accomplish this task. The negative impact on performance is negligible compared to the benefits of this series of data.

Security and denial of service prevention

In the edge deployment topology (explained later), load balancers often need to implement a variety of security functions, including frequency restrictions, authentication, and DoS prevention (such as IP address marking and identification, Tarpitting, etc.).

Configuration and control plan

The load balancer should be configurable. This is an important workload in large-scale deployment. In general, the system used to configure load balancers is called the "control plane" and has a variety of implementations. This part is discussed more deeply in my work Service Mesh Data Plan vs Control Plan.

Plenty more

This part only gives some introduction to the functional level of load balancer. There will be more in-depth discussion of the seven-tier load balancer below.

three

Topological classification of load balancers

We have made some general introduction to the concept of load balancer, the difference between layer-4 and layer-7 load balancers, and a summary of load balancer characteristics. Next, we will discuss the deployment topology of load balancers in distributed systems (each of the following topologies applies to four-and seven-tier load balancers).

Intermediate agent

4: intermediate agent load balancing topology

The topology depicted in figure 4 is most familiar to most readers. This category includes hardware products such as Cisco, Juniper and F5; cloud software solutions such as Amazone's ALB and NLB and Google's Cloud Load Balancer, as well as pure software autonomy solutions such as HAProxy, NGINX and Envoy. The advantage of the intermediate agent scheme lies in the simplicity provided to the users.

In general, users only need to connect to the load balancer through DNS, and there is no need to worry about other situations; the disadvantage is that the load balancer has the risk of single point of failure, and it is also a possible performance bottleneck. The intermediate agent is usually a black box that is inconvenient for operation and maintenance. What's the problem? Is it client or physical network? Is it an intermediate agent or a backend? It's hard to define.

Edge agent

5: edge Agent load balancing Topology

Figure 5 is actually a variant of the intermediate agent, a load balancer that can be accessed through Internet. In this scenario, load balancers usually need to provide some additional "API gateway" functions, such as TLS terminals, frequency restrictions, authentication, and traffic routing. The advantages and disadvantages are similar to those of intermediate service agents. Edge servers are usually a necessary part of a large Internet-oriented distributed system.

The client usually uses a network library that is not controlled by the service provider and accesses the system through DNS (neither the embedded client library nor the Sidecar agent topology discussed later is suitable for running directly on the client). In addition, for security reasons, it is a common requirement to use a single gateway to provide inbound traffic for all Internet-oriented traffic.

Embedded customer base

6: achieve load balancing by embedding client libraries

In order to overcome the single point of failure and performance problems associated with intermediary agents, many mature architectures embed load balancing directly into the client library shown in figure 6. The features supported by different libraries vary widely, with the most well-known feature-rich products including Finagle, Eureka/Ribbon/Hystrix, and gRPC (roughly based on an internal system called Stubby, which is roughly based on Google). The advantage of this approach is that all load balancing features are fully distributed to each client, thus avoiding the single point of failure and performance bottlenecks mentioned earlier.

The weakness of this approach is also obvious that all languages used by an organization need to implement such client libraries. Under the distributed architecture, there will be more and more requirements for this kind of multilingual support. In this environment, the cost of implementing a network library for each language can be prohibitive. Finally, it is also a big challenge to upgrade the library in a large service architecture, and running different versions in parallel in the production environment will put more pressure on the operation and maintenance.

Combined with the above analysis of advantages and disadvantages, it can be seen that this architecture can be successful if it can limit the use of programming languages and overcome the pain of upgrading.

Sidecar Agent

Load balancing implemented by 7:Sidecar Agent

A variation of the embedded client library is the Sidecar agent topology in figure 7. In recent years, this topology has become increasingly popular with the concept of "Service Mesh". The idea behind the Sidecar agent is to gain the advantages of embedded client libraries without worrying about programming language locking at the expense of some delays caused by interprocess communication. At present, the popular Sidecar load balancers include Envoy, NGINX, HAProxy and Linkerd. My two articles, Introducing Envoy and Service Mesh Data Plan vs Control Plan, describe this structure in more detail.

Summary and advantages and disadvantages of different load balancer topologies

The intermediate proxy topology is the simplest and most typical way. His weaknesses are: single point of failure, telescopic bottleneck and black box operation.

Edge proxy topologies are similar to intermediate proxies and usually cannot be ignored.

Embedding client libraries provides the best performance and scalability, but it is challenging to develop in multiple languages and upgrade libraries for all services.

The Sidecar proxy topology is weaker than the embedded client library, but it also avoids two major difficulties in this approach.

In general, I think that in the case of service-to-service, the Sidecar proxy topology (Service Mesh) will gradually replace all other topologies. In order to handle the traffic before entering the Service Mesh, the edge agent topology will persist for a long time.

four

The present situation of four-tier load balancing

Is the four-tier load balancer still useful?

This article has talked about the benefits of many layer-7 load balancers for modern protocols, and the functional details of layer-7 load balancers will be discussed later. Does this mean that four-tier load balancers no longer need to exist? No. Although I think that eventually layer-7 load balancer will completely replace layer-4 load balancer in service-to-service scenario, layer-4 load balancer is still very meaningful for edge communication. This is because all modern large-scale distributed architectures use a two-tier four-layer / seven-layer load balancer architecture to handle Internet traffic. The benefits of deploying a separate four-tier load balancer before the seven-tier load balancer are:

The seven-layer load balancer needs more precise analysis, transformation and routing at the application level, and its ability to load the original traffic (packets per second or bytes per second) is weaker than that of the optimized four-layer load balancer. This reality makes it easier for layer-4 load balancers to deal with denial of service (such as SYN flood, general package flood, etc.).

Compared with the four-tier load balancer, the development of the seven-tier load balancer is more active, deployed more frequently, and there will be more Bug. With the front-end layer-4 load balancer, you can perform health check and emptying during the upgrade of layer-7 load balancers. Modern layer-4 load balancers usually use BGP and ECMP (explained in more detail in later chapters), and the deployment of layer-7 load balancers is usually relatively simple. Finally, because the seven-layer load balancer is relatively complex, so it is more prone to problems. With the front-end four-layer load balancer, we can use the routing function to bypass the fault node when there is a problem to improve the overall stability of the system.

I'll talk about centralizing middle / edge four-tier load balancers of different designs below. The designs mentioned later usually cannot be used in client libraries or Sidecar topologies.

TCP/UDP terminal load balancer

8: four-layer terminal load balancer

The first four-tier load balancer still in use is the terminal load balancer in figure 8. This is consistent with what we talked about when we introduced the four-tier load balancing. In this type of load balancer, two separate TCP connections are used: one between the client and the load balancer, and the other between the load balancer and the back end.

There are two other reasons for layer 4 terminal load balancers:

The implementation is relatively simple.

The connection terminal close to the client has a significant impact on performance. In particular, if the customer uses a lossy network (such as a cellular network), it is easy to retransmit before entering a stable wired transmission to the final destination. In other words, this load balancer is suitable for raw TCP connections in Point of Presence scenarios.

TCP/UDP transparent load balancer

9: transparent load balancer

Another type of four-tier load balancer is the transparent load balancer described in figure 9. In this type of load balancing process, the TCP connection is not terminated by the load balancer, and each linked packet is forwarded to a selected backend after connection analysis and network address translation (NAT) process. First, let's define connection tracking and NAT:

Connection tracking: the process of tracking all active TCP connections. This includes a lot of content, such as whether the handshake is completed, whether the FIN has been received, how long the connection has been in a daze, which backend the connection is forwarded to, and so on.

NAT: the process of using connection tracking data to change the IP/ port information of a packet to pass through the load balancer.

With connection tracking and NAT, the load balancer can forward client TCP traffic to the backend almost intact. For example, if the client communicates with 1.2.3.4VR 80, the selected backend is 10.0.0.2 9000. The client's TCP packet arrives at the load balancer at 1.2.3.4Val 80. The load balancer replaces the target IP and port with 10.0.0.2 IP 9000; it also replaces the source IP with the load balancer's IP. When the backend responds to the TCP connection, the packet is returned to the load balancer, and the link tracking and NAT in the load balancer come into play again, sending the packet back.

Why would you use this kind of load balancer instead of the terminal type mentioned earlier?

Performance and resource consumption: transparent load balancers do not terminate TCP connections and do not need to cache any TCP connection windows. The state storage for each connection is very small and an efficient hash table query is usually used. For this reason, compared with the terminal load balancer, the transparent load balancer can handle a larger number of active links and more packets per unit time.

Allow back-end congestion control: TCP congestion control is a mechanism that allows Internete endpoints to control the flow of outgoing data to prevent overuse of bandwidth and buffering.

Baseline for direct server return (Direct Server Return = DSR) and layer-4 load balancing cluster: transparent load balancing is also the basis for advanced layer-4 load balancing (such as DSR and clusters using distributed consistent hashes).

Direct Server Return (DSR)

10: four-tier DSR

Figure 10 shows DSR load balancing. DSR builds on the transparent load balancer mentioned earlier. DSR is an optimization scheme in which only inbound / request packets pass through load balancing; outbound / response packets bypass the load balancer and return directly to the client. The interesting thing about using the DSR scheme is that many payloads respond with a much larger amount of data than the request (such as typical HTTP requests and responses). Assume that 10% of the traffic is a request and the other 90% is a response. If DSR load balancer is used, only 1x10 capacity is needed to meet the system needs. In the past, load balancing was very expensive, and this solution could significantly reduce system costs and improve reliability (less is better). The DSR load balancer extends the concept of transparent load balancing in the following ways:

Because the response packet no longer passes through the load balancer, the connection tracking has only part of the data, and the load balancer cannot know the full TCP connection status. However, the specific status can still be inferred through the observation of customer data packets and the situation of timeout in a daze.

Load balancers typically use GRE to encapsulate IP packets and send them from the load balancer to the back end. In this way, when the backend receives the encapsulated packet, it can unpack and obtain the port and address of the client. In this way, the back end can send the response packet directly to the client across the load balancer.

An important part of DSR load balancer is that the back-end part participates in the load balancing process. The backend needs to configure the GRE tunnel reasonably, and configure its own connection tracking and NAT according to the low-level details of the network.

Note that there are many different designs for components such as connection tracking, NAT, GRE, and so on, whether it is DSR or transparent load balancer. These settings range from the load balancer to the back end. This part is beyond the scope of this article and will not be carried out in depth.

Fault tolerance using highly available pairing

11: fault tolerance using HA pair and connection tracking

By now, we have to consider the fault-tolerant design of four-tier load balancing. Both DSR and transparent load balancers require a certain amount of link tracking and state data to be stored in the load balancer. What happens when the load balancer goes down? All connections that pass through this load balancer will be disconnected. May have a significant impact on the application.

Historically, layer-4 load balancers are hardware purchased from typical vendors (Cisco, Juniper, F5, etc.). These devices are very expensive and can handle a lot of communication. In order to prevent all connection interruptions caused by a single point of failure and cause application failures, load balancers are usually deployed in a highly available pairing, as shown in figure 11. A typical high-availability load balancer configuration would satisfy the following design:

A pair of highly available edge routers provides a range of virtual IP (VIP). These edge routers use Border Gateway Protocol (BGP) to publish virtual IP. The primary edge router has a higher BGP priority than the backup edge router, so normally it handles all traffic (BGP is a super complex protocol; for ease of writing, it is understood that BGP is a mechanism that allows network devices to claim to be able to handle traffic from other network devices, and each connection has a weight, thus affecting connected traffic).

Similarly, the main layer 4 load balancer declares that it is available to the edge router with a higher BGP weight, so it usually handles all traffic.

Two edge routers and two load balancers are cross-connected, which means that if an edge router or a load balancer goes down, or if for some reason BGP is declared down, the backup device is accessed and carries all traffic.

The above settings are still used by many high-traffic applications on the Internet, but this solution has some side effects:

VIP must allocate traffic correctly through a highly available load balancer. If the capacity of a single VIP exceeds that of a HA pair, the VIP needs to be split into multiple VIP.

The utilization of system resources is very low. Usually half of the capacity is idle. In the past, load balancers were very expensive, so the idle cost was also very high.

Modern distributed system design requires a better fault-tolerant design than the active / standby mode. For example, a system needs to keep running in the event of multiple simultaneous failures. If a high-availability load balancer pair fails at the same time, it will lead to complete paralysis.

The hardware equipment provided by suppliers is expensive, which will lead to dependence on specific manufacturers. It is an urgent need to replace commercial hardware facilities with software solutions that support horizontal expansion.

Fault tolerance and scaling using distributed consistent hashes

12: four-layer load balancer and consistent hash for fault tolerance and scaling

The previous article introduces the fault-tolerant setting of the load balancer through highly available pairing and the problems that follow. At the beginning of the millennium, some large Internet infrastructure providers began to design and deploy new parallel four-tier load balancing systems based on the figure 12 model. The objectives of such systems are:

Solve all the problems caused by load balancing solutions designed using pairwise HA patterns.

Migrated from the proprietary hardware mode of the manufacturer's load balancer and replaced it with standard servers and network cards combined with commercial software solutions.

The design of the layer-4 load balancer is the best reference for fault tolerance and scaling via clustering and distributed consistent hashing. The specific working mode is as follows:

N edge routers declare all anycast VIP on the same BGP weight. Use ECMP (equivalent multipath routing) to ensure that all single Flow can reach the same edge router. A Flow is usually a tuple of source IP/ port and destination IP/ port. (simply put, ECMP is a way to distribute packets over a separate weighted network using consistent hash connections.) The edge router does not care which packets go where. It is generally preferred to send all packets from one Flow to the same set of connections to avoid performance loss caused by out-of-order packets.

N layer 4 load balancers declare all VIP on the same BGP weight to the edge router. Again, with ECMP, the edge router chooses the same load balancer for the same Flow.

Each layer-four load balancer performs partial connection tracking and selects a backend for this Flow using a consistent hash. Packets from the load balancer to the back end are encapsulated using GRE.

Next, use DSR to send the packet directly from the back end to the client through the edge router.

The consistent hashing algorithm used by four-layer load balancer is an active research field, which involves reasonable allocation, reducing latency, back-end change cost and minimizing memory expenditure. A complete discussion of this aspect is also beyond the scope of this article.

Let's take a look at how this design overcomes the shortcomings of HA pairing:

New edge routers and load balancers can be added on demand. Consistent hashes are used at all levels, minimizing the number of Flow affected when new machines are added.

While maintaining the ability to support sudden consumption and fault tolerance, it can improve the utilization rate of resources.

Both edge routers and load balancers can use ordinary hardware, which is only a fraction of the cost of traditional hardware load balancing (explained further below).

At this point, some readers may ask, "Why not let the edge router communicate directly with the back end through the ECMP? why do we need a load balancer?" . The answer is DoS prevention and the difficulty of back-end operation and maintenance. Without load balancer support, each backend needs to join the BGP, and it is difficult to make rolling updates.

All modern four-tier load balancing systems are moving towards this solution (or its variants). Two of the best known examples are Google's Maglev and Amazon's Network Load Balancer (NLB). No open source load balancer has implemented this design yet, but as far as I know, some companies are going to release such a product in 2018. I am looking forward to his appearance, and this product will fill an important gap in open source network components.

The current technical status of seven-tier load balancing

Current state of the art in L7 load balancing The proxy wars in tech currently is quite literally the proxy wars. Or the "war of the proxies". Nginx plus, HAProxy, linkerd, Envoy all quite literally killing it. And proxy-as-a-service/routing-as-a-service SaaS vendors raising the bar as well. Very interesting times!

Yes, it is. In recent years, we have seen a revival of the development of seven-tier load balancing / proxy technology. With the continuous development of distributed micro-service system, this aspect will continue to improve. From a basic point of view, the current reliance on unreliable networks is becoming more and more serious, which brings higher difficulties in operation and maintenance. In the long run, the heavy use of technologies such as auto-scaling and container orchestration means that the era of static IP and static files is over. The system not only uses the network more, but also is more dynamic, and needs new load balancing function. In this section, we briefly summarize the functions of the current seven-tier load balancer.

Protocol support

Modern seven-layer load balancers add explicit support for many different protocols. The more load balancers know about application communications, the better they can do functions such as monitoring output, advanced load balancing and routing. For example, Envoy explicitly supports layer 7 protocol resolution and routing in the range of HTTP/1, HTTP/2, gRPC, Redis, MongoDB and DynamoDB. More protocols, including MySQL and Kafka, will gradually be added.

Dynamic configuration

As mentioned above, the popularity of distributed systems requires dynamic configuration capabilities to improve the dynamic and control capabilities of the system. Istio is an example of such a system. Please refer to Service Mesh Data Plan vs Control Plan for more information on this.

Advanced load balancing

Layer 7 load balancers generally have built-in advanced load balancing support, such as timeout, retry, frequency control, circuit breaker, Shadow, buffering, content-based routing, etc.

Observability

As mentioned above, a common function of load balancing, more and more dynamic systems are deployed, and the difficulty of debugging is also increasing. Robust protocol specification observation output may be one of the most important functions to be provided by layer 7 load balancers in the future. Statistics, distributed tracking, and the output of defined logs have become a necessary requirement for device-tier load balancing solutions.

Expansibility

Modern seven-tier load balancers need to be able to easily add customized functions. This function can be achieved by writing pluggable filters and loading them into the load balancer. Many load balancers also support scripting extensions, such as Lua.

Fault tolerance

It took a lot of breath to talk about the fault tolerance of four-tier load balancers. What about the fault tolerance of seven-tier load balancers? In general, we think of a seven-tier load balancer as stateless and disposable. Using common software, the seven-layer load balancer can be simply extended horizontally. In addition, the complexity of traffic processing and state tracking of seven-layer load balancing is much more than that of four layers. Setting up HA pairing for seven-layer load balancers is technically feasible, but it can be very complicated.

Generally speaking, whether it is layer 4 or layer 7 load balancing, the industry is trying to move out of the HA pairing mode to a consistent hash-based horizontal expansion mode.

More

Seven-tier load balancers are developing rapidly. You can refer to the Envoy architecture overview.

five

Global load balancing and central control plane

13: global load balancing

In the future, more and more independent load balancers will be presented as commercial devices. I think the real innovation and business opportunities come from the control plane. Figure 13 shows a global load balancing system. There are some different things in this example:

Each Sidecar agent communicates with the backend of three different regions.

As shown in the figure, 90% of the traffic will be sent to area C, and area An and B will each get 5%.

Both the Sidecar agent and the back end report periodically to the global load balancer. In this way, the global load balancer can make precise decisions based on delay, cost, load, failure rate and other data.

The global load balancer periodically configures each Sidecar agent with current routing information.

Global load balancer can do some things that a single load balancer cannot, such as:

Automatic detection of partition faults and routing detour.

Apply global security and routing policies.

Machine learning and neural networks are used to detect and prevent abnormal traffic, including distributed denial of service.

Provide central interface and visual support to enable engineers to monitor and maintain the entire distributed system from an aggregation point of view.

For a global load balancer to become a reality, its data plane must have strong dynamic configuration capabilities. Please refer to my blog: Envoy's universal data plane API, and Service Mesh Data Plan vs Control Plan.

six

Conclusions, and the future of load balancers

To sum up, the main purpose of this article is:

Load balancer is a key component of modern distributed system.

There are two kinds of load balancers: layer 4 and layer 7.

Both layer 4 and layer 7 load balancers are relevant to modern architectures.

The four-tier load balancer is migrating towards a horizontal scaling scheme based on consistent hash.

Due to the growth of dynamic micro-service architecture, seven-tier load balancers have developed rapidly.

The split of control plane and data plane, and global load balancing are the future development direction and opportunity source of load balancing.

The industry is aggressively moving towards open source solutions for standard hardware and software. I believe that traditional load balancing vendors like F5 will be replaced by open source software and cloud providers. Traditional router and switch manufacturers, such as Arista/Cumulus, will have better development in the short term, but will eventually be replaced by common cloud providers and their own physical networks.

On the whole, I think this is the miracle age of the calculator network. Most systems begin to shift to open source and software, greatly increasing the speed of iteration. In the future, distributed systems will march towards serverless computing, and the complexity of the underlying network and load balancing system is bound to go hand in hand.

After reading some of the above opinions on load balancing, if there is anything else you need to know, you can find what you are interested in in the industry information or find our professional and technical engineer to answer it. Technical engineers have more than ten years of experience in the industry.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.