Shulou(Shulou.com)05/31 Report--

How to build a control plane to manage Envoy management cluster network traffic, in view of this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible method.

Instruct to build a control plane at the service edge to manage the Envoy Proxy and use it as a service gateway or in a service grid

Envoy has become a very popular web component. Matt Klein wrote a blog post a few years ago about Envoy's dynamic configuration of API and how it has become one of the reasons why Envoy is becoming more and more adopted. He said in his blog post that this is the "unified data panel API" (UDPA). As many other projects adopt Envoy as their core component, it is no exaggeration to say that Envoy has not only established the standard API, but also for application layer 7 network solutions: "Envoy has become a unified data plane under the cloud native architecture."

Moreover, because of Envoy's unified data plane API, we can see that the industry has developed many management systems for configuration management of Envoy-based technical facilities. This article will discuss in depth what it takes to build a control plane for Envoy, and you can use this information to assess what kind of infrastructure is best for your organization and scenarios. Because this is a big topic, the author will produce a series of articles to explain it in detail (later I will also choose some articles that I am interested in for translation study).

There has been a lot of great discussion in the EnvoyCon/KubeCon forum, where many organizations have shared their experience with Envoy, including how to build their own control plane. Here are some reasons why they chose to build their own control plane:

Existing solutions are built on different data planes and already have a control plane that requires compatibility with Envoy.

Build without any open source infrastructure, or use other Envoy control planes (such as VMs, AWS,ECS, etc.).

You don't need to use all the features of Envoy, just some of them.

In order to better adapt to your workflow and working view, you need to develop a domain-specific API object model for Envoy configuration.

To use online, but found that other control planes are not mature enough.

However, just because some early users built their own control planes, that doesn't mean you should do the same thing. First of all, many of the control planes developed for Envoy in the middle of last year are quite mature, so you should study these existing control planes before deciding to redevelop another one. Second, as people at Datawire have found, and Daniel Bryant recently wrote, it's not that easy to build a control plane for Envoy.

I am involved in the development of several open source projects that build control planes for Enovy. For example, Gloo is a functional gateway that can be used as a powerful Kubernetes access service, an API gateway, or as a functional gateway for transition from single services to micro-services. Gloo has a control plane for Envoy, which can be used as an example of my series of articles to illustrate how to abstract the design on the control plane according to requirements to achieve plug-in management and extensibility management. Other implemented control planes that can be referenced, such as istio and Heptio Contour, are also good examples throughout my series of articles. If you are sure to develop your own control plane, then in addition to these, you can also refer to some other existing control planes.

In this series of articles, we will focus on the following key points:

A mechanism can be used to dynamically update the routing, service discovery and other configurations of Envoy.

Identify which components are used to make up your control plane, including back-end storage, service discovery API, security components, etc.

Set up the configuration objects and API for any given area in the most appropriate way according to the scene and the team organization.

Think about how to embed the control plane in the best way where needed.

How to deploy various control plane components.

Think about how to test the control plane.

To start this series of discussions, let's first look at how to update Envoy using Envoy's dynamic configuration API while Envoy is running to handle changes in topology and deployment.

Dynamically configure Envoy using Envoy's xDS API

The main convenience of building a control plane on top of Envoy is its data plane API. With the data plane API, we can dynamically configure most of the important runtime settings of Envoy. Envoy configuration through xDS API is designed to be ultimately consistent, and there is no way to update all agents in the cluster atomically. When there are configuration updates on the control plane, it makes the data plane agents available through xDS API, and each agent is independent of each other to obtain and apply these configurations.

Here are some of the runtime models that we can dynamically configure Envoy through xDS:

Snooping Discovery Service (LDS) API-LDS is used to send the port that the service is listening to.

Terminal Discovery Service (EDS) API- EDS user service discovery.

The Route Discovery Service (RDS) API- RDS is used for traffic routing decisions.

Cluster Discovery Service (CDS)-CDS is used for back-end services that can route traffic past.

Key Discovery Service (SDS)-SDS users distribute keys (certificates and keys).

These API are defined using proto3's Protocol Buffer, and there are already some related implementations that can be used for reference when building your own control plane:

Go-control-plane

Java-control-plane

Although each xDS (LDS/EDS/RDS/CDS/SDS, collectively referred to as xDS) is dynamically configurable, it doesn't mean you have to configure everything dynamically. You can combine adaptations to distinguish between static and dynamic configurations. For example, to implement a type of service discovery through configuration: you want the terminal to be dynamic, but the cluster already knows the routing information when it is deployed, so you can use Endpoint Discovery Service in Envoy to statically define the configuration of the cluster. If you are not sure which upstream cluster it is at the time of deployment, you can use Cluster Discovery Service to dynamically configure upstream discovery. The key is that you can build a workflow and process to statically configure the parts you need, and you can use dynamic xDS services to discover the parts you need at run time. One of the reasons why there are different control planes is that not everyone has a completely dynamic and alternative environment (where all configurations should be dynamic), which is almost impossible. According to the constraints of existing conditions and available workflows, take the appropriate level of dynamic configuration for your system, rather than full dynamic configuration.

In the implementation of Gloo, we build the control plane based on the implementation of go-control-plane, and realize the dynamic configuration from xDS API to Envoy. Istio and Heptio Contour also use this approach. The API of this control plane is implemented using gRPC streaming and leaves implementation interfaces, so we only need to implement these interfaces when we implement them. This approach integrates the Envoy data plane API into the control plane in a very efficient way.

GRPC streaming is not the only way to update the Envoy configuration. In xDS API in earlier versions of Envoy, polling was the only way to detect whether a new configuration was available. Although this is acceptable and in line with the principle of ultimate consistency of configuration updates, it is not efficient enough in network and computing use. It is also difficult to adjust and optimize the polling configuration to reduce the waste of resources.

Finally, some Envoy management systems are implemented by generating static Envoy configuration files and periodically overwriting Envoy to disk configuration files, and then performing a hot restart of the Envoy process. In a highly dynamic environment (like Kubernetes, virtually any short-lived computing platform counts), managing the generation, delivery, hot restart, and so on of such files can be cumbersome. Envoy initially operated in this way (that's what Lyft created the project), but it gradually evolved into the xDS API it is today.

It is a better way to use gRPC streaming and xDS API to realize dynamic configuration and control of Envoy. Similarly, not all Envoy configurations should be dynamic, especially if you don't need to configure them dynamically. But if you are in a highly dynamic environment (such as in Kubernetes), dynamically configuring Envoy is critical. Other environments may have the same need. In any case, using gRPC streaming API for dynamic configuration is ideal, with the following benefits:

Event-driven configuration updates; the configuration in the control plane is sent to Envoy when available.

There is no need to poll for configuration changes.

No hot restart of Envoy is required.

The traffic will not be interrupted.

This is the answer to the question about how to build a control plane to manage the network traffic of Envoy management cluster. I hope the above content can be of some help to you. If you still have a lot of doubts to be solved, you can follow the industry information channel for more related knowledge.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.