Shulou(Shulou.com)05/31 Report--

Today, I will talk to you about how to select the technical type of Kubernetes Ingress controller. Many people may not know much about it. In order to make you understand better, the editor has summarized the following contents for you. I hope you can get something according to this article.

Introduction: in the practice and deployment of Kubernetes, in order to solve the problems of Pod migration, Node Pod port and dynamic allocation of domain names, developers need to choose appropriate Ingress solutions. In the face of many Ingress products on the market, how should developers distinguish their advantages and disadvantages? And how to combine their own technology stack to choose the appropriate technology solution? In this article, Li Hui, the core R & D engineer of Tencent Cloud middleware, will introduce you to the technical selection of Kubernates Ingress controller.

01 noun interpretation

Reading below requires familiarity with the following basic concepts:

Cluster: a collection of cloud resources needed for a container to run, including several CVMs, load balancers and other cloud resources. Pod: an instance is made up of one or more related containers that share the same storage and network space. Workload (Node): Kubernetes resource object that manages the creation, scheduling, and automatic control of Pod replicas throughout the lifecycle. Service (Service): a microservice consisting of multiple instances of the same configuration (Pod) and rules for accessing these instances (Pod).

Ingress:Ingress is a collection of rules used to route external HTTP (S) traffic to a service (Service).

Current status of 02Kubernetes access

External access mode of Kubernetes

In Kubernetes, services and Pod IP are mainly used for service access within the cluster, but are not visible for applications outside the cluster. How to solve this problem? In order to enable external applications to access services in the Kubernetes cluster, the usual solutions are NodePort and LoadBalancer.

In fact, each of these two schemes has some disadvantages: the disadvantage of NodePort is that only one Service can be mounted on a port, and an additional load balancer needs to be built for higher availability. The disadvantage of LoadBalancer is that each service must have its own IP, whether it is intranet IP or extranet IP. More often, in order to ensure the capability of LoadBalancer, we generally need to rely on cloud service providers.

In the practice and deployment of Kubernetes, in order to solve the problems such as Pod migration, Node Pod port, domain name dynamic allocation, or Pod background address dynamic update, the Ingress solution is produced.

Shortcomings of 03Nginx Ingress

Ingress is a very important public network traffic portal in Kubernetes. The default value recommended in Kubernetes is Nginx Ingress, which I call Kubernetes Ingress in order to distinguish it from the commercial version of Ingress provided by Nginx later.

Kubernetes Ingress, as its name implies, is based on Nginx. Nginx is now the most popular Nginx HTTP Sever in the world. I believe everyone is familiar with Nginx, which is an advantage. It also has the advantage that very few configurations are required for Nginx Ingress to access the Kubernetes cluster, and there is a lot of documentation to guide you on how to use it. For most people who are new to Kubernetes or startups, Nginx Ingress is indeed a very good choice.

But when Nginx Ingress is used in some large environments, there are a lot of problems:

First problem: Nginx Ingress uses some of the features of OpenResty, but the final configuration load still depends on the original Nginx config reload. When the routing configuration is very large, Nginx reload will take a long time, as long as several seconds or even more than ten seconds, which will seriously affect the business, and even cause business interruption. The second problem: plug-in development for Nginx Ingress is very difficult. If you think that the plug-ins of Nginx Ingress itself are not enough, you need to use some customized plug-ins, and this additional development task is very painful for programmers. Because the plug-in capability and extensibility of Nginx Ingress itself are very poor.

04Ingress selection principle

Now that you've found a lot of problems with Nginx Ingress, are you considering choosing another open source, easier to use Ingress? There are at least a dozen Ingress on the market that are easier to use than Kubernetes Ingress, so how to choose the one that suits you from so many Ingress?

Ingress itself is based on HTTP gateways, and there are mainly several kinds of HTTP gateways on the market: Nginx, Golang native gateways, and the emerging Envoy. But each developer is good at different technology stacks, so the appropriate Ingress will be different.

So the question is, how do we choose a more useful Ingress? Or narrow it down to which Ingress should developers who are familiar with Nginx or OpenResty choose?

The following is to introduce some of my experience in selecting Ingress controllers.

1. Basic characteristics

First of all, I think the Ingress controller should have the following basic functions, if you do not even have these functions, then you can directly pass. Must be open source, non-open source can not be used. Pod changes frequently in Kubernetes, and service discovery is very important. Now that HTTPS is very popular, the capabilities of TLS or SSL are also very important, such as certificate management. Support for common protocols such as WebSocket and, in some cases, HTTP2, QUIC, and so on. two。 Basic software

As mentioned earlier, everyone is good at different technology platforms, so it is very important to choose a HTTP gateway that you are more familiar with. Such as Nginx, HAProxy, Envoy, or Golang native gateways. Because you are familiar with its principle, you can land quickly in use.

High performance is a very important feature in a production environment, but it is more important than high availability. This means that the availability and stability of the gateway you choose must be very strong. Only in this way can the service be stable. 3. Functional requirements

Putting aside the above two points, it is the special needs of the company's business for gateways. If you choose an open source product, you'd better be sure to use it out of the box. For example, if you need the ability to convert GRPC protocols, you certainly want to choose a gateway with such capabilities. Here is a brief list of the factors that affect your choice:

Protocol: whether to support HTTP2, HTTP3; load balancing algorithm: the most basic WRR, consistent hash load balancing algorithm can meet the demand, or need a more complex load balancing algorithm similar to EWMA. Authentication limit: only simple authentication, or more advanced authentication is required. Or integration is needed to quickly develop authentication features like Tencent Cloud IM. Kubernetes Ingress not only has the problems of long Nginx reload time and poor plug-in scalability mentioned above, but also has the problem that the back-end node's ability to adjust weights is not flexible enough.

Select APISIX

I personally recommend APISIX as an Ingress controller more than Kubernetes Ingress. Although it is much less functional than Kong, APISIX's good routing capabilities, flexible plug-in capabilities, and its own high performance can make up for some of the shortcomings in Ingress selection. For programmers based on Nginx or Openresty development, if you are not satisfied with the current Ingress, I recommend you to use APISIX as the Ingress.

How do you use APISIX as an Ingress? First of all, we need to make a distinction. Ingress is the definition of Kubernetes name or rule definition, and Ingress controller is a component that synchronizes the state of the Kubernetes cluster to the gateway. But APISIX itself is only an API gateway, how to implement APISIX into Ingress controller? Let's first take a brief look at how to implement Ingress.

To implement Ingress, there are essentially only two parts: the first part: you need to synchronize the configuration in the Kubernetes cluster, or the state in the Kubernetes cluster, to the APISIX cluster. Part 2: some concepts in APISIX, such as services, upstream, and so on, need to be defined as CRD in Kubernetes.

If the second part is implemented, APISIX can be generated quickly through the configuration of Kubernetes Ingress. APISIX-related configurations can be generated through APISIX Ingress controller. Currently, in order to quickly implement APISIX into an Ingress that supports Kubernetes, we have created an open source project called Ingress Controller.

Ingress controller architecture diagram

The image above shows the overall architecture of the Ingress controller project. The left part is the Kubernetes cluster, where you can import some yaml files to make changes to the configuration of Kubernetes. On the right is the APISIX cluster, as well as its control and data planes. As you can see from the architecture diagram, APISIX Ingress acts as the connector between the Kubernetes cluster and the APISIX cluster. It is mainly responsible for listening to the changes of nodes in the Kubernetes cluster and synchronizing the state in the cluster to the APISIX cluster. In addition, because Kubernetes advocates that all components should have high availability features, at the beginning of APISIX Ingress design, we use two-node or multi-node mode to ensure the high availability of APISIX Ingress Controller.

Horizontal comparison of all kinds of Ingress

Compared with the popular Ingress controllers on the market, let's simply compare the advantages and disadvantages of APISIX ingress. The picture above is a table made by foreign developers for Kubernetes Ingress selection. On the basis of the original table, combined with my own understanding, I added the function of APISIX Ingress. As we can see, the leftmost one is APISIX, followed by Kubernetes Ingress and Kong Ingress, and then Traefik, which is Golang-based Ingress. HAproxy is quite common and used to be a popular load balancer. Istio and Ambassador are two very popular Ingress abroad. Next, let's summarize the pros and cons of these Ingress:

The advantages of APISIX Ingress:APISIX Ingress have also been mentioned earlier. It has very strong routing capabilities, flexible plug-in expansion capabilities, and excellent performance. At the same time, its shortcomings are also very obvious, although APISIX has a lot of functions after open source, but the lack of landing cases, there is no relevant documentation to guide how to use these functions.

Kubernetes Ingress: the Nginx Ingress recommended by Kubernetes by default. Its main advantages are simple and easy to access. The disadvantage is that the time-consuming problem of Nginx reload cannot be solved at all. In addition, although there are many plug-ins available, the plug-in scalability is very weak.

Nginx Ingress: the main advantage is that it fully supports TCP and UDP protocols, but lacks other functions such as authentication, traffic scheduling and so on.

Kong: itself is an API gateway, it is also a pioneer, the introduction of API gateway to Kubernetes when Ingress. In addition, compared with the edge gateway, Kong does a very good job in authentication, current restriction, grayscale deployment and so on. Kong Ingress also has a big advantage: it provides some definitions of API and services, which can be abstracted as CRD of Kubernetes, and can be synchronized to the Kong cluster through K8S Ingress configuration. The disadvantage is that deployment is particularly difficult, and in terms of high availability, it is dwarfed by APISIX.

Traefik: Golang-based Ingress, which is a micro-service gateway itself, is widely used in Ingress scenarios. His main platform is based on Golang, and it supports a lot of protocols, so there are no shortcomings in general. If you are familiar with Golang, it is also recommended.

HAproxy: is a well-known load balancer. Its main advantage is that it has a very strong load balancing ability, and other aspects do not have an advantage.

Both Istio Ingress and Ambassador Ingress are based on the very popular Envoy. To tell you the truth, I don't think these two Ingress have any shortcomings. Perhaps the only disadvantage is that they are based on the Envoy platform, which we are not very familiar with, and the threshold for getting started will be relatively high.

After reading the above, do you have any further understanding of how to select the technical type of Kubernetes Ingress controller? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.