Shulou(Shulou.com)05/31 Report--

Xiaobian to share with you the technical selection skills of Kubernetes Ingress controller, I hope you have gained something after reading this article, let's discuss it together!

noun explanation

Reading this article requires familiarity with the following basic concepts:

Cluster: refers to the collection of cloud resources required for container operation, including several cloud resources such as Cloud Virtual Machine and Load Balancer. Instance (Pod): An instance consists of one or more related containers that share the same storage and network space. Workload (Node): Kubernetes resource object, used to manage the creation, scheduling, and automatic control of the entire lifecycle of Pod replicas. Service: A microservice consisting of multiple identically configured instances (pods) and rules for accessing those instances (pods).

Ingress: Ingress is a set of rules for routing external HTTP (S) traffic to a Service.

Kubernetes Access Status

External access to Kubernetes

In Kubernetes, service and Pod IP are mainly used by services to access within the cluster and are invisible to applications outside the cluster. How to solve this problem? To enable external applications to access services in Kubernetes clusters, the usual solutions are NodePort and LoadBalancer.

In fact, these two schemes each have some disadvantages: the disadvantage of NodePort is that one port can only mount one Service, and in order to achieve higher availability, an additional Load Balancer needs to be built. The disadvantage of LoadBalancer is that each service must have its own IP, whether it is an intranet IP or an extranet IP. In more cases, in order to ensure the ability of LoadBalancer, it is generally necessary to rely on cloud service providers. In the practice and deployment of Kubernetes, in order to solve problems such as Pod migration, Node Pod port, domain name dynamic allocation, or Pod background address dynamic update, Ingress solution was created.

Disadvantages of Nginx Ingress

Ingress is a very important external network traffic portal in Kubernetes. The default value recommended in Kubernetes is Nginx Ingress. In order to distinguish it from the commercial version of Ingress provided by Nginx later, I will call it Kubernetes Ingress.

Kubernetes Ingress, as its name suggests, is based on Nginx platform. Nginx is now the most popular Nginx HTTP Server in the world. I believe everyone is familiar with Nginx. This is an advantage. Another advantage is that Nginx Ingress requires very little configuration to access Kubernetes clusters, and there are plenty of documentation to guide you on how to use it. For most people new to Kubernetes or startups, Nginx Ingress is indeed a very good choice.

But when Nginx Ingress is used in some large environments, there will be many problems: the first problem: Nginx Ingress uses some of the features of OpenResty, but the final configuration load still depends on the original Nginx config reload. When the routing configuration is very large, Nginx reload will take a long time, up to a few seconds or even ten seconds, which will seriously affect the business, or even cause business interruption. The second problem: Nginx Ingress plug-in development is very difficult. If you think Nginx Ingress itself is not enough plugins, you need to use some custom plugins, this extra development task is very painful for programmers. Because Nginx Ingress's own plug-in capabilities and scalability are very poor.

Ingress selection principle

Since Nginx Ingress has a lot of problems, is it considered to choose other open source, better Ingress? There are at least a dozen Ingress on the market that are better than Kubernetes Ingress, so how do you choose the right one from so many Ingress?

Ingress itself is based on HTTP gateways, and there are mainly several HTTP gateways on the market: Nginx, Golang native gateways, and the newly emerging Envoy. However, each developer is good at different technology stacks, so the appropriate Ingress will be different.

So the question is, how do we choose a better Ingress? Or narrow it down, developers familiar with Nginx or OpenResty, which Ingress should they choose?

Here are some of my experiences with Ingress controller selection.

selection principle

1. basic characteristics

First of all, I think the Ingress controller should have the following basic functions. If you don't have these functions, you can pass it directly. What must be open source cannot be used without open source. Pods change very frequently in Kubernetes, and service discovery is very important. HTTPS is now very popular, TLS or SSL capabilities are also very important, such as certificate management functions. Support common protocols such as WebSocket, and in some cases, HTTP2, QUIC, etc. 2. As mentioned earlier in the basic software, everyone is good at different technology platforms, so it is also very important to choose an HTTP gateway that is more familiar to you. For example, Nginx, HAProxy, Envoy or Golang native gateway. Because you are familiar with its principle, it can achieve rapid landing in use. In a production environment, high performance is an important feature, but more important than that is high availability. This means that the gateway you choose must have very strong availability and stability, and only in this way can the service be stable. 3. Functional requirements aside from the above two points, it is the special requirements of the company's business for the gateway. You choose an open source product, preferably one that works out of the box. For example, if you need the ability to convert GRPC protocols, then of course you want the gateway you choose to have such a function. Here is a brief list of the factors that influence choice:

Protocol: whether HTTP2 and HTTP3 are supported; Load Balancer Algorithm: whether the most basic WRR and consistent hash Load Balancer algorithms can meet the requirements, or whether more complex EWMA Load Balancer algorithms are needed. Authentication flow: Simple authentication or more advanced authentication is required. Or integration is needed to quickly develop authentication functions like Tencent Cloud IM. Kubernetes Ingress not only has the problems mentioned above that Nginx reload takes a long time and plug-in scalability is poor, but also has the problem that the ability of backend nodes to adjust weights is not flexible enough.

Select APISIX

Compared to Kubernetes Ingress, I personally recommend APISIX as an Ingress controller. Although it is much less functional than Kong, APISIX has good routing capabilities, flexible plug-in capabilities, and its own high performance, which can make up for some shortcomings in Ingress selection. For programmers based on Nginx or Openrest, if you are not satisfied with Ingress now, I recommend you to use APISIX as Ingress.

How do I use APISIX as Ingress? We will first make a distinction, Ingress is the definition of Kubernetes name or rule definition, Ingress controller is a component that synchronizes Kubernetes cluster state to the gateway. But APISIX itself is just an API gateway. How can APISIX be implemented as an Ingress controller? Let's start with a brief description of how Ingress is implemented. To implement Ingress, there are essentially only two parts: The first part: You need to synchronize the configuration in the Kubernetes cluster, or the state in the Kubernetes cluster, to the APISIX cluster. Part 2: Some concepts in APISIX, such as services and upstream, need to be defined as CRDs in Kubernetes. If the second part is implemented, APISIX can be generated quickly by configuring Kubernetes Ingress. APISIX-related configurations can be generated via the APISIX Ingress controller. In order to quickly implement APISIX into Ingress that supports Kubernetes, we created an open source project called Ingress Controller.

Ingress controller architecture diagram The figure above shows the overall architecture diagram of the Ingress controller project. The left part is the Kubernetes cluster, where you can import some yaml files to change the Kubernetes configuration. On the right is the APISIX cluster with its control and data planes. As you can see from the architecture diagram, APISIX Ingress acts as a connector between Kubernetes clusters and APISIX clusters. It is mainly responsible for monitoring the changes of nodes in Kubernetes cluster and synchronizing the state in cluster to APISIX cluster. In addition, since Kubernetes advocates that all components should have high availability characteristics, at the beginning of APISIX Ingress design, we ensure that APISIX Ingress Controller guarantees high availability through dual-node or multi-node mode.

Summarize the horizontal contrast of various Ingress

Compared to the popular Ingress controller on the market, let's briefly compare the advantages and disadvantages of APISIX ingress. The above picture is a table made by foreign developers for Kubernetes Ingress selection. I added APISIX Ingress function to the original table based on my own understanding. We can see that APISIX is on the far left, followed by Kubernetes Ingress and Kong Ingress, and Traefik is Golang-based Ingress. HAproxy is more common and used to be a popular Load Balancer. Istio and Ambassador are two very popular Ingress abroad. Let's summarize the advantages and disadvantages of each of these Ingress:

APISIX Ingress: The advantages of APISIX Ingress have also been mentioned above. It has very powerful routing capabilities, flexible plug-in expansion capabilities, and excellent performance. At the same time, its shortcomings are also very obvious, although APISIX open source has a lot of features, but the lack of landing cases, there is no relevant documentation to guide you how to use these features.

Kubernetes Ingress: Nginx Ingress recommended by Kubernetes by default. Its main advantages are simplicity and easy access. The disadvantage is that Nginx reload takes a long time and cannot be solved. In addition, although there are many plug-ins available, the plug-in extension ability is very weak.

Nginx Ingress: The main advantage is that it fully supports TCP and UDP protocols, but it lacks other functions such as authentication mode and traffic scheduling.

Kong: It is itself an API gateway, which is also a pioneer, introducing API gateway into Kubernetes as Ingress. In addition, Kong does a very good job in authentication, current limiting, gray deployment, etc. compared to edge gateways. Kong Ingress also has a great advantage: it provides some API and service definitions, which can be abstracted into CRD of Kubernetes, and synchronization state can be completed to Kong cluster through K8S Ingress configuration. The downside is that deployment is particularly difficult, and it pales in comparison to APISIX in terms of high availability.

Traefik: Golang-based Ingress, which itself is a microservice gateway, is widely used in Ingress scenarios. His main platform is based on Golang, and there are many protocols that he supports. Generally speaking, there are no shortcomings. If you are familiar with Golang, it is also recommended to use it.

HAproxy: is a prestigious Load Balancer. its main advantage is its very strong Load Balancer capability, but otherwise it does not dominate.

Both Istio Ingress and Ambassador Ingress are based on the very popular Envoy. To be honest, I don't think these two Ingress have any shortcomings. Perhaps the only drawback is that they are based on Envoy platform. Everyone is not very familiar with this platform. The threshold for starting will be relatively high.

After reading this article, I believe you have a certain understanding of "What are the technical selection skills of Kubernetes Ingress controller". If you want to know more about it, please pay attention to the industry information channel. Thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.