Shulou(Shulou.com)06/01 Report--

This article introduces the relevant knowledge of "kubernetes Cloud Native Application load Balancer selection Analysis". Many people will encounter this dilemma in the operation of actual cases. Next, let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Introduction

Application ingress traffic management has always been one of the focuses of development, operation and maintenance. With the development and change of computing resources, network environment and application architecture, the development of access layer traffic management solution can be roughly divided into two stages: traditional architecture and cloud native containerization. In order to meet the efficiency and requirements of application delivery, different access layer solutions have emerged in different stages, from the initial simple load balancing to reverse proxies such as HAProxy and Nginx, to all kinds of Kubernetes Ingress Controller in today's containerized environment. What are the characteristics of each stage of development? What are the challenges? What are the solutions?

Phase application deployment resource granularity application architecture application access addressing traditional architecture physical / virtual machine (low resource utilization) single or simple split module manages cloud native containerized container (high resource utilization) serviced container IP dynamic change based on fixed IP address, registers updates through dynamic service

In the traditional architecture stage, the business is a single application and a three-tier architecture; it is deployed on physical / virtual machines; the network environment is based on IP address management, which is relatively fixed and basically unchanged; the iterative speed of business updates is slow, and the main requirement of the access layer is to have layer 4 and layer 7 load balancing capabilities, which can be supported by traditional load balancers. With the evolution of application architecture (the application has been split into certain modules) and the improvement of iterative efficiency, there are some more complex access layer demands: routing according to traffic content characteristics, grayscale publishing, current restriction, authentication, etc. Generally, by adding a layer of network proxy (e.g. Nginx) support after the load balancer, the network proxy Nginx has more layer 7 traffic processing capabilities. The above advanced functions such as content routing, grayscale publishing, authentication and current restriction can be expanded through the Lua of the OpenResty community.

The ideal state of cloud native containerization phase is that business developers only need to focus on the implementation of business logic, do not care about resource scheduling and operation and maintenance management, and can be used on demand and billed by quantity. The granularity of virtual machine / physical machine resources is rough, and the utilization efficiency is low, so it is necessary to plan the computing, storage and network resources in advance, which is far from the ideal state. In the cloud origin stage, container resources have finer granularity, high utilization, startup / destruction speed up to seconds, and flexible scaling (Kubernetes has become the industry standard for container orchestration and scheduling. The following container environments all refer to Kubernetes clusters) The network management environment has also changed, and the concept of Service has emerged. a micro-service is often carried by a set of flexible and dynamically scheduled containers (Pod), and the IP address of Pod changes dynamically. This group of Pod generally provides access to Service, and traffic management is based on Service. Service-oriented split business module is easier to build applications, coupled with the good elastic scalability of the container environment, the concept of DevOps can be well implemented, and the iterative pace of micro-services is accelerated, often requiring rolling updates. At this time, ingress traffic management faces the following new challenges:

It needs to be integrated with Kubernetes to support forwarding traffic to the specified Pod.

The iterative speed of the update is accelerated, and the demand for the grayscale release of the new version of the service is stronger.

With the emergence of the cluster concept, the service discovery between clusters is isolated, and the access layer needs to support cross-cluster service discovery (that is, the access layer can select backend as the Pod of multiple clusters). This is different from the traditional physical / virtual machine phase, where there is no cluster isolation. As long as network connectivity is ensured, the back end of the access layer can be configured as the IP address of any corresponding service.

In the process of migration from traditional phase to cloud native phase, VM and container environment are mixed.

Based on the above challenges, the access layer traffic management solutions for the following container environments have emerged:

Kubernetes's official definition of Ingress API: established network agents (e.g. Nginx,HAProxy) or cloud vendors' load balancing products (e.g. AWS Elastic Load Balancer, Tencent Cloud CLB) all implement their own Ingress Controller as ingress traffic management solutions for a single cluster. According to the capabilities of Ingress Controller, such as grayscale publishing, authentication and current limit, it can be expanded through Annotation. Some Ingress Controller also designed their own traffic management model and syntax.

Service Mesh Ingress: the service discovery and management boundaries of the service grid are greater than the cluster latitude. Take Istio Ingress Gateway as an example, based on Istio's cross-cluster service discovery capability, backend can come from services of different clusters, and also supports services registered in the grid and running on virtual machines. Istio has also designed its own management model and syntax, declarative support for consistent north-south + east-west traffic management.

Use the network proxy deployed on the original VM to forward traffic to the VM service or the Kubernetes cluster service.

The following article will start with the usage scenario of ingress traffic management in cloud native containerization environment, and show you various solutions of cloud native access layer traffic management and their advantages and disadvantages.

Cloud native access layer traffic management scenario and solution scenario 1: basic traffic management

The first use scenario for ingress traffic management is the need to expose the service to the outside for client invocation. The common way is to expose the service by URL, for example, an e-commerce website needs to route the request of / login to the login service, the request of / product to the commodity service, and so on. This scenario requires the access layer to have the ability to route based on traffic content.

Solution: Load Balancer + NodePort

In the early stage of containerization, applications are deployed on both virtual machines and Kubernetes clusters, and many users use the original cloud load balancer (e.g. Nginx, Tencent Cloud CLB) to forward requests to virtual machines and containers respectively. Due to the container network solution, the original load balancer cannot directly access Pod IP, so the services in the cluster need to be exposed through NodePort.

However, the scheme has the following problems:

Limited number of NodePort ports (default 30000-32767)

With the expansion of the cluster size, the Nginx configuration file becomes more and more complex and difficult to manage.

After users publish the application to the Kubernetes cluster, they need to modify the Nginx configuration separately, so the experience is not native to the cloud.

Solution: Kubernetes Ingress

Kubernetes provides Ingress API [1] to expose HTTP services within the cluster, and Ingress supports routing requests to different Service based on Host and Path. For Ingress to work, the cluster must have a running Ingress controller e.g. Nginx Ingress Controller). Native Ingress syntax provides simple Host,Path-based routing, as well as the ability to configure TLS.

1. Based on Host routing

ApiVersion: networking.k8s.io/v1kind: Ingressmetadata: name: public-services namespace: defaultspec: rules:-host: service1.tencent.com http: paths:-backend: serviceName: service1 servicePort: 80 path: /-host: service2.tencent.com http: paths:-backend: serviceName: service2 servicePort: 80 path: /

two。 Based on Path routing

ApiVersion: networking.k8s.io/v1kind: Ingressmetadata: name: public-services namespace: defaultspec: rules:-host: services.tencent.com http: paths:-backend: serviceName: service1 servicePort: 80 path: / service1-backend: serviceName: service2 servicePort: 80 path: / service2

3. TLS configuration

Ingress also provides TLS support to expose HTTP services in the cluster as HTTPS. We need to first store the SSL certificate in the cluster in the form of Secret, and then use Ingress to reference the Secret we just created.

ApiVersion: v1kind: Secretmetadata: name: public-services-tls namespace: defaultdata: tls.crt: base64 encoded cert tls.key: base64 encoded keytype: kubernetes.io/tls---apiVersion: networking.k8s.io/v1kind: Ingressmetadata: name: services-with-tls namespace: defaultspec: tls:-hosts:-services.tencent.com secretName: public-services-tls rules: http: paths:-backend: serviceName: service1 servicePort : 80 path: / service1-backend: serviceName: service2 servicePort: 80 path: / service2

Kubernetes Ingress summary: for simple routing of HTTP traffic, it is very easy to configure using Ingress, which is also the reason why Ingress is popular (according to CNCF 2020 Cloud Native Survey report [2], 50% of users are or are about to use third-party agents for application traffic forwarding, of which Nginx and Envoy [3] are the most popular Ingress provider).

But on the other hand, the function of native Ingress is very limited, which can not meet the needs of many complex scenarios. Many third-party Ingress Controller [4] extends the functions of native Ingress through annotation or new configuration models and syntax, but it is still limited by the isolation of service discovery between clusters, so it can only be used as a single cluster ingress traffic management scheme.

Scenario 2: grayscale release

After the service can be exposed to external access, you also need to consider how to release the version and iterate smoothly and risk-free. The two common practices are to cut part of the traffic according to weight or traffic content to verify the stability of the new version, and then gradually transition to the new version, that is, the well-known grayscale release and AB test.

Kubernetes Ingress API does not have the feature of grayscale publishing natively. Nginx ingress controller extends the function of native Ingress API through annotation to achieve grayscale publishing, but this method does not well support the publishing strategy to control application traffic. In contrast, Istio CRD configuration is more flexible and easy to use. Here is how to use Istio VirtualService to configure grayscale publishing routing rules.

1. Based on weight

Istio can configure weight-based grayscale publishing through Virtual Service. The following is an example of configuring 95% ingress traffic from {namespace} / {gateway} to be routed to the current version of {service} and 5% to the canary version:

ApiVersion:... kind: VirtualServicemetadata: name: canary-weightspec: hosts: -'* 'gateways:-{namespace} / {gateway} http:-route:-destination: host: {service} subset: current weight: 95-destination: host: {service} subset: canary weight: 5

two。 Based on the request content

VirtualService also supports the configuration of content-based grayscale publishing routing rules. The following is an example of configuring ingress traffic from {namespace} / {gateway} header cookie "version=stable" to the current version of {service} and "version=canary" to the canary version of {service}:

ApiVersion:... kind: VirtualServicemetadata: name: canary-contentspec: hosts: -'* 'gateways:-{namespace} / {gateway} http:-match:-headers: cookie: exact: version=stable route:-destination: host: {service} subset: current-match:-headers: Cookie: exact: version=canary route:-destination: host: {service} subset: canary scenario 3: application traffic authentication and current restriction

Authentication and current restriction are two important capabilities to ensure the security and robustness of north-south traffic.

The access layer is a unified entrance to access back-end services. Ensuring the security of the access layer is an important scenario of access layer traffic management. Generally, authentication and authorization rules need to be configured at the entrance. Under traditional architecture, authentication and authorization function is generally implemented through code logic. Istio provides AuthorizationPolicy and RequestAuthentication CRD after 1.5, which allows flexible configuration of authentication and authorization rules in the entry layer.

1. Request authentication (JWT)

The Json Web Token carried by the authentication request at the entrance allows the request carrying a legal token and rejects the request carrying an illegal token.

The following is an example of a configuration that uses Istio RequestAuthentication to configure Ingress Gateway to carry legitimate JWT requests:

ApiVersion:.. kind: RequestAuthenticationmetadata: name: jwt-example namespace: istio-systemspec: selector: matchLabels: istio: ingressgateway jwtRules:-issuer: {issuer that issued the JWT} jwksUri: {URL of the provider's public key set to validate signature of the JWT}

two。 Authorization

Configure an authorization policy at the entrance to allow / deny traffic access according to the characteristics of the traffic content, such as configuring an IP blacklist / whitelist at the entrance; or an external authentication service, hoping that the ingress component can connect with the external authentication service and allow / deny traffic according to the authentication result returned by it.

The following is an example of configuring an IP block whitelist for Ingress Gateway using Istio AuthorizationPolicy:

ApiVersion:... kind: AuthorizationPolicymetadata: name: white-list namespace: istio-systemspec: selector: matchLabels: app: istio-ingressgateway action: ALLOW rules:-from:-source: ipBlocks: {single IP or CIDR}

Istio 1.9enhances the support of AuthorizationPolicy for interfacing with external authentication systems. Ingress Gateway can be configured to allow or deny traffic according to the results returned by external authentication systems.

ApiVersion:... kind: AuthorizationPolicymetadata: name: ext-authz namespace: istio-systemspec: selector: matchLabels: app: istio-ingressgateway action: CUSTOM provider: name: "my-ext-authz-service" rules:.

3. The scale of the current-limiting service is relatively large. When the back-end service is provided to many tenants, you need to control the rate of requests at the entrance. For example, each User ID can only request the "/ product" interface 100 times per minute. In order to use the current-limiting feature of Istio Ingress Gateway, you first need to install Ratelimit service, which can be implemented by yourself or directly use the community ratelimit [6], and then use Envoyfilter to configure current-limiting rules. For more information, please see the official document [7].

Scenario 4: ingress traffic management in multi-cluster heterogeneous scenarios

With the increase of business scale, or the improvement of requirements for disaster recovery, data compliance and isolation between services, services will consider and implement the deployment of multiple Kubernetes clusters, and even heterogeneous mixing of containerized and non-containerized environments, bringing a series of new challenges to ingress traffic management.

Multi-Kubernetes clusters are generally based on the considerations of disaster recovery and business isolation:

(1) disaster recovery. Kubernetes clusters have regional attributes, and the same application may be distributed in many different geographical regions according to the access validity and disaster recovery requirements of the service provided by application delivery. Disaster recovery of multiple (public) and hybrid cloud (IDC + public cloud) architectures also requires the deployment of multiple clusters. Cross-region and multi-cluster disaster recovery and nearest access can be provided through DNS resolution, but DNS has cache, and the actual effective time of failover may be longer, and it is impossible to cut part of the traffic to the backup region according to the health of the service, so you can only switch it all.

Istio is based on the following capabilities: 1. Multi-cluster service discovery capability; 2. Region awareness, fault awareness, disaster recovery traffic capacity planning, can be achieved: 1. When all cluster services are healthy, route to the corresponding service according to the request source; 2. When a partial failure occurs in the service of one cluster, a certain proportion of the traffic is transferred to the backup service of other clusters depending on the health of the service.

(2) business isolation. According to the CNCF 2020 Cloud Native Survey report [2], application isolation with multiple clusters is second only to namespace isolation, with the utilization rate rising from 47% in 2019 to 50% in 2020. When multiple services still share a traffic ingress, the access layer needs to have the ability to discover multi-cluster services and route traffic to the services of the specified cluster according to the specified policy.

Solution: Service Mesh Ingress

One challenge for Kubernetes Ingress Controller is that Kubernetes clusters isolate service discovery between clusters, and Ingress Controller can only be used as a cluster-level traffic portal. With the help of the ability of control plane service discovery, Service Mesh technology can discover or register multiple cluster services or even heterogeneous services, break through the service discovery barriers between clusters, is not limited by the application deployment platform, and naturally provide consistent access traffic forwarding management capabilities.

Istio as the most popular Service Mesh open source project, its access layer Istio Ingress Gateway also provides support for Ingress API, but it is not recommended to use Ingress to configure Ingress Gateway, which greatly weakens the ability of Istio. Istio provides a higher degree of abstraction to the traffic management model, and can directly use Istio API to achieve more flexible traffic management capabilities, such as gray publishing, cross-cluster routing, region awareness and other advanced features.

Istio Ingress Gateway is based on Envoy [3]. Envoy, originally created by Lyft, is a high-performance service agent software designed for cloud native scenarios. It was donated by Lyft to the CNCF community and graduated from CNCF.

1. Multi-Kubernetes cluster service management

Istiod can obtain endpoints information through the API Server of all clusters in the grid. After aggregating the information of multiple clusters, the resulting configuration can be pushed to Ingress Gateway,Ingress Gateway and the request can be forwarded to all Pod in the grid on demand.

two。 Region-aware load balancing

In the service grid, the geographic information of a Pod includes the following three parts [8]:

Region (region): usually represents a larger geographical area (e.g Beijing and Shanghai). In Kubernetes, the region of a node is determined by the tag topology.kubernetes.io/region

Zone (availability zone): a region usually contains multiple availability zones (e.g. Beijing Zone 1, Beijing Zone 2), in Kubernetes, the availability zone of nodes is determined by the tag topology.kubernetes.io/zone

Sub-zone: allows further partition of the availability zone to achieve more fine-grained control, for example, it can be divided according to rack (rack). There is no concept of sub-zone in Kubernetes. Istio uses the topology.istio.io/subzone tag of the node to define the sub-zone.

If you use a Kubernetes service hosted by a cloud vendor, the Region and Zone tags of the node have been configured by the cloud vendor. For example, in the TKE cluster, the nodes in Shanghai Zone 2 will have the following tags:

Topology.kubernetes.io/region: sh

Topology.kubernetes.io/zone: "200002"

The clusters in the grid may be distributed in different regions and different availability zones, and in most cases, we want to minimize cross-region / cross-availability zone request calls, as this will increase the request latency. Therefore, the access layer needs to have the ability to perceive endpoints geographic information and support the configuration of load balancing and failover policies according to geographic information.

(1) Regional failover

When regional load balancer is enabled, Istio will tell Ingress Gateway to forward the request nearby. When all instances are normal, the request will remain in the same location, and when the instance is abnormal, the traffic will be distributed to the instances in the next priority region.

For example, the priority of Ingress Gateway forwarding requests at bj.bj-01 is as follows:

Priority geographic location

1bj.bj-01Region Zone exactly matches 2bj.bj-02Region matches Zone does not match 3sh.sh-01/sh-02Region Zone does not match

(2) region-weighted load balancing

Region-weighted load balancer can distribute a certain percentage of user-defined traffic to certain regions. For example, we can use the following configuration to distribute traffic:

Global: localityLbSetting: enabled: true distribute:-from: bj/bj-01/* to: "bj/bj-01/*": 70 "bj/bj-02/*": 20 "sh/sh-01/*": 10

3. Heterogeneous service entry traffic management

In addition to multi-cluster, in the process of cloud native transformation, users often face the situation that some of the services have been containerized and run in the Kubernetes cluster, some of the inconvenient services are still in virtual machines, and some even use cloud vendor serverless cloud function service (e.g. AWS lambda). The access layer needs to have the ability to register / discover heterogeneous services to manage the north-south traffic of heterogeneous deployment services.

Services on virtual machines can be registered into the grid through WorkloadGroup and WorkloadEntry provided by Istio, and the same service can run on both Kubernetes clusters and virtual machines.

Summary of Istio Ingress Gateway: Istio Ingress Gateway provides support for multi-cluster service discovery, region awareness, traffic capacity planning, and more powerful and flexible traffic management API in scenarios such as ingress grayscale release, security, and multi-cluster heterogeneous traffic management, but at the same time, users have to face the complexity of Istio. Need to invest resources and manpower cost operation and maintenance Istiod and Istio Ingress Gateway, integrated metric,trace,log and other observability and certificate management surrounding system cost is high, but also need to correctly configure a variety of CRD (Gateway VirtualService DestinationRule, etc.).

Functional comparison of access layer solutions

The following is a comparison of the common access layer solutions in Tencent Cloud container environment.

Multi-cluster grayscale release / cross-cluster disaster recovery Demo

Below, you will use Tencent Cloud Service Grid TCM console to demonstrate the grayscale publishing and disaster recovery of Service Mesh Ingress in a multi-Kubernetes cluster environment.

Create a service grid and add two service discovery clusters for deploying services (basic monitoring metrics are automatically connected to Cloud Monitoring, which can be viewed on the console, and cloud native monitoring can be enabled depending on the situation to meet custom monitoring requirements), and select enable Ingress Gateway.

Use Destination Rule to define the version of the frontend service (the frontend service has the same deployment in both clusters)

Use Gateway to configure ingress gateway snooping rules, enable port 443 https access, and use Tencent Cloud SSL platform server certificate

Use VirtualService to configure routing rules, 50% traffic is routed to v1 version, 50% to v2 version

After there is an access request, check the workload (frontend,frontend-canary) monitoring. Both versions have traffic, with a ratio of about 1:1.

The grayscale ends, the weight is changed, and 100% of the traffic is routed to the v2 version. Check the monitoring data of the workload again and find that all traffic has been requested to frontend-canary.

Next, we simulate the failure of frontend service in one cluster by adjusting the number of Pod of frontend service workload in one cluster to 0. After we find that one cluster frontend service fails, we can still access the service normally. Looking at the workload monitoring of frontend service in the other cluster, we will find that the inbound bandwidth has doubled, indicating that after the service failure in one cluster, traffic disaster recovery has been switched to the other cluster.

If there is a need to expand east-west traffic management, you can inject envoy sidecar into the business, and you can use the same set of Istio API to achieve consistent management of north-south east-west traffic. Out of the box, you can use network topology, call tracking and other observability features.

Tencent Cloud Service Grid TCM is a fully Istio-compatible Service Mesh product of Tencent Cloud. Control plane component hosting has been implemented. When using TCM Ingress Gateway, you only need to deploy a set of data plane envoy pod in the business cluster to use all the ingress traffic management capabilities of the above Istio Ingress Gateway out of the box. At the same time, TCM integrates Tencent Cloud Monitoring and certificate peripheral products, providing out-of-the-box observability and certificate configuration features.

This is the end of the content of "kubernetes Cloud Native Application load Balancer selection Analysis". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.