Shulou(Shulou.com)06/01 Report--

How to solve the problem of rapid delivery of applications in K8s, in view of this problem, this article introduces the corresponding analysis and solutions in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible method.

Preface

The software technology is updated quickly, but the goal we pursue is always the same, that is, under the premise of security and stability, increase the deployment frequency of applications and shorten the iterative cycle of product functions. this advantage is that enterprises can obtain product value in a shorter time, get customer feedback and respond to customer needs more quickly, so as to further enhance the competitiveness of products. In addition, enterprises can also release more resources into the research and development of innovative business and create more value, which is a virtuous circle process.

It is true that the rapid iteration of application products can bring us a variety of benefits, but the challenges also coexist with them. Higher frequency of application release means that online businesses have a greater risk of unexpected failures, in addition to fully testing and verifying iterative functions in the pre-release test environment before the product is launched, the formulation of the most appropriate application release strategy is another very important topic, because it can minimize the risk of business failure and losses.

Key points of Cloud Native Application delivery

We say that frequent product iterations mean a greater risk of failure, especially for traditional applications, especially for cloud native applications. Because cloud native applications are usually cloud-based distributed deployment models, and each application may be called by multiple functional components to provide complete services together, each component has its own independent iterative process and plan. In this case, the more functional components, the greater the probability of error. So how to improve these pain points at the application delivery level, we summarize the following key points for cloud native application delivery.

How to take full advantage of cloud native infrastructure. This advantage can be summarized into two points: elasticity and high availability.

How to have the ability to migrate and deliver across platforms. There are great differences in computing, storage and network resources at the bottom of the infrastructure. in the past, the difference in infrastructure was determined by upper-layer applications, while the delivery of cloud native applications required the ability to migrate and deliver across platforms.

How to realize the autonomy of application operation and maintenance. Autonomy is not equal to automation. Automation refers to triggering a process that automatically achieves a desired result at the end of the process, while autonomy means that if a copy of one of the functional components fails, the application can automatically remove the fault copy and supplement the new application copy when the highly available running state is applied.

How to make applications more predictable. The delivery end state of the application is predictable when we write the application orchestration template, and the risk is minimized if the delivery of the application becomes more predictable.

How to improve the average recovery time of applications faster. If the failure of the application beyond the scope of application autonomy requires human intervention, then the faster average recovery time means lower business loss.

Kubernetes is a portable, extensible, open source platform for managing containerized workloads and services that facilitate declarative configuration and automation. Its own platform capabilities have met most of the requirements we mentioned earlier. Kubernetes uses container technology to deploy applications, and such benefits include, but are not limited to:

More agile application creation and deployment

Portability

Environmental consistency

Loosely coupled and distributed

Resource isolation

Efficient and high-density utilization of resources

Kubernetes also provides powerful capabilities for application management, scheduling, monitoring, and operation and maintenance:

Service discovery and load balancing capabilities

Automatic deployment and rollback of applications

Autonomous repair ability of application

Storage choreography ability

Key and configuration management capabilities

But Kubernetes also has many functions that are not provided but can be extended, such as log collection, monitoring and alarm and so on. The following picture shows the architecture of Aliyun CCS, which enhances and enhances the capabilities that are closely related to users on the basis of supporting standard Kubernetes, including providing maximum flexible and low-cost global access, strong security architecture support, and deep integration of Aliyun basic resources and services, and has verified and precipitated a large number of user experience after double 11. At the same time, it supports proprietary, managed, non-service, edge and dragon bare metal and other product forms, all of our later demonstrations are done on this platform.

Boundaries of application delivery

What are the boundaries of application delivery in Kubernetes?

Starting from the simple point, we can think that the delivery of the application is its network service model, the back-end resources of the service and the persistent storage of business data, which are abstracted into service, deployment/pod,volume resources and so on.

Take a wordpress application as an example, which includes two functional components: the front-end component handles user requests and the back-end component stores data. The front-end component includes a frontend service and three pod, and the back-end component includes a backend service and a pod component, so the resources delivered by this wordpress application are two service and a total of four back-end pod. We manage this backend pod resource through deployment in Kubernetes. Service resource is equivalent to a load balancer that routes requests to the backend pod. It involves calls between components in the cluster and external users access services in the cluster, so there are different categories.

Depending on how the service is exposed, it can be divided into the following categories:

ClusterIP

The access within the cluster is realized by assigning a fixed virtual IP (Cluster IP) accessible within the cluster to the Service of Kubernetes. Is the most common way.

ApiVersion: v1kind: Servicemetadata: name: wordpressspec: type: ClusterIP # default service type, the service is only exposed as accessible within the cluster ports:-port: 80 # exposed to the service port inside the cluster targetPort: 80 # container listeners service port protocol: TCP selector: app: wordpress # forwards requests to the backend podNodePort with the same tag

NodePort maps the port of service to a port of the cluster node. If you do not specify this port, the system will choose a random port. Most of the time we should let Kubernetes choose the port, and it is too expensive for the user to choose the available port.

ApiVersion: v1kind: Servicemetadata: name: wordpressspec: type: NodePort # NodePort service Type The service exposes a fixed static port for external access to ports:-port: 80 # to the service port inside the cluster targetPort: 80 # container listens to the service port protocol: TCP nodePort: 31570 # outside the cluster can access the service through this port selector: app: wordpress # forward requests to the back-end pod with the same tag

Although the NodePort approach can expose services to access outside the cluster, it also has many disadvantages:

Each port can only be one service.

The port range is limited, usually 30000-32767

If the IP address of the node changes, you need to make some changes to adapt.

Therefore, this approach is generally not recommended in production, but it can be used if your application is cost-sensitive and tolerates the unavailability window of the service.

LoadBalancer

LoadBalancer is a standard way for services to be exposed outside the cluster or on the public network, but it relies on the capability of a load balancer provided by cloud provider. The load balancer allocates an ip address and listens to the specified port of the backend service, and the requested traffic is forwarded to the corresponding backend service through the specified port.

ApiVersion: v1kind: Servicemetadata: name: wordpressspec: type: LoadBalancer # LoadBalancer service, which generally depends on the load balancing capability provided by the public cloud vendor ports:-port: 80 # exposed to the service port within the cluster targetPort: 80 # container listening service port protocol: TCP selector: app: wordpress # forwards the request to the backend pod with the same tag

Ingress

ClusterIP service type is limited to intra-cluster communication. NodePort can expose service access entry, but each node occupies a port, which increases the complexity of port management. LoadBalancer usually requires the support of third-party cloud providers, which has certain constraints. The Ingress service type is different from the previous three service types. In fact, it is not a service type, but is similar to the existence of a cluster service entry. It can route traffic to the corresponding back-end service based on the different paths or subdomains you configure, which is more like a "smart routing" service.

We have previously introduced the resource types involved in some application publishing, as well as several modes of service resource types. How does service find the corresponding back-end pod? this is the function of tagging. We can label the pod and service of each application with the same tag. The mechanism of this tagging is the key point of several application publishing strategies we will talk about later.

Release strategy of application

In the Kubernetes cluster, in addition to selecting the service exposure mode according to the business requirements, in order to make the application provide services smoothly during the upgrade, it is very important to choose a correct publishing strategy.

Scroll publish

The first application publishing strategy is rolling publishing, which is also a more common strategy. It gradually deploys the new version of the application by replacing instances one by one until all instances are replaced.

As shown in the figure below, the current version of the service provided by my application is v1, and there are three copies of the back end of this service, but when I update version v2, it is a copy and a sub-local replacement begins until the back end of the final service is completely replaced with the v2 version.

The orchestration file for an application example is as follows:

Go-demo-v1.yaml

ApiVersion: apps/v1kind: Deploymentmetadata: name: go-demospec: replicas: 3 selector: matchLabels: app: go-demo template: metadata: labels: app: go-demospec: containers:-name: go-demo image: registry.cn-hangzhou.aliyuncs.com/haoshuwei24/go-demo:v1 imagePullPolicy: Always ports:-containerPort: 8080---apiVersion: v1kind: Servicemetadata: Name: go-demospec: ports:-port: 80 targetPort: 8080 name: go-demo selector: app: go-demo type: ClusterIP

Deploy version v1

$kubectl apply-f go-demo-v1.yaml

View pod running status

$kubectl get poNAME READY STATUS RESTARTS AGEgo-demo-78bc65c564-2rhxp 1 bank 1 Running 0 19sGodemomuri 78bc65c564-574z6 1 Running 0 19sgo-demo-78bc65c564-sgl2s 1 bank 1 Running 0 19s

Access to application services

$while sleep 0.1; do curl http://172.19.15.25; doneVersion: v1Version: v1

Update go-demo-v1.yaml to go-demo-v2.yaml and update mirror tag

... registry.cn-hangzhou.aliyuncs.com/haoshuwei24/go-demo:v2...

Deploy version v2

$kubectl apply-f go-demo-v2.yaml

You can see that pod will be replaced by the new version of pod one by one.

$kubectl get po-wNAME READY STATUS RESTARTS AGEapplication-demo-8594ff4967-85jsg 1 3m24sapplication-demo-8594ff4967-d4sv8 1 Running 0 3m24sapplication-demo-8594ff4967-d4sv8 1 Terminating 0 3m22sapplication-demo-8594ff4967-w6lpz 0 pound 1 Terminating 0 3m20sapplication-demo-b98d94554-4mwqd 1 Running 0 3sapplication-demo-b98d94554-ng9wx 0/1 ContainerCreating 0 1sapplication-demo-b98d94554-pmc5g 1/1 Running 0 4s

The access service will find that both versions v1 and v2 will be accessed during the rolling upgrade of the application, depending on the startup speed of the application.

$while sleep 0.1; do curl http://172.19.15.25; doneVersion: v1Version: v2Version: v1Version: v1Version: v2Version: v1Version: v1Version: v2

The advantage of scrolling publishing is that it is relatively simple and does not take up too much computing resources. The disadvantages are:

Versions are slowly replaced between instances

This rolling release may take some time

Unable to control traffic

In terms of the final state of the application in the cluster, there are either version 1 application backend or version 2 backend in the cluster; if version 2 is defective, then the online service is applied to the overall user. although we have a mechanism to roll back quickly, it is still too expensive for the whole user to use the failure.

Blue and green release

The second is the blue-green release. The blue / green release means that the back-end pod of application version 1 and version 2 are deployed in the environment and decide which version to release by controlling traffic switching. Compared with the rolling release, the application terminal under the blue-green publishing strategy can have both version 1 and version 2 pod. We can decide which version of the backend the current service uses through the switching of service traffic.

The orchestration file for an application example is shown below.

Go-demo-v1.yaml

ApiVersion: apps/v1kind: Deploymentmetadata: name: go-demo-v1spec: replicas: 4 selector: matchLabels: app: go-demo version: v1 template: metadata: labels: app: go-demo version: v1spec: containers:-name: go-demo image: registry.cn-hangzhou.aliyuncs.com/haoshuwei24/go-demo:v1 imagePullPolicy: Always ports:-containerPort: 8080

Go-demo-v2.yaml

ApiVersion: apps/v1kind: Deploymentmetadata: name: go-demo-v2spec: replicas: 4 selector: matchLabels: app: go-demo version: v2 template: metadata: labels: app: go-demo version: v2spec: containers:-name: go-demo image: registry.cn-hangzhou.aliyuncs.com/haoshuwei24/go-demo:v2 imagePullPolicy: Always ports:-containerPort: 8080

Service.yaml

ApiVersion: v1kind: Servicemetadata: name: go-demospec: ports:-port: 80 targetPort: 8080 name: go-demo selector: app: go-demo version: v1 type: ClusterIP

Deploy the above three resources

$kubectl apply-f go-demo-v1.yaml-f go-demo-v2.yaml-f service.yaml

If you access the service, you can see that only the version 1 service is currently accessed.

$while sleep 0.1; do curl http://172.19.8.137; doneVersion: v1Version: v1

Modify version=v2 under spec.selector of service.yaml

ApiVersion: v1kind: Servicemetadata: name: go-demospec: ports:-port: 80 targetPort: 8080 name: go-demo selector: app: go-demo version: v2 type: ClusterIP

Redeploy

$kubectl apply-f service.yaml

Revisiting the service, you can see that it quickly switched to version 2.

$[root@iZbp13u3z7d2tqx0cs6ovqZ blue-green] # while sleep 0.1; do curl http://172.19.8.137; doneVersion: v2Version: v2Version: v2

We just said that a rolling upgrade takes time, and even if it is rolled back, it takes a certain amount of time to roll back. In the case of defective applications in the new version, the blue-green release strategy can quickly cut traffic before v1 and v2, so the time to switch traffic is much shorter than that of rolling upgrades. However, the disadvantage of blue-green release is the same as that of rolling release is that this defect will affect the overall user. The service is either 100% switched to version 2 or 100% to version 1, which is an operation of either 0 or 100. Even if the blue-green release strategy can greatly shorten the failure recovery time, it is unacceptable in some scenarios. Moreover, there are two copies of pod in the cluster environment, which is twice as resource-intensive as a rolling release.

Canary release (grayscale release)

The third release strategy to be introduced is Canary release. Canary deployment is that application version 1 and version 2 are deployed in the environment at the same time, and user requests may be routed to the back end of version 1 or to the back end of version 2. In order to enable some new users to access the version 2 application. Under this release strategy, we can gradually control the switching of the application to the new version by adjusting the percentage of traffic. Compared with the blue-green deployment, it not only inherits the advantages of the blue-green deployment, but also consumes twice the resources required for the blue-green deployment. In the case of defects in the new version, only a small number of users are affected, and the loss is reduced to a minimum.

As for the concept of grayscale publishing, some people think that it is about the same thing as canary publishing, while others think they are different. It is the same as the release process of the canary, but with a different purpose:

Canary release is more inclined to get some feedback from users quickly. For example, I may not be sure whether the user experience of my new version can be well accepted by the public. I expect to get some timely feedback from online users. Iterate the v3 version after making functional experience adjustments on the product side

The grayscale release is that my product function has been designed and developed very well, now is to gradually replace the old online version, but to control the risk that the release may bring, so the grayscale release.

Application 1 of the example is as follows. In this example, we use the number of pod to control the traffic ratio.

Go-demo-v1.yaml sets the number of copies to 9

ApiVersion: apps/v1kind: Deploymentmetadata: name: go-demo-v1spec: replicas: 9 selector: matchLabels: app: go-demo version: v1 template: metadata: labels: app: go-demo version: v1spec: containers:-name: go-demo image: registry.cn-hangzhou.aliyuncs.com/haoshuwei24/go-demo:v1 imagePullPolicy: Always ports:-containerPort: 8080

Go-demo-v2.yaml sets the number of copies to 1

ApiVersion: apps/v1kind: Deploymentmetadata: name: go-demo-v2spec: replicas: 1 selector: matchLabels: app: go-demo version: v2 template: metadata: labels: app: go-demo version: v2spec: containers:-name: go-demo image: registry.cn-hangzhou.aliyuncs.com/haoshuwei24/go-demo:v2 imagePullPolicy: Always ports:-containerPort: 8080

Service.yaml

ApiVersion: v1kind: Servicemetadata: name: go-demospec: ports:-port: 80 targetPort: 8080 name: go-demo selector: app: go-demo type: ClusterIP

Deploy the above three resources

$kubectl apply-f go-demo-v1.yaml-f go-demo-v2.yaml-f service.yaml

To access the service, you can see that basically 10% of the traffic is switched to version 2.

$while sleep 0.1; do curl http://172.19.8.248; doneVersion: v1Version: v2Version: v1Version: v1

In addition, we can use nginx ingress controller to control traffic switching, which is more accurate.

Go-demo-v1.yaml

ApiVersion: apps/v1kind: Deploymentmetadata: name: go-demo-v1spec: replicas: 3 selector: matchLabels: app: go-demo version: v1 template: metadata: labels: app: go-demo version: v1spec: containers:-name: go-demo image: registry.cn-hangzhou.aliyuncs.com/haoshuwei24/go-demo:v1 imagePullPolicy: Always ports:-containerPort: 8080

Go-demo-v2.yaml

ApiVersion: apps/v1kind: Deploymentmetadata: name: go-demo-v2spec: replicas: 1 selector: matchLabels: app: go-demo version: v2 template: metadata: labels: app: go-demo version: v2spec: containers:-name: go-demo image: registry.cn-hangzhou.aliyuncs.com/haoshuwei24/go-demo:v2 imagePullPolicy: Always ports:-containerPort: 8080

Service-v1.yaml

ApiVersion: v1kind: Servicemetadata: name: go-demo-v1spec: ports:-port: 80 targetPort: 8080 name: go-demo selector: app: go-demo version: v1 type: ClusterIP

Service-v2.yaml

ApiVersion: v1kind: Servicemetadata: name: go-demo-v2spec: ports:-port: 80 targetPort: 8080 name: go-demo selector: app: go-demo version: v2 type: ClusterIP

Ingress.yaml, set nginx.ingress.kubernetes.io/service-weight: | go-demo-v1: 100,100: 0, version 1-100% traffic, version 2-0% traffic

ApiVersion: extensions/v1beta1kind: Ingressmetadata: annotations: nginx.ingress.kubernetes.io/service-weight: | go-demo-v1: 100 Go-demo-v2: 0 name: go-demo labels: app: go-demospec: rules:-host: go-demo.example.com http: paths:-path: / backend: serviceName: go-demo-v1 servicePort: 80-path: / backend: serviceName: go-demo-v2 servicePort: 80

Deploy the above four resources

$kubectl apply-f go-demo-v1.yaml-f go-demo-v2.yaml-f service-v1.yaml-f service-v2.yaml-f nginx.yaml

Access to the service can see 100% of the traffic on version 1.

$while sleep 0.1; do curl http://go-demo.example.com; doneVersion: v1Version: v1

Update ingress.yaml to set the traffic ratio to 50:50

ApiVersion: extensions/v1beta1kind: Ingressmetadata: annotations: nginx.ingress.kubernetes.io/service-weight: | go-demo-v1: 50 Go-demo-v2: 50 name: go-demo labels: app: go-demospec: rules:-host: go-demo.example.com http: paths:-path: / backend: serviceName: go-demo-v1 servicePort: 80-path: / backend: serviceName: go-demo-v2 servicePort: 80

To access the service, you can see 50% of the traffic on version 1, 50% on version 2.

$while sleep 0.1; do curl http://go-demo.example.com; doneVersion: v2Version: v1Version: v2Version: v2Version: v1Version: v1Version: v2Version: v2

Update ingress.yaml to set the traffic ratio to 0100

ApiVersion: extensions/v1beta1kind: Ingressmetadata: annotations: nginx.ingress.kubernetes.io/service-weight: | go-demo-v1: 0 Go-demo-v2: 100 name: go-demo labels: app: go-demospec: rules:-host: go-demo.example.com http: paths:-path: / backend: serviceName: go-demo-v1 servicePort: 80-path: / backend: serviceName: go-demo-v2 servicePort: 80

Access to the service can see 100% of the traffic to version 2

$while sleep 0.1; do curl http://go-demo.example.com; doneVersion: v2Version: v2

Whether it is canary release or grayscale release, the disadvantage is that the release cycle is relatively slow.

Among these release strategies

When you update an application in a development test environment, use a scrolling release

In a production environment, rolling updates or blue-green releases can be used if the new version has been fully tested in advance

If updates to the new version of the application need to maximize risk control and reduce the impact of failures on users, use canary or grayscale releases.

The above is the introduction of several release strategies that we commonly use in Kubernetes.

This is the answer to the problem of how to solve the problem of rapid delivery of applications in K8s. I hope the above content can be of some help to you. If you still have a lot of doubts to be solved, you can follow the industry information channel for more related knowledge.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.