Shulou(Shulou.com)06/02 Report--

I told you the principle of building an image before, so how can we quickly deploy it to a kuberentes cluster after we have it?

In the early days, everyone was used to using the kubernetes interface, or the cli command line to do this, but the complexity of yaml file syntax, a large number of containers and the concept of kuernetes are difficult to understand and undoubtedly become no obstacles to containerized delivery.

To solve these problems, Huawei Cloud Container Service has launched a wizard-based image deployment, which reduces the difficulty of deploying images for the first time to a certain extent through step-by-step guidance, shielding or abstraction of complex concepts, but the flexibility is relatively poor. For scenarios that often change versions, you still need to use native interfaces for operation, so that the complexity of the overall R & D delivery process is not reduced much.

In view of this situation, Huawei Cloud Container team has launched another service product, container delivery pipeline, which is easier to use and more automated, which is based on container technology and implements the concept of DevOps. It provides continuous integration and delivery capabilities around container business end-to-end scenarios, including code compilation, image construction and delivery, automatic deployment, upgrade rollback, and provides full capacity interfaces. It is easy to combine with the existing processes of the enterprise, at the same time, the interface shields the underlying container concept, and the interface is re-encapsulated, and the interface definition is closer to the CI/CD business concept, so that users who are familiar with the CI/CD process can quickly switch.

As an important part of the product, the release component directly affects the entire release cycle. Today, we will talk about some of the implementation principles of this component.

01

Kubernetes Deployment

Kubernetes Deployment provides an official method for updating Pod and ReplicaSet (the next generation Replication Controller). We can fill in the configuration and version information we need in the Deployment object, and the Deployment controller changes the actual state to the desired state, for example, to upgrade nginx:v1.0 to nginx:v2.0, we just need to create a Deployment,Kubernetes and upgrade automatically according to Deployment. With the iterative update of Kubernetes, the cloud container engine service currently provides several versions of clusters, so how to make the deployment components support different cluster versions? Since our CI/CD tool provides yaml page editing for deployment, the deployment component will be based on the apiversion in deployment: apps/v1, apps/v1beta1, extensions/v1beta1. Provides different versions of the API interface. The self-encapsulating interface makes the publishing component have strong initiative and is not subject to the third-party library.

02

How to judge the success of the release?

After solving the version problem, the most important question is how to determine the release result of the component. For a CI/CD tool, the criteria for judging the success of a workload are not just that the Pod is in the running state or that the workload is available. What we need to make sure is that the image or configuration of the workload is running in a new version. So we judge that the sign of success should be that the new pod is in the running state, so how to find these new pod? The following figure shows the relationship between Deployment,ReplicaSet and Pod. Taking a stateless workload as an example, we query the "deployment.kubernetes.io/revision" of Annotations in deployment to find the ReplicaSet controlled by Deployment according to this revision. Finally, look for these new pod according to ReplicaSet's label. We have found these new Pod, so now we need to analyze the state of pod.

The PodPhase field in the K8s source code indicates the different stages of pod:

Pending: K8s has accepted the request to create pod, but the container has not been started successfully. This phase includes pre-scheduling, downloading images, and so on.

Running: pod has been bound to the node node, and all containers have been started successfully, or at least one container is running or is being restarted.

Succeeded: all containers in pod have exited normally, and K8s will never restart them automatically.

Failed: all containers in pod have been terminated, and at least one container has terminated in failure (exiting a non-zero exit code or being stopped by the system).

Unknown: due to some unknown reason, the status of the pod cannot be obtained, usually because of the host communication error of the pod.

The above four stages are only a rough state stage. For each stage, there is more detailed pod conditions information, and the elements of the podcondition array contain type and status fields, which are divided into the following four types:

"Ready": Pod can provide services

"PodScheduled": pod is in scheduling

"Unschedulable": the scheduler cannot schedule Pod at this time, for example, due to lack of resources or other restrictions

"Initialized": container initialization for all pod has been completed.

"ContainersReady": all the containers in pod are ready.

The status field is indicated by "true" and "false" indicates that it is not. We find that when the stage is Running and the Ready status in condition is True, it means that the container in pod can provide services. Therefore, we can now judge whether the new pod upgrade is successful or not.

03

How can I tell if the release failed?

Now that we can judge the success, the next step is to judge the state of failure. In fact, in theory, the success or failure of the load is not a deterministic thing. K8s itself has a retry mechanism, and there are also probabilistic scheduling failures. So at first we thought about using a timeout mechanism to determine the release failure. However, the later analysis takes into account that the release, as an important component, needs to know what the problem is the first time, so that operations such as version fallback can be carried out quickly.

For the state of the container, the containerstatus in pod contains a more detailed state of the current container. Its state field indicates the current state of the container. State is divided into three states: waiting, running and terminated, corresponding to three different types of structures. The structure in the wait contains the reason and information for being in a non-running state. Other structures will not be repeated here. Combined with podphase, podcondition, and containerstatus, we summarize the following upgrade exception states (if incomplete, welcome to add):

PodPhase = = Failed

PodPhase = = Succeeded

The reason why an item in ContainerStatuses is state.waiting and waiting is not ContainerCreating or PodInitializing (ContainerCreating indicates container creation, PodInitializing indicates pod initialization)

There is an entry in ContainerStatuses that is state.running and containerStatuses.ready = = false

There is an item in Conditions whose type is Ready and status is False and Containerstatues exists.

There is an item in Conditions whose type is PodScheduled and status is False: scheduling failed

Because the status of pod is dynamic, in order to ensure the accuracy of the release results, we choose pod to be set to failure state when more than 3 times, that is, release failure.

After reading the above principle analysis, do you have any gains, or do you have any ideas to communicate with us? Would you please leave your valuable comments and suggestions below? Maybe in the next version, your ideas will appear in our products.

For related services, please visit https://www.huaweicloud.com/product/swr.html?cce_helpcenter_2019

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.