Shulou(Shulou.com)05/31 Report--

This article mainly explains "how to deal with the problem of Kubernetes application deployment". The content of the article is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn how to deal with the problem of Kubernetes application deployment.

1. The overall idea of dealing with the problem of application deployment.

There may be various problems in deploying containerized applications to Kubernetes clusters. According to the architecture design principle of Kubernetes, the main problems of containerized applications in providing services lie in three points:

1) the problem of the application itself: this problem is the problem of the application itself and will not be described in detail in this article.

2) the problem of Pod as the host of containerized application logic: the problem in this part is mainly related to whether containerized applications are deployed and running normally in the container cloud, such as CPU, memory, storage resources and so on.

3) the problem of proxy containerized application service: third-party services or users will access containerized applications through proxy services. If there is a problem with proxy services, container cloud applications will not be able to provide service capabilities. This will involve the existence of services and whether DNS parsing is correct.

In this article, take the deployed high-availability MySQL as an example to show how to locate and deal with problems. In addition, in order to access the MySQL database outside the Kubernetes cluster, the NodePort type service of MySQL master is exposed, and the service name is mysql-0-svc.

2. Debug Pods

Before debugging Pod, check the running status of Pod with the kubectl get pods command.

$kubectl get pods-namespace=kube-public

For a specific Pod, you can view the detailed information through the kubectl describe pods command.

$kubectl describe pods/mysql-0-namespace=kube-public

In the lifecycle of Pod, there are several states:

Pending: Pod has been accepted by the Kubernetes system, but one or more container images have not been created. This includes the time that Pod is being scheduled and downloading images from the network.

Running: Pod has been bound to a Node and all containers have been created. At least one container is already running, or in the process of starting or restarting.

Succeeded: all containers in Pod have been successfully terminated and will not be restarted.

Failed: all containers have been terminated in Pod, and at least one container has been terminated abnormally. That is, the container exits in a non-zero state or is forcibly terminated by the system.

Unknown: for some reason, the Pod cannot be obtained, typically failing to communicate with the host of the Pod.

Waiting: for some reason, Pod has been dispatched to the Node node, but it does not work properly.

Crashing: Pod is in a crash state for some reason.

According to the state of Pod, the corresponding processing methods are different.

2.1 Pod is on standby (Pending)

If the Pod is stuck in the standby (Pending) state, it means that it cannot be scheduled to the Node node. This is usually caused by a lack of resources of some type, so that Pod cannot be scheduled. By looking at kubectl describe... The output of the command should have a reason why Pod cannot be scheduled. These reasons include:

Not enough resources: the CPU or memory in the cluster may have been exhausted, in which case you need to delete the Pod, adjust resource requests, or add new Node nodes to the cluster.

HostPort in use: Pod is bound to a limited number of hostPort. In most cases, it is not necessary to use hostPort, you can try to use services to expose Pod. If you do need hostPort, you can only schedule as many Pod as the nodes in the Kubernetes cluster.

2.2 Pod is waiting (Waiting)

If Pod is in the Waiting state, it has been scheduled to a working Node, but it cannot run on that Node. Similarly, through kubectl describe... Should be able to get useful information. The most common reason for being in a Waiting state is that the mirror cannot be pulled. There are three things to check:

Make sure the mirror name is correct.

Confirm whether this image exists in the image repository?

On the machine, run the docker pull command to see if you can pull the image.

2.3 Pod collapse (Crashing) or other unhealthy

First, view the log of the current container by executing kubectl logs ${POD_NAME} ${CONTAINER_NAME}:

$kubectl logs mysql-0 mysql--namespace=kube-public

If the container has crashed before, you can access the crash log of the previous container using the following command:

$kubectl logs-previous mysql-0 mysql--namespace=kube-public

Alternatively, you can use kubectl exec to run commands within the container:

$kubectl exec ${POD_NAME}-c ${CONTAINER_NAME}-- ${CMD} ${ARG1} ${ARG2}... ${ARGN}

Note that this-c ${CONTAINER_NAME} is optional and can be omitted for Pod that contains only a single container.

If none of these methods work, you can find the host running the pod and connect to the host through SSH.

2.4 Pod is running, but not as required

If Pod does not work as expected, there may be an error in the Pod description and the error was ignored when creating the Pod. It is usually possible that part of the Pod description is incorrectly nested or the key name is incorrect, so the key is ignored. For example, if command,commnd is misspelled, a Pod will be created, but the command line will not be used as desired.

First, the first thing to do is to delete the Pod and try to create it again with the-validate option. For example, run kubectl create-validate-f mypod.yaml. If you misspell command,commnd, you will get the following error:

I0805 10 https://github.com/kubernetes/kubernetes/issues/6842pods/mypod 43 schema.go:129 25.129850 46757 schema.go:126] unknown field: commndI0805 10 14 43 unknown field 25.129973 46757 schema.go:129] this may be a false alarm, see

Next, check to see if the Pod on apiserver matches the Pod you want to create. For example, run kubectl get pods/mypod-o yaml > mypod-on-apiserver.yaml, and then compare the original Pod description mypod.yaml with the description file mypod-on-apiserver.yaml returned from apiserver. The "apiserver" version usually has something that is not on the original version, which does not affect it. However, if there are lines on the original version that are not on the apiserver version, it may mean that there is a problem with the original version of the Pod description specification.

3. Debug proxy service

According to the architecture design of Kubernetes, users or other applications access containerized applications through proxy services. Therefore, you need to debug to confirm that the proxy service is normal, and the work that needs to be done includes:

1) check whether the proxy service itself exists

2) check whether the proxy service can be parsed normally through DNS

3) check whether the proxy service itself is correct.

3.1 check whether the service exists

When debugging a service, the first step is to check whether the service exists. As explained earlier in this article, MySQL master is exposed through the NodePort type in Kubernetes. By executing the kubectl get svc command, you can get whether the corresponding service exists:

$kubectl get svc/mysql-0-svc-namespace=kube-public

As you can see from the results returned, this service exists in the Kubernetes cluster.

3.2 whether the proxy service can be parsed normally through DNS parsing

For containerized applications in the same namespace, you can access MySQL master directly through the name of the proxy service (mysql-0-svc).

$kubectl exec-it redis-ha-redis-ha-sentinel-5947b9569-r2b56-namespace=kube-public-nslookup mysql-0-svc

For containerized applications of different namespaces in the Kubernetes cluster, you need to access the MySQL master by adding the namespace name (mysql-0-svc.kube-public):

$kubectl exec-it gf1-6497d5df45-98g8v-nslookup mysql-0-svc.kube-public

From what is returned, you can see that the proxy service can be parsed correctly through DNS.

3.2.1 whether DNS is working properly

If the service cannot be parsed properly through the above operations, check whether the Kubernetes master is working properly with the command kubectl exec-it ${POD_NAME}-nslookup:

$kubectl exec-it gf1-6497d5df45-98g8v-nslookup kubernetes.default

If this operation also fails, you need to check that the DNS service in the Kubernetes cluster is running properly.

3.3 whether the agency service itself is correct

If the proxy service also exists, there is no problem with DNS parsing. You need to check whether there is a problem with the proxy service itself:

$kubectl get service mysql-0-svc-o yaml-- namespace=kube-public

For example, is the port accessed correctly? Does targetPort point to the correct Pods port? Whether the port protocol here is consistent with the port protocol exposed by Pod, and so on.

Thank you for reading, the above is the content of "how to deal with the problem of Kubernetes application deployment". After the study of this article, I believe you have a deeper understanding of how to deal with the problem of Kubernetes application deployment, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.