Shulou(Shulou.com)05/31 Report--

This article mainly introduces "what are the problems with the cross-version upgrade of Kubernetes1.20.4". In the daily operation, I believe many people have doubts about the problems with the cross-version upgrade of Kubernetes1.20.4. The editor consulted all kinds of materials and sorted out simple and easy-to-use methods of operation. I hope it will be helpful to answer the questions of "what are the problems with the cross-version upgrade of Kubernetes1.20.4?" Next, please follow the editor to study!

Kubernetes 1.20.4 has been released, and one cluster has been successfully upgraded from 1.20.2. However, the previous version of the other cluster is a bit old, and there was a problem during the cross-version upgrade. Later, it was all redone, and there were problems with the new installation, unable to kubeadm init and kubeadm join. Finally, it was done well. Some process records are as follows:

Certificate problem

The following occurs:

(base) supermap@podc01:/etc$ sudo kubeadm join 10.1.1.202 discovery-token-ca-cert-hash sha256:7eff3c734585308e0934c4af34a67edff0a98c5a3d9e99c24f1c5cdd09d3f519 6443-- token 4q3hdy.y7xjfjh0u1vqdx7k-- discovery-token-ca-cert-hash sha256:7eff3c734585308e0934c4af34a67edff0a98c5a3d9e99c24f1c5cdd09d3f519-- control-plane [preflight] Running pre-flight checks [preflight] Reading configuration from the cluster... [preflight] FYI: You can look at this config file with 'kubectl-n kube-system get cm kubeadm-config-o yaml'error execution phase preflight: One or more conditions for hosting a new control plane instance is not satisfied.failure loading Certificate for CA: couldn't load the certificate file / etc/kubernetes/pki/ca.crt: open / etc/kubernetes/pki/ca.crt: no such file or directoryPlease ensure that:* The cluster has a stable controlPlaneEndpoint address.* The certificates that must be shared among control plane instances are provided.To see the stack trace of this error execute with-- vault 5 or higher

The-- upload-certs parameter was omitted when it turned out to be kubeadm init. After it is added, the error message above will not appear.

Some people use this method to solve the problem, but I haven't tried it, so I don't feel like it:

194 scp-rp / etc/kubernetes/pki/ca.* master02:/etc/kubernetes/pki195 scp-rp / etc/kubernetes/pki/sa.* master02:/etc/kubernetes/pki196 scp-rp / etc/kubernetes/pki/front-proxy-ca.* master02:/etc/kubernetes/pki197 scp-rp / etc/kubernetes/pki/etcd/ca.* master02:/etc/kubernetes/pki/etcd198 scp-rp / etc/kubernetes/admin.conf master02:/etc/kubernetesmaster node settings

Set master node to be able to install other loads. As follows:

Kubectl taint nodes-all node-role.kubernetes.io/master-

CoreDNS problem

There is a problem with CoreDNS, and pod failed to start, as follows:

Supermap@podc02:~$ kubectl get pod-n kube-systemNAME READY STATUS RESTARTS AGEcoredns-74ff55c5b-dtwdz 0 32metcd-podc02 1 ContainerCreating 0 32mcoredns-74ff55c5b-jns5b 0 32mkube-apiserver-podc02 1 ContainerCreating 0 32metcd-podc02 1 Running 0 32mkube-apiserver-podc02 1/1 Running 0 32mkube-controller-manager-podc02 1/1 Running 0 32mkube-proxy-45jxl 1/1 Running 0 32mkube-scheduler-podc02 1/1 Running 0 32m

⚠️ was later found to be a network driver problem, just reinstall flannel.

Flannel installation

Flannel project has been moved to flannel-io, the original address and raw.githubxxxx can not be accessed, need to go to the new address to download.

Wget https://github.com/flannel-io/flannel/releases/download/v0.13.0/flannel-v0.13.0-linux-amd64.tar.gz

The above one is OK, maybe another network won't work again.

Unable to access github, the following error always occurs.

Fatal: unable to access' https://github.com/openthings/kubernetes-tools.git/' failed failed: Error in the pull function.

And then it got better for no reason.

Some people say that these software should be installed, but it is useless to install them.

Supermap@pods01:~/openthings$ sudo apt-get-y install build-essential nghttp2 libnghttp2-dev libssl-dev

See below for more methods of ⚠️.

Systemd compatibility

With docker 19.03, I haven't upgraded for a long time. But both Ubuntu and systemd are upgrading.

There are always kubeadm init failures, so commenting out the systemd of / etc/docker/daemon.json is a success.

Doesn't kubernetes recommend systemd for cgroupfs? I don't know what's going on. Upgrade docker next time and try again.

Sudo kubeadm join 10.1.1.201 control-plane 6443-token k4l26p.d99xrvu2higwz9ow-discovery-token-ca-cert-hash sha256:eda3e649672134c93d11bdb741672b3add5073eb3f4da021274dc51f9278d5f1-control-plane-certificate-key 0a3656c05b225b35724851d08a52ab5ba8c0b70ea64fd4beeb5d727225b63ce4

If the token expires, you can regenerate it with the following command:

Sudo kubeadm init phase upload-certs-upload-certsCNI problem

The following CNI error message appears:

March 18 17:57:27 podc01 kubelet [312941]: E0318 17 Container runtime network not ready 57 podc01 kubelet 27.777448 312941 kubelet.go:2184] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network pl > March 18 17:57:32 podc01 kubelet [312941]: W0318 17 Container runtime network not ready 57 podc01 kubelet 32.184598 312941 cni.go:239] Unable to update cni config: no networks found in / etc/cni/net.d

If you can't handle it in many ways, copy 10-flannel.conflist from another machine:

Sudo scp supermap@10.1.1.201:~/10-flannel.conflist / etc/cni/net.d/

These are the contents of the document:

{"name": "cbr0", "cniVersion": "0.3.1", "plugins": [{"type": "flannel", "delegate": {"hairpinMode": true, "isDefaultGateway": true}}, {"type": "portmap", "capabilities": {"portMappings": true}}]} GnuTLS error

The method found on the Internet has not yet been tried:

Got reason of the problem, it was gnutls package. It's working weird behind a proxy. But openssl is working fine even in weak network. So workaround is that we should compile git with openssl. To do this, run the following commands:

Sudo apt-get updatesudo apt-get install build-essential fakeroot dpkg-devsudo apt-get build-dep gitmkdir ~ / git-opensslcd ~ / git-opensslapt-get source gitdpkg-source-x git_1.7.9.5-1.dsccd git-1.7.9.5

(Remember to replace 1.7.9.5 with the actual version of git in your system.)

Then, edit debian/control file (run the command: gksu gedit debian/control) and replace all instances of libcurl4-gnutls-dev with libcurl4-openssl-dev.

Then build the package (if it's failing on test, you can remove the line TEST=test from the file debian/rules):

Sudo apt-get install libcurl4-openssl-devsudo dpkg-buildpackage-rfakeroot-b

Install new package:

I386: sudo dpkg-I. / git_1.7.9.5-1_i386.deb

X86'64: sudo dpkg-I.. / git_1.7.9.5-1_amd64.deb

Github access failure

Locate the hosts file in the system:

Window:C:\ Windows\ System32\ drivers\ etc\ hosts or r Linux:/etc/hosts

Put in the following two IP addresses:

# GitHub Start 140.82.114.4 github.com199.232.69.194 github.global.ssl.fastly.net# GitHub End

Save and exit.

Execute ipconfig/flushdns from the CMD command line, and then you can go to the Github URL.

Visit this address https://github.com.ipaddress.com/www.github.com to find the ip address information of github.

Cluster configuration

Finally, the cluster was restored:

(base) supermap@podc01:~$ kubectl get node-owideNAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIMEpodc01 Ready control-plane,master 16h v1.20.4 10.1.1.201 Ubuntu 20.10 5.8.0-45-generic docker://20.10.5podc02 Ready control-plane Master 16h v1.20.4 10.1.202 Ubuntu 20.04.2 LTS 5.4.0-67-generic docker://19.3.8podc04 Ready control-plane,master 16h v1.20.4 10.1.204 Ubuntu 20.04.2 LTS 5.4.0-67-generic docker://19.3.8pods01 Ready control-plane Master 16h v1.20.4 10.1.193 Ubuntu 20.04.2 LTS 5.4.0-67-generic docker://19.3.8pods02 Ready control-plane,master 131mv1.20.4 10.1.234 Ubuntu 20.04.2 LTS 5.4.0-67-generic docker://19.3.8pods03 Ready control-plane Master 68mv1.20.4 10.1.205 Ubuntu 20.04.2 LTS 5.4.0-67-generic docker://19.3.8pods04 Ready control-plane,master 50mv1.20.4 10.1.206 Ubuntu 20.04.2 LTS 5.4.0-67-generic docker://19.3.8pods05 Ready control-plane Master 36mv1.20.4 10.1.34 Ubuntu 20.04.2 LTS 5.4.0-66-generic docker://19.3.8pods06 Ready control-plane,master 6m22s v1.20.4 10.1.167 Ubuntu 20.04.2 LTS 5.4.0-66-generic docker://19.3.8

Other exceptions occurred on three nodes:

One of the nodes can be updated normally after rebooting several times.

The other node, podc03, cannot be restarted many times. It is probably dead.

Another node showed that the file system was read-only, could not be updated, and was later repaired.

Enter the menu at startup and select repair.

Run fsck, and then restart.

At this point, the study on "what are the problems with the cross-version upgrade of Kubernetes1.20.4" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.