Shulou(Shulou.com)06/03 Report--

Description

Calico is a pure three-layer protocol that provides multi-host communication between OpenStack virtual machines and Docker containers. Calico does not use overlay network drivers such as flannel and libnetwork overlay network drivers

It is a pure three-layer method, using virtual routing instead of virtual switching, and each virtual route propagates reachable information (routing) to the rest of the data center through the BGP protocol.

To view the official documentation upgrade operation, you need to do the following considerations.

Etcd for 2.6.x is different from 3.x (for etcd storage only), 2.6 uses etcdv2, while 3.x uses etcdv3. If you want to upgrade from 2.6.x to 3.x, you have to be at least 2.6.5 +.

Therefore, in view of the existing situation, we need to upgrade to 2.6.5 + first, and then upgrade 3.x.

2.6.1 upgrade to 2.6.12

2019-12-25

Existing environment, calico data stored using etcdv2.

[root@k8s-1 kubelet] # which etcdv2alias etcdv2='export ETCDCTL_API=2 / bin/etcdctl-ca-file / etc/etcd/ssl/etcd-root-ca.pem-cert-file / etc/etcd/ssl/etcd.pem-key-file / etc/etcd/ssl/etcd-key.pem-endpoints https://10.111.32.239:2379,https://10.111.32.241:2379, Https://10.111.32.242:2379'[root@k8s-1 kubelet] # etcdv2 ls / calico/ipam/v2/assignment/ipv4/calico/ipam/v2/assignment/ipv4/ block [root @ k8s-1 kubelet] # etcdv2 ls / calico/ipam/v2/assignment/ipv4/block/calico/ipam/v2/assignment/ipv4/block/10.20.134.64-26/calico/ipam/v2/assignment/ipv4/block/10.20.253.64-26/calico/ipam/v2 / assignment/ipv4/block/10.20.28.192-26/calico/ipam/v2/assignment/ipv4/block/10.20.51.128-26/calico/ipam/v2/assignment/ipv4/block/10.20.78.0-26/calico/ipam/v2/assignment/ipv4/block/10.20.112.64-26/calico/ipam/v2/assignment/ipv4/block/10.20.15.128-26/calico/ipam/v2/assignment/ipv4/block/10.20 .235.0-26/calico/ipam/v2/assignment/ipv4/block/10.20.53.64-26/calico/ipam/v2/assignment/ipv4/block/10.20.72.128-26

According to the instructions in the documentation, upgrading to 3.0 requires at least 2.6.5 + and requires some manual action, because 3.x uses etcdv3 and 2.6.x uses etcdv2.

Now that the cluster is using version 2.6.1, upgrade it to 2.6.5 + first.

Choose the latest 2.6.12 out of 2.6.

Download the calico.yaml file

[root@docker-182 v2.6] # wget https://docs.projectcalico.org/v2.6/getting-started/kubernetes/installation/rbac.yaml[root@docker-182 v2.6] # wget https://docs.projectcalico.org/v2.6/getting-started/kubernetes/installation/hosted/calico.yaml# changes the configuration in calico.yaml [root@docker-182 v2.6] # sh-x modify_calico_yaml.sh prefetches the image [root@docker-182 V2.6] # grep image calico.yaml image: quay.io/calico/node:v2.6.12 image: quay.io/calico/cni:v1.11.8 image: quay.io/calico/kube-controllers:v1.0.5 image: quay.io/calico/kube-controllers:v1.0.5

Some of the upgrade steps mentioned in the document, such as upgrading calico-kube-controllers first, and then upgrading calico-node 's daemonset, apply the new resource file directly here.

Does not contain rbac resources for calico.

[root@docker-182 v2.6] # k239 apply-f calico.yamlconfigmap "calico-config" unchangedsecret "calico-etcd-secrets" unchangeddaemonset "calico-node" configureddeployment "calico-kube-controllers" configureddeployment "calico-policy-controller" configuredserviceaccount "calico-kube-controllers" unchangedserviceaccount "calico-node" unchanged submit update

After the submission, daemonset's calico-node has not been updated. Now delete pod and make it updated.

[root@k8s-1 v2.6] # kubectl-n kube-system get pod-o wide | grep calicocalico-kube-controllers-6768b96c5f-rdbjp 1 Running 0 4m 10.111.32.243 k8s-4.geotmt.comcalico-node-45lnh 0 ContainerCreating 0 4h 10.111.32.241 k8s-2.geotmt.comcalico-node -49mq7 1 Running 1 5h 10.111.32.243 k8s-4.geotmt.comcalico-node-m86hr 1 Running 0 5h 10.111.32.244 k8s-5.geotmt.comcalico-node-mm5fz 0/1 ContainerCreating 0 4h 10.111.32.239 k8s-1.geotmt.comcalico-node-shrfw 1/1 Running 0 4h 10.111.32.242 k8s-3.geotmt.comcalico-node-xx8hk 1/1 Running 0 5h 10.111.32.245 Test after k8s-6.geotmt.com update

One of the examples is the new calico-node, which has two containers.

[root@k8s-1 v2.6] # kubectl-n kube-system get pod-o wide | grep calico | grep k8s-6calico-node-fj4t8 2 Running 0 25s 10.111.32.245 k8s-6.geotmt.com

Test the pod of other ping nodes is normal

Bash-4.4# ip a1: lo: mtu 65536 qdisc noqueue state UNKNOWN qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever2: tunl0@NONE: mtu 1480 qdisc noop state DOWN qlen 1000 link/ipip 0.0.0.0 brd 0.0.0.04: eth0@if30: mtu 1500 qdisc noqueue state UP link / ether 6e:20:a3:45:42:49 brd ff:ff:ff:ff:ff:ff inet 10.20.235.12 scope global eth0 valid_lft forever preferred_lft foreverbash-4.4# ping 10.20.15.135PING 32 scope global eth0 valid_lft forever preferred_lft foreverbash-4.4# ping 10.20.15.135PING 10.20.15.135 (10.20.15.135): 56 data bytes64 bytes from 10.20.15.135: seq=0 ttl=62 time=1.133 ms64 bytes from 10.20.15.135: seq=1 ttl=62 time=0.631 ms

In this version, you still need to add toleration manually to deploy pod on the master node.

Upgrade to 2.6.12 is complete.

2.6.12 considerations before upgrading to 3.0 You must first upgrade to Calico v2.6.5 (or a later v2.6.x release) before you can upgrade to Calico v3.0.12. (Important: Calico v2.6.5 was a special transitional release that included changes to enable upgrade to v3.0.1 release; do not skip this step!) If you are using the etcd datastore, you should upgrade etcd to the latest stable v3 release.

Both of the above are satisfied.

[root@k8s-1 net.d] # etcdctl versionetcdctl version: 3.3.11API version: 3.3etcd datastore upgrade stepsInstall and configure calico-upgradeTest the data migration and check for errorsMigrate Calico dataUpgrade Calico installation and configuration calico-upgrade [root@docker-182 ansible] # wget https://github.com/projectcalico/calico-upgrade/releases/download/v1.0.5/calico-upgrade[root@docker-182 k8s_239] # ansible-playbook install_calico-upgrade.yml

Perform tests using dry-run

[root@k8s-1 calico-upgrade] # calico-upgrade dry-run-- output-dir=tmp-- apiconfigv1 / etc/calico/apiconfigv1.cfg-- apiconfigv3 / etc/calico/apiconfigv3.cfg perform upgrade [root@k8s-1 calico-upgrade] # calico-upgrade start-- ignore-v3-data-- apiconfigv1 / etc/calico/apiconfigv1.cfg-- apiconfigv3 / etc/calico/apiconfigv3.cfgPreparing reports directory * creating report directory if it does not exist * validating permissions and removing old reportsChecking Calico version is suitable for migration * determined Calico version Of: v2.6.12 * the v1 API data can be migrated to the v3 APIValidating conversion of v1 data to v3 * handling FelixConfiguration (global) resource * handling ClusterInformation (global) resource * handling FelixConfiguration (per-node) resources * handling BGPConfiguration (global) resource * handling Node resources * handling BGPPeer (global) resources * handling BGPPeer (node) resources * handling HostEndpoint resources * handling IPPool resources * handling GlobalNetworkPolicy resources * handling Profile resources * handling WorkloadEndpoint resources * data conversion successfulData conversion validated successfullyValidating the v3 datastore * the v3 datastore is not empty- -Successfully validated v1 to v3 conversion.You are about to start the migration of Calico v1 dataformat to Calico v3 dataformat. During this time and until the upgrade is completed Calico networkingwill be paused-which means no new Calico networked endpoints can be created.No Calico configuration should be modified using calicoctl during this time.Type "yes" to proceed (any other input cancels): yesPausing Calico networking * successfully paused Calico networking in the v1 configurationCalico networking is now paused-waiting for 15sQuerying current v1 snapshot and converting to v3 * handling FelixConfiguration (global) resource * handling ClusterInformation (global) resource * handling FelixConfiguration (per-node) resources * handling BGPConfiguration (global) resource * handling Node resources * handling BGPPeer (global) resources * handling BGPPeer (node) resources * handling HostEndpoint resources * handling IPPool resources * handling GlobalNetworkPolicy resources * handling Profile resources * handling WorkloadEndpoint resources * data converted successfullyStoring v3 data * Storing resources in v3 format * success: resources stored in v3 datastoreMigrating IPAM data * listing and converting IPAM allocation blocks * listing and converting IPAM affinity blocks * listing IPAM handles * storing IPAM data in v3 format * IPAM data migrated successfullyData migration from v1 to v3 successful * check the output for details of the migrated resources * continue by upgrading your calico/node versions to Calico v3.x- -Successfully migrated Calico v1 data to v3 format.Follow the detailed upgrade instructions available in the release documentationto complete the upgrade. This includes: * upgrading your calico/node instances and orchestrator plugins (e.g. CNI) to the required v3.x release * running 'calico-upgrade complete' to complete the upgrade and resume Calico networkingSee report (s) below for details of the migrated data.Reports:- name conversion: / root/calico-upgrade/calico-upgrade-report/convertednames download v3.0 resource file [root@docker-182 v3.0] # wget https://docs.projectcalico.org/v3.0/getting-started/kubernetes/ Installation/ rbac.yaml [root @ docker-182 v3.0] # wget https://docs.projectcalico.org/v3.0/getting-started/kubernetes/installation/hosted/calico.yaml

For changes in 3. 0, please refer to 3.0release note

Download the required images in advance

[root@docker-182 v3.0] # grep image calico.yaml image: quay.io/calico/node:v3.0.12 image: quay.io/calico/cni:v3.0.12 image: quay.io/calico/kube-controllers:v3.0.12 perform upgrade [root@docker-182 v3.0] # k239 apply-f calico.yamlconfigmap "calico-config" configuredsecret "calico-etcd-secrets" unchangeddaemonset "calico-node" configureddeployment "calico- Kube-controllers "configuredserviceaccount" calico-kube-controllers "unchangedserviceaccount" calico-node "unchanged

The pod here can be scrolled and restarted after all the pod upgrades are completed.

Execute the calico-upgrade command to confirm that the upgrade is completed [root@k8s-1 calico-upgrade] # calico-upgrade complete-- apiconfigv1 / etc/calico/apiconfigv1.cfg-- apiconfigv3 / etc/calico/apiconfigv3.cfgYou are about to complete the upgrade process to Calico v3. At this point, thev1 format data should have been successfully converted to v3 format And allcalico/node instances and orchestrator plugins (e.g. CNI) should be runningCalico v3.x.Type "yes" to proceed (any other input cancels): yesCompleting upgradeEnabling Calico networking for v3 * successfully resumed Calico networking in the v3 configuration (updated ClusterInformation) Upgrade completed successfully -Successfully completed the upgrade process.

If the above order is not carried out, the following error will be reported

E1225 19 1dd28cf0 56 demo-deployment-6f4c6779b-b8zqq_default 04.837028 3281 kuberuntime_sandbox.go:54] CreatePodSandbox for pod "demo-deployment-6f4c6779b-b8zqq_default (1dd28cf0-270d-11ea-bd6c-c6a864ab864a)" failed: rpc error: code = Unknown desc = NetworkPlugin cni failed to set up pod "demo-deployment-6f4c6779b-b8zqq_default" network: Calico is currently not ready to process requestsE1225 1915 5V 04.837049 3281 kuberuntime_manager.go:647] createPodSandbox for pod "demo-deployment-6f4c6779b-b8zqq_default (1dd28cf0-270dmurbd6c- C6a864ab864a) "failed: rpc error: code = Unknown desc = NetworkPlugin cni failed to set up pod" demo-deployment-6f4c6779b-b8zqq_default "network: Calico is currently not ready to process requestsE1225 1915 56 code 04.837167 3281 pod_workers.go:186] Error syncing pod 1dd28cf0-270d-11ea-bd6c-c6a864ab864a (" demo-deployment-6f4c6779b-b8zqq_default (1dd28cf0-270d-11ea-bd6c-c6a864ab864a) ") Skipping: failed to "CreatePodSandbox" for demo-deployment-6f4c6779b-b8zqq_default (1dd28cf0-270d-11ea-bd6c-c6a864ab864a) "with CreatePodSandboxError:" CreatePodSandbox for pod\ "demo-deployment-6f4c6779b-b8zqq_default (1dd28cf0-270d-11ea-bd6c-c6a864ab864a)\" failed: rpc error: code = Unknown desc = NetworkPlugin cni failed to set up pod\ "demo-deployment-6f4c6779b-b8zqq_default\" network: Calico is currently not ready to process requests "

Upgrade to 3.0.12 successful.

Upgrade from 3.0.12 to 3.11

According to the Upgrading Calico on Kubernetes description of 3.11. When upgrading, you only need to submit a new resource file (this environment does not involve Application Layer Policy).

This version of calico can fully support the datastore of K8s api. When updating, you should pay attention to whether the download of the file is consistent with your environment.

Download the version of etcd datastore in this environment.

Download the resource file [root@docker-182 v3.11] # wget https://docs.projectcalico.org/v3.11/manifests/calico-etcd.yaml# to modify the configuration of etcd [root@docker-182 v3.11] # bash-x modify_calico_yaml.sh pre-download image [root@docker-182 v3.11] # grep image calico-etcd.yaml image: calico/cni:v3.11.1 image: calico/pod2daemon -flexvol:v3.11.1 image: calico/node:v3.11.1 image: calico/kube-controllers:v3.11.1 submit the new version [root@docker-182 v3.11] # k239 apply-f calico-etcd.yamlsecret "calico-etcd-secrets" unchangedconfigmap "calico-config" configuredclusterrole "calico-kube-controllers" configuredclusterrolebinding "calico-kube-controllers" configuredclusterrole "calico-node" configuredclusterrolebinding "calico-node" configureddaemonset "calico-node" configuredserviceaccount "calico-node" unchangeddeployment " Calico-kube-controllers "configuredserviceaccount" calico-kube-controllers "unchanged verifies the new version

Check out the new version of pod. There is only one container in each pod. This version uses install-cni and flexvol-driver (not in the old version) as initContainers, so there is only one container resident.

[root@docker-182] # k239-n kube-system get pod-o wide | grep calicocalico-kube-controllers-85dc4fd46b-4wnmt 1 Running 0 1m 10.111.32.243 k8s-4.geotmt.comcalico-node-4bgkc 1 Running 059s 10.111.32.241 k8s-2.geotmt.comcalico-node-5jg2t 1/1 Running 0 31s 10.111.32.244 k8s-5.geotmt.comcalico-node-9fn6r 1/1 Running 0 43s 10.111.32.245 k8s-6.geotmt.comcalico-node-9n7dn 1/1 Running 0 1m 10.111.32.243 k8s-4.geotmt.comcalico-node-fxr46 1/1 Running 0 1m 10.111.32.239 k8s-1.geotmt.comcalico-node-pgh6c 1/1 Running 0 1m 10.111.32.242 k8s-3.geotmt.com

Test pod cross-host communication

[root@k8s-1] # kubectl exec-it demo-deployment-6f4c6779b-b8zqq / bin/bashbash-4.4# ping 10.20.235.12PING 10.20.235.12 (10.20.235.12): 56 data bytes64 bytes from 10.20.235.12: seq=0 ttl=62 time=1.232 Ms ^ C-10.20.235.12 ping statistics-1 packets transmitted, 1 packets received 0 packet lossround-trip min/avg/max = 1.232 packet lossround-trip min/avg/max 1.232 msbash-4.4# ping 10.20.253.80PING 10.20.253.80 (10.20.253.80): 56 data bytes64 bytes from 10.20.253.80: seq=0 ttl=62 time=1.730 ms64 bytes from 10.20.253.80: seq=1 ttl=62 time=1.385 Ms ^ C-10.20.253.80 ping statistics-- 2 packets transmitted, 2 packets received 0 packet lossround-trip min/avg/max = 1.385 qdisc noqueue state UNKNOWN qlen 1.557 msbash-4.4# ip 1.730 inet A1: lo: mtu 65536 qdisc noqueue state UNKNOWN qlen 1000 link/loopback 0000 qdisc noop state DOWN qlen 0000 qdisc noop state DOWN qlen 0000 scope host lo valid_lft forever preferred_lft forever2: tunl0@NONE: mtu 1480 qdisc noop state DOWN qlen 1000 link/ipip 0.0.0.0 Brd 0.0.0.04: eth0@if51: mtu 1500 qdisc noqueue state UP link/ether fa:d1:55:42:ab:6c brd ff:ff:ff:ff:ff:ff inet 10.20.15.163/32 scope global eth0 valid_lft forever preferred_lft forever

Test pod rebuild assigned address, successful

[root@k8s-1 ~] # kubectl delete pod nginx-deployment-7b66d98974- 2rh87pod "nginx-deployment-7b66d98974- 2rh87" deleted [root @ k8s-1 ~] # kubectl get pod nginx-deployment-7b66d98974-nd8h7-o wide NAME READY STATUS RESTARTS AGE IP NODEnginx-deployment-7b66d98974-nd8h7 1 Running 0 1m 10.20.253.86 k8s-4.geotmt.com

Calico 3.0.12 to 3.11.1 was upgraded successfully.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.