Shulou(Shulou.com)05/31 Report--

This article mainly introduces "what is the method of manual rotation of certificates in Rancher 2.3". In daily operation, I believe many people have doubts about the method of manual rotation of certificates in Rancher 2.3. The editor consulted all kinds of materials and sorted out simple and easy-to-use methods of operation. I hope it will be helpful for you to answer the doubts of "what is the method of manual rotation of certificates in Rancher 2.3?" Next, please follow the editor to study!

Foreword

It has been a year since Rancher 2.3 was officially released, and the first batch of users who use Rancher 2.3 may encounter an expired Rancher Server certificate without automatic rotation. This causes Rancher Server to fail to start and the log to report an error:

Please note:

The failure to start Rancher Server will not affect the downstream cluster, and the downstream cluster can still be operated through kubeconfig.

Please note:

The failure to start Rancher Server will not affect the downstream cluster, and the downstream cluster can still be operated through kubeconfig.

This will only happen on docker run startup or Rancher that uses less than k3s v1.19 as a local cluster. This will only happen on docker run startup or Rancher that uses less than k3s v1.19 as a local cluster.

Recurring problem

In order to better understand the problem, the problem will be reproduced in the form of manually modifying the system time.

Current time: 10:37:59 CST on Friday, October 30, 2020

1. Start Rancher v2.3.1 and add a downstream cluster. For more information, please see the official website:

Https://docs.rancher.cn/docs/rancher2/installation/other-installation-methods/single-node-docker/_index/

Https://docs.rancher.cn/docs/rancher2/cluster-provisioning/_index

2. After launching Rancher, the expiration time seen on the browser is 10:29:35 China Standard time on Saturday, October 30th, 2021.

3. Check the expiration time of K3s certificate in Rancher Server container is Oct 30 02:28:49 2021 GMT

Root@rancher1:~# docker exec-it rancher_server_id bashroot@25c228f6a4c8:/var/lib/rancher# for i in `ls/ var/lib/rancher/k3s/server/tls/* .crt`; do echo $I; openssl x509-enddate-noout-in $I Done/var/lib/rancher/k3s/server/tls/client-admin.crtnotAfter=Oct 30 02:28:49 2021 GMT/var/lib/rancher/k3s/server/tls/client-auth-proxy.crtnotAfter=Oct 30 02:28:49 2021 GMT/var/lib/rancher/k3s/server/tls/client-ca.crtnotAfter=Oct 28 02:28:49 2030 GMT/var/lib/rancher/k3s/server/tls/client-controller.crtnotAfter=Oct 30 02:28:49 2021 GMT/var/lib/rancher/k3s/server/tls/ Client-kube-apiserver.crtnotAfter=Oct 30 02:28:49 2021 GMT/var/lib/rancher/k3s/server/tls/client-kube-proxy.crtnotAfter=Oct 30 02:28:49 2021 GMT/var/lib/rancher/k3s/server/tls/client-scheduler.crtnotAfter=Oct 30 02:28:49 2021 GMT/var/lib/rancher/k3s/server/tls/request-header-ca.crtnotAfter=Oct 28 02:28:49 2030 GMT/var/lib/rancher/k3s/server/tls/server-ca.crtnotAfter=Oct 28 02:28: 49 2030 GMT/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crtnotAfter=Oct 30 02:28:49 2021 GMT

4. Adjust the server time to 5 days after the certificate expires, for example: 20211105

Root@rancher1:~# timedatectl set-ntp noroot@rancher1:~# date-s 20211105Fri Nov 5 00:00:00 CST 2021root@rancher1:~# dateFri Nov 5 00:00:00 CST 2021

At this point, Rancher UI is no longer accessible:

And the Rancher container is constantly restarted due to the expiration of the built-in K3s certificate.

Manually rotate the certificate

The above phenomenon is due to the expiration of the built-in K3s certificate in Rancher Server, which causes K3s to fail to start, thus causing the Rancher Server container to fail to start.

In order to continue working on the Rancher Server container, you need to adjust the system time to before the K3s certificate expires.

Root@rancher1:~# date-s 20211025Mon Oct 25 00:00:00 CST 2021

If you start Rancher without the-- restart=unless-stopped parameter, you need to start Rancher Server manually.

Next, we can enter the container to manually delete the K3s certificate, and then restart Rancher. After a successful restart, the K3s certificate will be regenerated.

Root@rancher1:~# docker exec-it rancher_server_id bashroot@25c228f6a4c8:/var/lib/rancher# rm-rf / var/lib/rancher/k3s/server/tls/*.crtroot@25c228f6a4c8:/var/lib/rancher# exitexitroot@rancher1:~# docker restart rancher_server_id

If the following log appears in Rancher Server, you need to restart Rancher Server again:

16:01:00 on 2021-10-24 [INFO] Waiting for server to become available: Get https://localhost:6443/version?timeout=30s: x509: certificate signed by unknown authority Verification

1. Adjust the server time again to 5 days after the certificate expires, for example: 20211105

Root@rancher1:~# date-s 20211105Fri Nov 5 00:00:00 CST 2021

After the certificate is updated, we need to confirm that the K3s certificate has been updated successfully, and we also need to check whether the downstream cluster will have an impact.

2. Confirm that the K3s certificate has been updated

Root@rancher1:~# docker exec-it rancher_server_id bashroot@25c228f6a4c8:/var/lib/rancher# for i in `ls/ var/lib/rancher/k3s/server/tls/* .crt`; do echo $I; openssl x509-enddate-noout-in $I Done/var/lib/rancher/k3s/server/tls/client-admin.crtnotAfter=Oct 24 16:00:54 2022 GMT/var/lib/rancher/k3s/server/tls/client-auth-proxy.crtnotAfter=Oct 24 16:00:54 2022 GMT/var/lib/rancher/k3s/server/tls/client-ca.crtnotAfter=Oct 22 16:00:54 2031 GMT/var/lib/rancher/k3s/server/tls/client-controller.crtnotAfter=Oct 24 16:00:54 2022 GMT/var/lib/rancher/k3s/server/tls/ Client-kube-apiserver.crtnotAfter=Oct 24 16:00:54 2022 GMT/var/lib/rancher/k3s/server/tls/client-kube-proxy.crtnotAfter=Oct 24 16:00:54 2022 GMT/var/lib/rancher/k3s/server/tls/client-scheduler.crtnotAfter=Oct 24 16:00:54 2022 GMT/var/lib/rancher/k3s/server/tls/request-header-ca.crtnotAfter=Oct 22 16:00:54 2031 GMT/var/lib/rancher/k3s/server/tls/server-ca.crtnotAfter=Oct 22 16:00: 54 2031 GMT/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crtnotAfter=Oct 24 16:00:54 2022 GMT

The expiration time of K3s certificate has been updated from Oct 30 02:28:49 2021 GMT to Oct 24 16:00:54 2022 GMT

3. Confirm that the browser certificate has been updated

The certificate expiration on the browser has been updated from 10:29:35 China Standard time on Saturday, October 30, 2021 to 00:01:34 China Standard time on Tuesday, October 25, 2022.

4. Confirm that the downstream cluster is not affected.

Cluster status is Active

Check the health status of the cluster Pod

At this point, the study on "what is the method of manual rotation of certificates in Rancher 2.3" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.