Shulou(Shulou.com)06/03 Report--

Kubernetes cluster certificate resigning

According to the https://github.com/strongit/kubeadm-ha/ installation steps, the cluster after kubeadm init installation has the problem of certificate expiration. The repairs are as follows:

The idea is as follows

1. Keep ca.crt ca.key front-proxy-ca.crt front-proxy-ca.key, and the root certificate is valid for ten years.

2. Openssl re-endorsement

3. Kubeadm alpha phase generates config

[root@k8s-master01 pki] # cat csr.conf [req] default_bits = 2048prompt = nodefault_md = sha256req_extensions = dn [dn] C = CNST = BeiJingL = BeiJingO = k8sOU = SystemCN = kubernetes [req_ext] subjectAltName = @ alt_names [alt_names] DNS.1 = kubernetesDNS.2 = kubernetes.defaultDNS.3 = kubernetes.default.svcDNS.4 = kubernetes.default.svc.clusterDNS.5 = kubernetes.default.svc.cluster.localDNS.6 = k8s-master01DNS.7 = k8s -master02DNS.8 = k8s-master03IP.1 = IPIP.2 = IP [v3_ext] authorityKeyIdentifier=keyid Issuer:alwaysbasicConstraints=CA:FALSEkeyUsage=digitalSignature,nonRepudiation,keyEncipherment,dataEnciphermentextendedKeyUsage=serverAuth ClientAuthsubjectAltName=@alt_names openssl genrsa-out apiserver.key 2048 openssl req-new-key apiserver.key-out apiserver.csr-config csr.conf openssl x509-req-in apiserver.csr-CA ca.crt-CAkey ca.key-CAcreateserial-out apiserver.crt-days 10000-extensions v3_ext-extfile csr.conf openssl x509-noout-text-in. / apiserver.crt | grep "Not" openssl genrsa-out apiserver-kubelet-client.key 2048 openssl req-new-key apiserver-kubelet-client.key-out apiserver-kubelet- Client.csr-config csr.conf openssl x509-req-in apiserver-kubelet-client.csr-CA ca.crt-CAkey ca.key-CAcreateserial-out apiserver-kubelet-client.crt-days 10000-extensions v3_ext-extfile csr.conf openssl x509-noout-text-in. / apiserver-kubelet-client.crt | grep "Not" openssl genrsa-out front-proxy-client.key 2048 openssl req-new-key front-proxy-client.key-out front-proxy-client.csr-config csr.conf openssl x509-req -in front-proxy-client.csr-CA front-proxy-ca.crt-CAkey front-proxy-ca.key-CAcreateserial-out front-proxy-client.crt-days 10000-extensions v3_ext-extfile csr.conf openssl x509-noout-text-in. / front-proxy-client.crt | grep "Not" kubeadm alpha phase certs all-config kubeadm-config.yamlkubeadm alpha phase kubelet config write-to-disk-config kubeadm-config.yamlkubeadm alpha phase kubelet write-env-file-config kubeadm-config.yamlkubeadm alpha Phase kubeconfig kubelet-config kubeadm-config.yamlkubeadm alpha phase kubeconfig all-config kubeadm-config.yamlkubeadm alpha phase controlplane all-config kubeadm-config.yamlsystemctl restart kubeletkubeadm alpha phase mark-master-config kubeadm-config.yamlcp / etc/kubernetes/admin.conf ~ / .kube/config

After restarting the cluster, execute kubelet logs pods XXXX-n kube-system with the following error: Error from server (Forbidden): Forbidden (user=kubernetes, verb=get, resource=nodes, subresource=proxy) (pods/log kube-scheduler-k8s-master01)

Solution: kubectl create clusterrolebinding system:kubernetes-clusterrole=cluster-admin-user=system:kubernetes

Tls settings for ingress

The steps to set the tls security certificate for the domain name in ingress are as follows:

Create a self-signed secret key and ssl certificate

Save the certificate to a Secret resource object in the kubernetes cluster

Set the Secret resource object to ingress.

Depending on whether there are one or more domain names of the website, the operations in the first two steps are slightly different, and the operations in step 3 are the same. Take the operation of multiple domain names as an example:

Generate ca certificate

[root@kubenode1] # mkdir-p / etc/kubernetes/ingress

[root@kubenode1 ~] # cd / etc/kubernetes/ingress/

[root@kubenode1 ingress] # openssl genrsa-out ca.key 2048

[root@kubenode1 ingress] # openssl req-x509-new-nodes-key ca.key-days 3560-out ca.crt-subj "/ CN=ingress-ca"

Modify the openssl.cnf file

# for multiple domain names, additional x509v3 configuration files are required to generate ssl certificates

# set multiple domain names in the [alt_names] field

[root@kubenode1 ingress] # cp / etc/pki/tls/openssl.cnf.

[root@kubenode1 ingress] # vim openssl.cnf

[req]

# line 126, uncomment

Req_extensions = v3_req # The extensions to add to a certificate request

[v3_req]

# Extensions to add to a certificate request

BasicConstraints = CA:FALSE

KeyUsage = nonRepudiation, digitalSignature, keyEncipherment

# add a new section after line 224

SubjectAltName = @ alt_names

[alt_names]

DNS.1 = nginx01-svc-tls.me

DNS.2 = nginx02-svc-tls.me

Generate ingress ssl certificate

# generate ingress ssl certificates based on modified openssl.cnf and ca certificates

# generate secret key

[root@kubenode1 ingress] # openssl genrsa-out ingress.key 2048

# generate csr file

[root@kubenode1 ingress] # openssl req-new-key ingress.key-out ingress.csr-subj "/ CN=nginx-svc-tls"-config openssl.cnf

# generate a certificate

[root@kubenode1 ingress] # openssl x509-req-in ingress.csr-CA ca.crt-CAkey ca.key-CAcreateserial-out ingress.crt-days 3650-extensions v3_req-extfile openssl.cnf

Generate Secret resource objects

The main function of Secret object is to keep private data, such as password, OAuth Tokens,ssh Keys and so on. Storing private information in Secret objects is more secure and easier to use and distribute than directly in Pod or Docker image.

After the Secret object is created, you can call it in three ways:

When you create a Pod, you automatically use the

Use by mounting Secret to Pod

Used when downloading Docker image, referenced by specifying the spc.ImagePullSecrets of the Pod.

# Edit the secret-ingress.yaml file and copy the contents of ingress.key and ingress.crt to the yaml file

# Note that the values of the subfields of the "data" field of 1:Secret must be BASE64 encoded

# Note 2: when copying the contents of key and crt, remove the newline character and change it to one line.

[root@kubenode1 ingress] # cd / usr/local/src/yaml/ingress/

[root@kubenode1 ingress] # touch secret-ingress.yaml

[root@kubenode1 ingress] # vim secret-ingress.yaml

ApiVersion: v1

Kind: Secret

Metadata:

Name: secret-ingress

# 1.8.x replace Opaque with kubernetes.io/tls

Type: kubernetes.io/tls

Data:

Tls.crt: MIIC/TCCAeWgAwIBAgIJALGRacBg2fWIMA0GCSqGSIb3DQEBCwUAMBUxEzARBgNVBAMMCmluZ3Jlc3MtY2EwHhcNMTgwMjI4MDM0NDI1WhcNMjgwMjI2MDM0NDI1WjAYMRYwFAYDVQQDDA1uZ2lueC1zdmMtdGxzMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAuZvMBYF104JPtMZFFUxCpGGODFG4rGffN1FFC98CGt99QAwVMfABGDMU8zfa21twxON1v3WK8HdJH5KRdLOIRQnhuMHsC174sb/+FuOa0GhStgmNX0f2jGETuImPQ82faXACnUkUYuvYG5odbY+tS+LQBtIormpxWRlNNTVzT3jFD6JECVZzpMyCJutkwxJC083PS1VE9ki+7mgpPWbb9BqT0Tn672x4cHI8LZ5snr1fpR8I0sqADXY+KpFQeh7UJsWZjfr00wDBsg76aF3TNK+pecXnBNYPZ6o7sOGXvagAxU58xjjz75TwMQ7NnqF584fshvQLnzeTGhDbXx4GHQIDAQABo00wSzAJBgNVHRMEAjAAMAsGA1UdDwQEAwIF4DAxBgNVHREEKjAoghJuZ2lueDAxLXN2Yy10bHMubWWCEm5naW54MDItc3ZjLXRscy5tZTANBgkqhkiG9w0BAQsFAAOCAQEAaUQH21wct78wW1tAz1+3j9SGLLk7kd06PcnYH1pBGW3wlMFiusMOUdVfm2aPwkX7VFOfeo6LYtILisQ9+wJraQcGd31H5M/ILSH2bhd739CcySqm3aEYHplQCfepsRRcINp82N3GLjcT6sCeuvoC4l+rUDIKMEPt7Skwj1HjCj/NSpHguzmtmRG4PCvv/3nCrcntGLKxsKBD85llpRlT9/Q+9eMmQz7YRUjUEr/3cSmBfRjcBQhB7yXZyDLRbObAc1BMgxBRX/oexNIHCMNdRFSP6wAtlyajytOhcixBMu3RQ1g0lrFaT9vQWPXs6HV8xC6VvfhdSQN8B1T65klNNg==

Tls.key: MIIEpAIBAAKCAQEAuZvMBYF104JPtMZFFUxCpGGODFG4rGffN1FFC98CGt99QAwVMfABGDMU8zfa21twxON1v3WK8HdJH5KRdLOIRQnhuMHsC174sb/+FuOa0GhStgmNX0f2jGETuImPQ82faXACnUkUYuvYG5odbY+tS+LQBtIormpxWRlNNTVzT3jFD6JECVZzpMyCJutkwxJC083PS1VE9ki+7mgpPWbb9BqT0Tn672x4cHI8LZ5snr1fpR8I0sqADXY+KpFQeh7UJsWZjfr00wDBsg76aF3TNK+pecXnBNYPZ6o7sOGXvagAxU58xjjz75TwMQ7NnqF584fshvQLnzeTGhDbXx4GHQIDAQABAoIBAFMGCI3R6eWRXZvsMEyljw2+gW6rQ2MDF4rD9JGp0GQ64ei7PuPWinbLqqxcqK4ESf4YDLx2lI6ZnQDda+j6wZK4J9qgC7jOY4oG6l5MsxxT/eNlhHJBW1xRtCOQjJ/0o0DjlJfMb60L99/o4Q73/Ll8HDdg3EegX1FOiwWpAgpipA+WyosAtrfR8DjOAVMavlhkCejmgupWU7syuVmVQ0Dz/z9zPESI1b6pHO0Js4Keb8vnUHPLNcq1HCdCMK+wrdUaW2YmuAr9uoF7Wqvp7MCog//cQX93mijJzW8GFPrSt2y4NHN6AnUw6PE3aoMgF1my7O1xLwOjCQz+eW8voyECgYEA8luy6iEDYkfq+tkxA9kl3CgXVk5WgiE/4mVaEjOIT2llgM+8K3TAn8EGATk5s79phn/MRfqi8YQ13Z9dzhp3R/ARynD+/TVRzMHe5830ysBScHaW4vxvPXEn2uBtB8TC8goxmoIu9My5H746ceyY2xBEn8HA0XZ7pQTrCRimcmUCgYEAxA5a3g/Ni/uwTUAsQJNUPyvjcYxq+E3S2VNsYZiOiogKqXeE0QtasNMh2L7Wv9aan5Xca7eKbHP4fZFxLif/YrwwcmktIX3u5vkGyq2VCAw5V8iGD3vdbDJvAc2+YVBoeWf4w4eDST2Ir6xrM3WCtXR35EM0Jhw+8PAdytIKrVkCgYAqEK5yIr7CnTb0ySPPxi3jE3ZRfZFYTssW0X6bsCQVnHaIsAW6CS6xy7/uEG+qeiuns6DR+Jm1j7wFtnaComdXrhx4ZbpsWofTIUc+NqopUs48ROkVhrkMEgrX26Iw+f7YIdrQNY5O4QW0s8DTKzywsRcoH2oHMShu0Pa2gnfJXQKBgQClzLn9t4GNk0EKY23JAo8piTUkbqp76Fyam5k5g+lvsBLMNB4nJyIADd07bFRyEcvbj8HDeolepEiN8HS1ou+wERQrfVTEURq7S/f5aQhysNvBp/vvlkGv4YrNDLCm3Xgsy8etm6lkQ9yXLAnQj90FFUTazhaI8DQuT/Hx9uU+qQKBgQCBEpc98YikgYmZk/6kyzUP3l+MIj5i3UK/7ZG3QOpTAeTbzBQQX0s31b2Lf9M+SN2+2XJb/0OUr3RKKkuf5KgedMll7hNaEaFu9z5qPepFUlKWZz2MkIRSljecbSJ8ZfGz2wCUhQoW8KLQY9ftEaz+27eEJ0FxHhuhe5+yQMpkKA==

# generate Secret resource object

[root@kubenode1 ingress] # kubectl create-f secret-ingress.yaml

# Editing the yaml file above, using the "kubectl create" command to generate Secret objects is clearer in the steps

# but you can use the "kubectl create secret tls" command to create Secret objects directly

[root@kubenode1 ingress] # kubectl create secret tls secret-ingress-- key / etc/kubernetes/ingress/ingress.key-- cert / etc/kubernetes/ingress/ingress.crt

Create a backend service

# Editing the backend service nginx01-svc-tls.yaml

[root@kubenode1 ingress] # touch nginx01-svc-tls.yaml

[root@kubenode1 ingress] # vim nginx01-svc-tls.yaml

ApiVersion: extensions/v1beta1

Kind: Deployment

Metadata:

Name: nginx01-tls

Spec:

Replicas: 1

Template:

Metadata:

Labels:

Name: nginx01-tls

Spec:

Containers:name: nginx01-tls

Image: nginx:latest

Ports:containerPort: 80

ApiVersion: v1

Kind: Service

Metadata:

Name: nginx01-svc-tls

Spec:

Ports:

# Port number that Service service listens to

Port: 443

# Port number provided by Pod that provides real services at the backend

TargetPort: 80

Name: https

Selector:

Name: nginx01-tls

# Editing the backend service nginx02-svc-tls.yaml

[root@kubenode1 ingress] # cp nginx01-svc-tls.yaml nginx02-svc-tls.yaml

[root@kubenode1 ingress] # sed-I's | nginx01 | nginx02 | g 'nginx02-svc-tls.yaml

# generate backend service

[root@kubenode1 ingress] # kubectl create-f nginx01-svc-tls.yaml

[root@kubenode1 ingress] # kubectl create-f nginx02-svc-tls.yaml

# modify the html file of the nginx container that provides the backend service

# enter the container to modify via "kubectl exec-ti-c / bin/bash"; pod-name can be obtained through the command "kubectl get pods-o wide"; container-name is the name defined in the yaml file

[root@kubenode1 ingress] # kubectl get pods-o wide

# the index.html file of the official nginx container is in the / usr/share/nginx/html/ directory

[root@kubenode1 ingress] # kubectl exec-ti nginx01-tls-59fbf6696c-qfq4k-c nginx01-tls / bin/bash

Root@nginx01-tls-59fbf6696c-qfq4k:/# echo "Welcome to test site nginx01-svc-tls" > / usr/share/nginx/html/index.html

Root@nginx01-tls-59fbf6696c-qfq4k:/# cat / usr/share/nginx/html/index.html

Root@nginx01-tls-59fbf6696c-qfq4k:/# exit

[root@kubenode1 ingress] # kubectl exec-ti nginx02-tls-5559fd9bc7-dfbrp-c nginx02-tls / bin/bash

Root@nginx02-tls-5559fd9bc7-dfbrp:/# echo "Welcome to test site nginx02-svc-tls" > / usr/share/nginx/html/index.html

Root@nginx02-tls-5559fd9bc7-dfbrp:/# cat / usr/share/nginx/html/index.html

Root@nginx02-tls-5559fd9bc7-dfbrp:/# exit

Create an ingress object

Copy the code

# Editing ingress object yaml file

# add "tls" subdomain under "spec" field, add "hosts" field to multiple domain names, and "secretName" field call the corresponding Secret resource

# 1 ingress object can only use one Secret object (the "secretName" field value is unique), that is, only one certificate can be used. This formal requirement supports all domain names under the "hosts" field.

# the "secretName" field must be placed at the end of the domain name list

# the domain name of "hosts" field needs to match the domain name of "rules" field

# ingress by default, when the certificate is not configured or the certificate is misconfigured, a tls certificate will be given by default. If two values are configured in the "secretName" field, all domain names will use the default certificate. If one domain name is missing in the "hosts" field, the default certificate will be used for the missing domain name.

# it may take some time to update the ingress certificate to take effect

[root@kubenode1 ingress] # touch nginx-svc-tls-ingress.yaml

[root@kubenode1 ingress] # vim nginx-svc-tls-ingress.yaml

ApiVersion: extensions/v1beta1

Kind: Ingress

Metadata:

Name: ingress-tls

Spec:

Tls:hosts:nginx01-svc-tls.menginx02-svc-tls.me

SecretName: secret-ingress

Rules:host: nginx01-svc-tls.me

Http:

Paths:backend:

ServiceName: nginx01-svc-tls

# backend service listening port, which is different from container listening port that provides real service

ServicePort: 443host: nginx02-svc-tls.me

Http:

Paths:backend:

ServiceName: nginx02-svc-tls

ServicePort: 443

# generate ingress object

[root@kubenode1 ingress] # kubectl create-f nginx-svc-tls-ingress.yaml

[root@kubenode1 ingress] # kubectl get ingress

Verification

# use-- resolve parameter to simulate dns resolution, and the destination address is the domain name

# http access is redirected, and https access is normal

[root@kubenode1 ingress] # curl-- resolve nginx01-svc-tls.me:80:172.30.200.21 http://nginx01-svc-tls.me

[root@kubenode1 ingress] # curl-- resolve nginx01-svc-tls.me:443:172.30.200.21-k https://nginx01-svc-tls.me

# or use the-H parameter to set the domain name to be accessed in the http header, and the destination address is the ip address

[root@kubenode1 ingress] # curl-H 'Host:nginx01-svc-tls.me'-k https://172.30.200.23

[root@kubenode1 ingress] # curl-H 'Host:nginx02-svc-tls.me'-k https://172.30.200.23

Access the host host in the local browser (please bind the domain name in advance): http://nginx01-svc-tls.me

Using http access, redirect automatic jump to https access

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.