Shulou(Shulou.com)06/01 Report--

This article mainly introduces the use of TLS bootstrapping in kubernetes, has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let the editor take you to understand it.

One: preface

When TLS authentication is enabled in the cluster, the kubelet components of each node must use a valid certificate issued by CA used by apiserver to communicate with apiserver. If there are more nodes, it will be very cumbersome to sign a certificate for each node. The TLS bootstrapping function is to let kubelet first connect to apiserver with a predetermined low-privilege user, and then apply for a certificate from apiserver. The certificate of kubelet is dynamically signed by apiserver.

Bootstrap is a program that exists in many systems. For example, Linux's bootstrap,bootstrap is usually loaded as a preconfiguration when it is turned on or when the system starts, which can be used to generate a specified environment. Kubernetes's kubelet can also load such a configuration file when it starts, and its content is similar to the following:

02b50b05283e98dd0fd71db496ef01e8 authorkubeletlybootstrap 10001, "system:kubelet-bootstrap"

The workflow under the RBAC authorization model is roughly as follows:

II: terms related to TLS bootstrapping

1.kubernet related port

When working, the kubelet component adopts an active query mechanism, that is, it periodically requests apiserver to get the tasks it should handle, such as which pod is assigned to itself, so as to deal with these tasks; at the same time, kubelet itself will expose two ports of its own api, which are used to expose its own private api, which are 10250 and 10255 respectively. For port 10250, kubelet uses TLS encryption to provide appropriate authentication; for port 10255, kubelet exposes the private api of the component itself as read-only without authentication

That is to say, certificates are actually used in two places on kubelet, one is the certificate used to communicate with API server, and the other is the certificate required by kubelet's 10250 private api port.

2.CSR request type

All CSR requests initiated by kubelet are actually signed by controller manager. For controller manager, there are three types of CSR requests initiated by kubelet under TLS bootstrapping:

1. Nodeclient: the client authentication request O=system:nodes of the user, CN=system:node: (node name).

2. Selfnodeclient: update nodes with client certificates with the same O and CN.

3. Selfnodeserver: the node that updates the service certificate.

The CSR of nodeclient type is generated only when it is started for the first time. The CSR request of selfnodeclient type is actually generated by the certificate that kubelet renew uses when communicating with apiserver as client, while the CSR request of selfnodeserver type is generated when kubelet first applies for or subsequently renew its own 10250 api port certificate.

Third, the specific boot process of TLS bootstrapping

1.TLS action

It is well known that the function of TLS is to encrypt communications and prevent eavesdropping by middlemen; at the same time, it is impossible to establish a connection with apiserver if the certificate is not trusted, not to mention whether or not you have the permission to request specified content from apiserver.

2.RBAC action

When TLS solves the communication problem, then the permission problem should be solved by RBAC (you can use other permission models, such as ABAC); RBAC specifies which api a user or user group (subject) has the right to request; when cooperating with TLS encryption, apiserver actually reads the CN field of the client certificate as the user name and the O field as the user group.

The above description: first, if you want to communicate with apiserver, you must use a certificate issued by apiserver CA, so that you can form a trust relationship and establish a TLS connection; second, you can provide users and user groups required by RBAC through the CN and O fields of the certificate.

3.kubelet starts the process for the first time

Since the TLS bootstrapping function is to let the kubelet component go to apiserver to apply for a certificate and then use it to connect to apiserver;, how can I connect to apiserver without a certificate when starting for the first time?

A token.csv file is specified in the apiserver configuration, which is a preset user configuration; at the same time, the user's Token and apiserver's CA certificates are written into the bootstrap.kubeconfig configuration file used by kubelet; so that on the first request, kubelet uses the apiserver CA certificate in bootstrap.kubeconfig to establish TLS communication with apiserver, and uses the user Token in bootstrap.kubeconfig to declare his RBAC authorization identity to apiserver.

When you start for the first time, you may encounter an error that kubelet reports that 401 does not have access to apiserver; this is because by default, kubelet declares its identity through the default user Token in bootstrap.kubeconfig, and then creates a CSR request; but don't forget that this user doesn't have any permissions without us processing, including creating a CSR request. So you need the following command to create a ClusterRoleBinding that binds the preset user kubelet-bootstrap to the built-in ClusterRole system:node-bootstrapper so that it can initiate a CSR request

Click (here) to collapse or open

Cd / etc/kubernetes

Export KUBE_APISERVER= "https://x.x.x.x:6443"

# set cluster parameters

Kubectl config set-cluster kubernetes\

-- certificate-authority=/etc/kubernetes/ssl/ca.pem\

-- embed-certs=true\

-- server=$ {KUBE_APISERVER}\

-- kubeconfig=bootstrap.kubeconfig

# set client authentication parameters

Kubectl config set-credentials kubelet-bootstrap\

-- token=$ {BOOTSTRAP_TOKEN}\

-- kubeconfig=bootstrap.kubeconfig

# setting context parameters

Kubectl config set-context default\

-- cluster=kubernetes\

-- user=kubelet-bootstrap\

-- kubeconfig=bootstrap.kubeconfig

# set up and down by default?

Kubectl config use-context default-kubeconfig=bootstrap.kubeco

Nfig

4. Issue the certificate manually

After kubelet is started for the first time, if there is no problem with the user Token and the RBAC is set accordingly, then the CSR request initiated by kubelet should be seen in the cluster:

After a CSR request occurs, you can use kubectl to manually issue (allow) the certificate of kubelet

Kubectl certificate approve node-csr-ICyEqgl55a222oGUbA3U5CE22roAKo6AoGT6Eff_ehY

When the certificate is successfully signed, the kubelet of the target node writes the certificate to the directory specified by the-- cert-dir= option

The certificate used by kubelet to communicate with apiserver is kubelet-client.crt, and the remaining kubelet.crt will be used by kubelet server (10250) for authentication. Note that the certificate of kubelet.crt is a self-signed CA independent of apiserver CA, and the kubelet component will regenerate it after deletion.

IV: certificates and configuration files

1.token.csv

The file is a user description file in the basic format of Token, user name, UID, and user group; this file is loaded by apiserver when apiserver is started, and then a user is created in the cluster; then he can be authorized with RBAC; when the component of the user Token accesses apiserver, apiserver processes the request according to the permissions defined by RBAC

2.bootstarp.kubeconfig

The file contains the Token of the user in token.csv and the apiserver CA certificate. The file is loaded when kubelet is started for the first time, and the apiserver CA certificate is used to establish TLS communication with apiserver, and the user Token is used as the identity to initiate a CSR request like apiserver.

3.kubelet-client.crt

The file is generated after kubelet completes the TLS bootstrapping, and the certificate is signed by controller manager, after which kubelet will load the certificate to establish TLS communication with apiserver, using the CN field of the certificate as the user name and the O field as the user group to initiate other requests to apiserver

4.kubelet.crt

This file is generated only after kubelet has completed TLS bootstrapping and has not configured-- feature-gates=RotateKubeletServerCertificate=true; in this case, it is a self-signed CA certificate independent of apiserver CA and valid for 1 year; it is used as a kubelet 10250 api port

5.apiserver profile

Click (here) to collapse or open

# # #

# # kubernetes system config

# #

# # The following values are used to configure the kube-apiserver

# #

#

# # The address on the local server to listen to.

KUBE_API_ADDRESS= "- advertise-address=10.116.137.196-bind-address=10.116.137.196-insecure-bind-address=10.116.137.196"

#

# # The port on the local server to listen on.

# KUBE_API_PORT= "--port=8080"

#

# # Port minions listen on

# KUBELET_PORT= "--kubelet-port=10250"

#

# # Comma separated list of nodes in the etcd cluster

KUBE_ETCD_SERVERS= "--etcd-servers= https://10.116.137.196:2379,https://10.116.82.28:2379,https://10.116.36.57:2379"

#

# # Address range to use for services

KUBE_SERVICE_ADDRESSES= "--service-cluster-ip-range=10.254.0.0/16"

#

# # default admission control policies

KUBE_ADMISSION_CONTROL= "--admission-control=ServiceAccount,NamespaceLifecycle,NamespaceExists,LimitRanger,ResourceQuota"

#

# # Add your own!

KUBE_API_ARGS= "- authorization-mode=RBAC-runtime-config=rbac.authorization.k8s.io/v1beta1-kubelet-https=true-experimental-bootstrap-token-auth-token-auth-file=/etc/kubernetes/token.csv-service-node-port-range=30000-32767-tls-cert-file=/etc/kubernetes/ssl/kubernetes.pem-tls-private-key-file=/etc/kubernetes/ssl/kubernetes-key.pem-client-ca-file=/etc/kubernetes/ssl/ca.pem- -service-account-key-file=/etc/kubernetes/ssl/ca-key.pem-etcd-cafile=/etc/kubernetes/ssl/ca.pem-- etcd-certfile=/etc/kubernetes/ssl/kubernetes.pem-etcd-keyfile=/etc/kubernetes/ssl/kubernetes-key.pem-- enable-swagger-ui=true-- apiserver-count=1-- audit-log-maxage=30-- audit-log-maxbackup=3-- audit-log-maxsize=100-- audit-log-path=/var/lib/audit.log-- event-ttl=1h "

6.kubelet profile

Click (here) to collapse or open

# # kubelet (minion) config

#

# # The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)

KUBELET_ADDRESS= "--address=10.116.82.28"

#

# # The port for the info server to serve on

# KUBELET_PORT= "--port=10250"

#

# # You may leave this blank to use the actual hostname

KUBELET_HOSTNAME= "--hostname-override=10.116.82.28"

#

# # location of the api-server

# KUBELET_API_SERVER= "--api-servers= http://10.116.137.196:8080"

#

# # pod infrastructure container

# KUBELET_POD_INFRA_CONTAINER= "--pod-infra-container-image=sz-pg-oam-docker-hub-001.tendcloud.com/library/pod-infrastructure:rhel7"

KUBELET_POD_INFRA_CONTAINER= "--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure"

#

# # Add your own!

KUBELET_ARGS= "--cgroup-driver=systemd-- cluster-dns=10.254.0.2-- experimental-bootstrap-kubeconfig=/etc/kubernetes/bootstrap.kubeconfig-- kubeconfig=/etc/kubernetes/kubelet.kubeconfig-- require-kubeconfig-- cert-dir=/etc/kubernetes/ssl-- cluster-domain=cluster.local. Hairpin-mode promiscuous-bridge-- serialize-image-pulls=false-- runtime-cgroups=/systemd/system.slice-- kubelet-cgroups=/systemd/system.slice"

Thank you for reading this article carefully. I hope the article "what is the use of TLS bootstrapping in kubernetes" shared by the editor will be helpful to you. At the same time, I also hope you will support us and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.