Shulou(Shulou.com)05/31 Report--

In this article, the editor introduces in detail "what are the certificates of kubernetes", the content is detailed, the steps are clear, and the details are handled properly. I hope that this article "what are the certificates of kubernetes" can help you solve your doubts? let's follow the editor's ideas to slowly deepen, let's learn new knowledge.

Foolishly unclear kubernetes certificate

Kubeadm generated a bunch of certificates is not very fooling, these things are not so magical, to delve into their underwear.

Root@k8s-master:/etc/kubernetes/pki# tree. |-- apiserver.crt |-- apiserver-etcd-client.crt |-- apiserver-etcd-client.key |-- apiserver.key |-- apiserver-kubelet-client.crt |-- apiserver-kubelet-client.key |-- ca.crt |-- ca.key |-- etcd | |-- ca.crt | |-- ca.key | |-- healthcheck-client.crt |-- healthcheck-client.key | |-- peer.crt | |-- peer. Key | |-- server.crt | `--server.key |-- front-proxy-ca.crt |-- front-proxy-ca.key |-- front-proxy-client.crt |-- front-proxy-client.key |-- sa.key`-- sa.pub1 directory 22 files starts with RSA

To have an in-depth understanding of the role of certificates, we first need to understand some principles and have some basic knowledge, such as what is asymmetric encryption, what is public key, private key, what is digital signature and so on. Let's start with RSA algorithm.

Asymmetric encryption generates a key pair, such as the sa.key sa.pub above, one for encryption and one for decryption.

Plaintext + public key = > ciphertext

Ciphertext + private key = > plaintext

Then there is no private key at this time, it is difficult to decrypt the ciphertext.

Take a closer look at the principle. If you don't want to pay attention, you can skip the following part of the principle:

Suppose we want to encrypt a word Caesar, first turn it into a string of numbers, such as Ascii code X = 067097101115097114, which is the clear code we need to encrypt. Now let's encrypt X.

Find two large prime numbers P and Q to calculate their product N = P * Q and then let M = (P-1) (Q-1)

Find a number E satisfies that E and M have no common divisor except 1

Find a number D which satisfies E times D divided by M remainder 1, E * D mod M = 1

Now E is the public key, which can be encrypted to anyone.

D is the private key, which is used for decryption. You must keep it yourself.

The N that connects the public key and the private key is public, why can this be made public? it is because it is very easy to calculate N according to P Q, but it is very difficult to decompose N into P Q two large prime numbers, so it is difficult to crack the existing computer computing power.

Now let's encrypt:

Pow (XMague E) mod N = Y Y is ciphertext, and it is impossible to calculate X (plaintext) without D (private key) immortal.

Decryption:

Pow (YMagazine D) mod N = X X is plaintext, and plaintext comes out.

Isn't mathematics amazing? now you can think of sa.key = D sa.pub = E.

Digital signature

Suppose you write a letter to your boss saying, "I admire you, boss", and then ask your colleague to send the letter to your boss. How can you be sure that you wrote the letter, and how to prevent your colleague from changing it to "boss you are a SB" during the delivery process?

You can do this. First you generate a key pair, give the public key to the boss, then make a hash summary of the contents of the letter, and then encrypt the digest with the private key. The result is the signature.

In this way, the boss decrypts the letter with the public key and finds that the hash value obtained is consistent with the hash value of the letter, thus confirming that you wrote the letter.

So digital signature is an application of encryption technology, and the difference between fully encrypted information and fully encrypted information is that the information here is public, and your colleagues can see you flatter your boss.

Digital certificate root certificate and certificate

Usually when we configure the https service, we need to apply for a certificate from the "authority".

The process goes like this:

The website creates a key pair to provide public keys and organizational and personal information to the authorities

The authoritative organization issues certificates.

Friends who browse the web use the root certificate and public key of the authority to decrypt the signature, compare the abstract, and determine the legitimacy.

Validity time for client to verify domain name information (browsers basically have built-in CA public keys of major authorities)

This certificate contains the following:

Applicant's public key

Applicant organization and personal information

Issuing agency CA information, valid time, serial number, etc.

Signature of the above information

The root certificate is also known as the self-signed certificate, which is the certificate issued to yourself. CA (Certificate Authority) is called a certificate authority, and the ca certificate in K8s is the root certificate.

Kubernetes certificate

With the above foundation, let's officially begin.

Classify first:

Key pair: sa.key sa.pub root certificate: ca.crt etcd/ca private key: ca.key and other certificates

First of all, the other certificates are issued by the CA root certificate, kubernetes and etcd use a different CA, it is important that the certificate is used for client-side verification or server-side verification. Let's look at the following one:

Service Account key pair sa.key sa.pub

Provided to kube-controller-manager for use. Kube-controller-manager signs the token through sa.key, and the master node verifies the signature through the public key sa.pub. For example, kube-proxy runs in the form of pod, and in pod, you directly use service account and kube-apiserver for authentication. In this case, you no longer need to create a certificate for kube-proxy separately, but use token verification directly.

Root certificate

Pki/ca.crtpki/ca.key

For k8s cluster certificate issuing authority

Apiserver certificate

Pki/apiserver.crtpki/apiserver.key

Kubelet certificate

Pki/apiserver-kubelet-client.crtpki/apiserver-kubelet-client.key

In order for kubelet to actively access kube-apiserver, kube-apiserver also needs to initiate a request to kubelet, so both parties need to have their own root certificate and server certificate and client certificate issued by the root certificate. In kube-apiserver, server-side certificates and client-side certificates with CN user name information are generally specified for https access. In the startup configuration of kubelet, only the ca root certificate is specified, but the server-side certificate for https access is not explicitly specified. When generating the server-side certificate, the server address or hostname is generally specified, and the kube-apiserver changes relatively infrequently, so the IP or hostname / domain name used as kube-apiserver can be pre-allocated at the beginning of creating the cluster. However, because the kubelet deployed on the node node changes frequently due to the change of the size of the cluster, it is impossible to predict all the IP information of the node, so the server certificate is not explicitly specified on the kubelet, but only the ca root certificate is specified, which allows kubelet to automatically generate the server certificate according to the local host information and save it to the configured cert-dir folder.

Aggregation certificate

Proxy root certificate:

Pki/front-proxy-ca.crtpki/front-proxy-ca.key

The client certificate issued by the proxy root certificate:

Pki/front-proxy-client.crtpki/front-proxy-client.key

For example, when using kubectl proxy proxy access, kube-apiserver uses this certificate to verify that the client certificate is a self-issued certificate.

Etcd root certificate

Pki/etcd/ca.crtpki/etcd/ca.key

Etcd nodes communicate with each other with peer certificate

Issued by root certificate

Pki/etcd/peer.crtpki/etcd/peer.key

Liveness probe client certificate in pod

Pki/etcd/healthcheck-client.crtpki/etcd/healthcheck-client.key

You can view the yaml probe configuration:

Liveness: exec [/ bin/sh-ec ETCDCTL_API=3 etcdctl\-- endpoints= https://[127.0.0.1]:2379\-- cacert=/etc/kubernetes/pki/etcd/ca.crt\-- cert=/etc/kubernetes/pki/etcd/healthcheck-client.crt\-key=/etc/kubernetes/pki/etcd/healthcheck-client.key get foo]\ delay=15s timeout=15s period=10s # success=1 # failure=8

Certificate for apiserver to access etcd

Pki/apiserver-etcd-client.crtpki/apiserver-etcd-client.key

Note the difference between the client certificate and the server certificate, which usually verifies the address and domain name.

Code implementation

Kubeadm wrote the certificate time to 1 year (client-go wrote it), which was a sad story, causing sealos to strip out the logic of certificate generation to allow the installation to support any expiration time.

The following is an in-depth experience of kubeadm certificate generation according to the source code. It may be a bit tiring to look at the kubeadm code directly, and it is easier to read the core code from the sealos/cert directory.

In order to highlight the core logic, remove some error handling details from the code and read the github.com/fanux/sealos/cert source code if you are interested.

Key pair generation / / create sa.key sa.pub for service Accountfunc GenerateServiceAccountKeyPaire (dir string) error {key, err: = NewPrivateKey (x509.RSA) pub: = key.Public () err = WriteKey (dir, "sa", key) return WritePublicKey (dir, "sa", pub)}

Generate the private key. The keyType here is x509.RSA.

Func NewPrivateKey (keyType x509.PublicKeyAlgorithm) (crypto.Signer, error) {if keyType = = x509.ECDSA {return ecdsa.GenerateKey (elliptic.P256 (), rand.Reader)} return rsa.GenerateKey (rand.Reader, rsaKeySize)} generate CA certificate

Ca.crt (self-signed certificate) ca.key (private key) will be returned

Func NewCaCertAndKey (cfg Config) (* x509.Certificate, crypto.Signer, error) {key, err: = NewPrivateKey (x509.UnknownPublicKeyAlgorithm) cert, err: = NewSelfSignedCACert (key, cfg.CommonName, cfg.Organization, cfg.Year) return cert, key, nil}

Generate a self-signed certificate based on the private key, and NotAfter is the expiration time of the certificate. We friendly to add a variable instead of writing it to death:

/ NewSelfSignedCACert creates a CA certificatefunc NewSelfSignedCACert (key crypto.Signer, commonName string, organization [] string, year time.Duration) (* x509.Certificate, error) {now: = time.Now () tmpl: = x509.Certificate {SerialNumber: new (big.Int) .SetInt64 (0), Subject: pkix.Name {CommonName: commonName, Organization: organization }, NotBefore: now.UTC (), NotAfter: now.Add (duration365d * year). UTC (), KeyUsage: x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature | x509.KeyUsageCertSign, BasicConstraintsValid: true, IsCA: true } certDERBytes, err: = x509.CreateCertificate (rand.Reader, & tmpl, & tmpl, key.Public (), key) return x509.ParseCertificate (certDERBytes)}

Pay close attention to the CommonName and Organization fields, which are very useful. For example, we create a K8s user to specify which user group the user belongs to, corresponding to the above two fields.

For example, if the fanux in the certificate belongs to the sealyun organization, then generating a kubeconfig is equivalent to having the user fanux, so that K8s only needs to verify the signature when doing authentication, and does not need to access the database to do authentication, which is very conducive to the horizontal expansion of apiserver.

Generate other certificates

The key pair is still generated by itself, and then the root certificate information is brought with it when signing the certificate.

Func NewCaCertAndKeyFromRoot (cfg Config, caCert * x509.Certificate, caKey crypto.Signer) (* x509.Certificate, crypto.Signer, error) {key, err: = NewPrivateKey (x509.UnknownPublicKeyAlgorithm) cert, err: = NewSignedCert (cfg, key, caCert, caKey) return cert, key, nil}

At this point, CommonName must be required, and Usages also has to specify whether it is used by the server or the client. Pay attention to the difference from the above SelfSign.

/ / NewSignedCert creates a signed certificate using the given CA certificate and keyfunc NewSignedCert (cfg Config, key crypto.Signer, caCert * x509.Certificate, caKey crypto.Signer) (* x509.Certificate, error) {serial, err: = rand.Int (rand.Reader, new (big.Int) .SetInt64 (math.MaxInt64)) if len (cfg.CommonName) = 0 {return nil Errors.New ("must specify a CommonName")} if len (cfg.Usages) = 0 {return nil, errors.New ("must specify at least one ExtKeyUsage")} certTmpl: = x509.Certificate {Subject: pkix.Name {CommonName: cfg.CommonName, Organization: cfg.Organization,} DNSNames: cfg.AltNames.DNSNames, IPAddresses: cfg.AltNames.IPs, SerialNumber: serial, NotBefore: caCert.NotBefore, NotAfter: time.Now (). Add (duration365d * cfg.Year). UTC (), KeyUsage: x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature, ExtKeyUsage: cfg.Usages } certDERBytes, err: = x509.CreateCertificate (rand.Reader, & certTmpl, caCert, key.Public (), caKey) return x509.ParseCertificate (certDERBytes)} all certificates in kubernetes

Root certificate list

Var caList = [] Config {{Path: BasePath, BaseName: "ca", CommonName: "kubernetes", Organization: nil, Year: 100, AltNames: AltNames {}, Usages: nil,} {Path: BasePath, BaseName: "front-proxy-ca", CommonName: "front-proxy-ca", Organization: nil, Year: 100, AltNames: AltNames {}, Usages: nil,} {Path: EtcdBasePath, BaseName: "ca", CommonName: "etcd-ca", Organization: nil, Year: 100, AltNames: AltNames {}, Usages: nil,},}

List of other signing certificates

Var certList = [] Config {{Path: BasePath, BaseName: "apiserver", CAName: "kubernetes", CommonName: "kube-apiserver", Organization: nil, Year: 100 AltNames: AltNames {/ / you also need to add the server IP user-defined domain name to DNSNames: [] string {"apiserver.cluster.local", "localhost", "master" during the actual installation. "kubernetes", "kubernetes.default", "kubernetes.default.svc",}, IPs: [] net.IP {{127pr 0pr 0jue 1} },}, Usages: [] x509.ExtKeyUsage {x509.ExtKeyUsageServerAuth}, / / the purpose is server verification}, {Path: BasePath, BaseName: "apiserver-kubelet-client", CAName: "kubernetes" CommonName: "kube-apiserver-kubelet-client", Organization: [] string {"system:masters"}, Year: 100, AltNames: AltNames {}, Usages: [] x509.ExtKeyUsage {x509.ExtKeyUsageClientAuth},}, {Path: BasePath BaseName: "front-proxy-client", CAName: "front-proxy-ca", CommonName: "front-proxy-client", Organization: nil, Year: 100, AltNames: AltNames {}, Usages: [] x509.ExtKeyUsage {x509.ExtKeyUsageClientAuth} }, {Path: BasePath, BaseName: "apiserver-etcd-client", CAName: "etcd-ca", CommonName: "kube-apiserver-etcd-client", Organization: [] string {"system:masters"}, Year: 100 AltNames: AltNames {}, Usages: [] x509.ExtKeyUsage {x509.ExtKeyUsageClientAuth},}, {Path: EtcdBasePath, BaseName: "server", CAName: "etcd-ca", CommonName: "etcd" / / kubeadm etcd server certificate common name uses node names This is also the Organization: nil, Year: 100, AltNames: AltNames {}, / / node name that needs to be changed when calling. Node IP, etc., plus Usages: [] x509.ExtKeyUsage {x509.ExtKeyUsageServerAuth, x509.ExtKeyUsageClientAuth},}, {Path: EtcdBasePath, BaseName: "peer", CAName: "etcd-ca", CommonName: "etcd-peer" / / synonymous with etcd server Organization: nil, Year: 100, AltNames: AltNames {}, / / synonymous with etcd server Usages: [] x509.ExtKeyUsage {x509.ExtKeyUsageServerAuth, x509.ExtKeyUsageClientAuth},}, {Path: EtcdBasePath BaseName: "healthcheck-client", CAName: "etcd-ca", CommonName: "kube-etcd-healthcheck-client", Organization: [] string {"system:masters"}, Year: 100, AltNames: AltNames {} Usages: [] x509.ExtKeyUsage {x509.ExtKeyUsageClientAuth},},}

It is very important to note that IP and domain name need to be added when installing the certificate verified by server, and the commonName of etcd should also be set to node name.

Look at the certificate information generated at last:

Apiserver:

[root@iZ2ze4ry74x8bh4cweeg69Z pki] # openssl x509-in / etc/kubernetes/pki/apiserver.crt-text-nooutCertificate:... Signature Algorithm: sha256WithRSAEncryption Issuer: CN=kubernetes Validity Not Before: Mar 31 09:18:06 2020 GMT Not After: Mar 8 09:18:06 2119 GMT Subject: CN=kube-apiserver... X509v3 extensions: X509v3 Key Usage: critical Digital Signature, Key Encipherment X509v3 Extended Key Usage: TLS Web Server Authentication X509v3 Subject Alternative Name: DNS:iz2ze4ry74x8bh4cweeg69z, DNS:kubernetes, DNS:kubernetes.default, DNS:kubernetes.default.svc, DNS:kubernetes.default.svc.cluster.local, DNS:apiserver.cluster.local, DNS:apiserver.cluster.local, IP Address:10.96.0.1 IP Address:172.16.9.192, IP Address:127.0.0.1, IP Address:172.16.9.192, IP Address:172.16.9.193, IP Address:172.16.9.194, IP Address:10.103.97.2 Signature Algorithm: sha256WithRSAEncryption

Etcd server:

[root@iZ2ze4ry74x8bh4cweeg69Z pki] # openssl x509-in / etc/kubernetes/pki/etcd/server.crt-text-nooutCertificate: Data: Version: 3 (0x2) Serial Number: 1930981199811083392 (0x1acc392ba2b27c80) Signature Algorithm: sha256WithRSAEncryption Issuer: CN=etcd-ca Validity Not Before: Mar 31 09:18:07 2020 GMT Not After: Mar 8 09:18:07 2119 GMT Subject: CN=iz2ze4ry74x8bh4cweeg69z... X509v3 Extended Key Usage: TLS Web Server Authentication, TLS Web Client Authentication X509v3 Subject Alternative Name: DNS:iz2ze4ry74x8bh4cweeg69z, DNS:localhost, IP Address:172.16.9.192, IP Address:127.0.0.1, IP Address:0:0:0:0:0:0:0:1 Signature Algorithm: sha256WithRSAEncryption generate user certificate and kubeconfig

Now there is an intern fanux to the company, also want to use K8s, resolutely do not rest assured that the admin kubeconfig to him, then how to do? With the above basics, I will further teach you how to assign a separate kubeconfig to fanux.

Load the root certificate and private key from disk

Generate the user's certificate of fanux. Common name is fanux.

Encode into pem format

Write kubeconfig, write disk

Func GenerateKubeconfig (conf Config) error {certs, err: = cert.CertsFromFile (conf.CACrtFile) caCert: = certs [0] cert: = EncodeCertPEM (caCert) caKey,err: = TryLoadKeyFromDisk (conf.CAKeyFile) / / here conf.User is fanux and conf.Groups is the user group Can be multiple clientCert,clientKey,err: = NewCertAndKey (caCert,caKey,conf.User,conf.Groups,conf.DNSNames,conf.IPAddresses) encodedClientKey Err: = keyutil.MarshalPrivateKeyToPEM (clientKey) encodedClientCert: = EncodeCertPEM (clientCert) / / Triple information for building kubeconfig config: = & api.Config {Clusters: map [string] * api.Cluster {conf.ClusterName: {Server: conf.Apiserver / / Cluster address, such as root certificate in https://apiserver.cluster.local:6443 CertificateAuthorityData: cert, / / pem format For https},}, Contexts: map [string] * api.Context {ctx: {/ / triple information, user name fanux, cluster name above And namespace there is no Cluster: conf.ClusterName, AuthInfo: conf.User,},}, AuthInfos: map [string] * api.AuthInfo {/ / user information, so it is useless for you to change the user in kubeconfig directly. Because k8s only recognizes the name in the certificate conf.User:&api.AuthInfo {ClientCertificateData: encodedClientCert, / / pem format user certificate ClientKeyData: encodedClientKey, / / pem format user private key},} CurrentContext: ctx, / / current context, kubeconfig can support multi-user and multi-cluster} err = clientcmd.WriteToFile (* config, conf.OutPut) return nil}

The user certificate and private key are generated. Just like the certificate signed above, user is fanux and group is user group:

Func NewCertAndKey (caCert * x509.Certificate, caKey crypto.Signer, user string, groups [] string, DNSNames [] string,IPAddresses [] net.IP) (* x509.Certificate, crypto.Signer, error) {key,err: = rsa.GenerateKey (rand.Reader, 2048) serial, err: = rand.Int (rand.Reader) New (big.Int) .SetInt64 (math.MaxInt64) certTmpl: = x509.Certificate {Subject: pkix.Name {CommonName: user, Organization: groups,}, DNSNames: DNSNames, IPAddresses: IPAddresses, SerialNumber: serial NotBefore: caCert.NotBefore, NotAfter: time.Now (). Add (time.Hour * 24 * 36599). UTC (), KeyUsage: x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature, ExtKeyUsage: [] x509.ExtKeyUsage {x509.ExtKeyUsageClientAuth},} certDERBytes, err: = x509.CreateCertificate (rand.Reader, & certTmpl, caCert, key.Public () CaKey) cert,err: = x509.ParseCertificate (certDERBytes) return cert,key,nil}

Then the partner's kubeconfig is generated without any permission at this time:

Kubectl-kubeconfig. / kube/config get podError from server (Forbidden): pods is forbidden: User "fanux" cannot list resource "pods" in API group.

Finally, it is OK to play RBAC. Here you can bind an administrator right directly.

Kind: ClusterRoleBindingapiVersion: rbac.authorization.k8s.io/v1metadata: name: user-admin-testsubjects:- kind: User name: "fanux" # Name is case sensitive apiGroup: rbac.authorization.k8s.ioroleRef: kind: ClusterRole name: cluster-admin # using admin role apiGroup: rbac.authorization.k8s.io here, this article "what are the certificates of kubernetes" has been introduced. If you want to master the knowledge points of this article, you still need to practice and use it. If you want to know more about related articles, welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.