Shulou(Shulou.com)06/01 Report--

Kubeadm installation Kubernetes practice process, I believe that many inexperienced people do not know what to do, so this article summarizes the causes of the problem and solutions, through this article I hope you can solve this problem.

Kubeadm install Kubernetes

Take 1.9.3 as an example:

1.10.x installation reference: https://my.oschina.net/u/2306127/blog/1806263

Domestic 1.10.2, https://www.cnblogs.com/RainingNight/archive/2018/05/02/8975838.html

Give people roses, the hand has an aftertaste of fragrance. Source of this article-https://my.oschina.net/u/2306127/blog/1628082

Environment:

The host uses MacOS+VirtualBox, the virtual machine is Ubuntu Desktop 16.04.4, and the network is set to bridge.

The kubernetes version 1.9.3 uses 17.03.0 (later upgraded to docker-ce 18.02, which also works).

Note:

Kubeadm is still in rapid development and evolution. If you encounter problems, you can check the latest information on the official website (https://kubernetes.io/docs/setup/independent/install-kubeadm/).

If you encounter a problem, you can search https://github.com/kubernetes/kubernetes/issues first.

Or go to github.com/kubernetes for feedback and submit issue to help improve.

Download shell scripts and tools, https://github.com/openthings/kubernetes-tools.

Kubernetes is a powerful platform for container governance. However, for a system that can manage large clusters, it is not easy to install. And because the original Kubernetes-related Docker images and installation files are hosted on gcloud, and link points are inaccessible, manual manipulation is inevitable. Coupled with the rapid development of Kubernetes itself, the differences of various pits and different versions make it more difficult to run Kubernetes successfully.

The easiest way is to use minikube (see: https://my.oschina.net/u/2306127/blog/1621468) or Docker for Mac/Windows (see: https://my.oschina.net/u/2306127/blog/1606599)), but this can only be used in a development environment (it is also useful for desktop-level services), and does not support multi-machine clusters and multi-node scaling.

Dockerhub image of Kubernetes, https://hub.docker.com/r/mirrorgooglecontainers

Reference for the specific use of Kubeadm, https://my.oschina.net/u/2306127/blog/1627651

Kubeadm is a kubernetes installation tool that can be used to quickly deploy kubernets clusters, but the above problem still exists. We can pull the image from the dockerhub and then change it to the desired name. Prior to this, some small manual settings are required in the current version (the basic service of the current version of Kubernetes has been fully containerized, and the installation process of the future version may be fully automated), as follows:

1. Preset

Due to the current version compatibility limitations, several settings need to be adjusted on Ubuntu:

1.1 turn off swap

Temporarily shut down, run: swapoff-a. Next time, there's more.

Permanently delete or close the swap partition in / etc/fstab and comment it out with #.

1.2 shut down and clean up ufw

The following command clears all existing firewall rules:

Iptables-F1.3 set cgroups

Make sure that the cgroup driver used by kubelet is consistent with Docker. Or update the Docker using the following method:

Cat / etc/docker/daemon.json {"exec-opts": ["native.cgroupdriver=systemd"]} EOF

Alternatively, set the cgroup driver of kubernetes, for example, the-- cgroup-driver flag of kubelet to be the same as Docker (e.g. Cgroupfs).

1.4 install docker 17.03

The current docker version verified by Kubernetes 1.9.3 is up to CE 17.03, which is installed using the following method:

Apt-get updateapt-get install-y\ apt-transport-https\ ca-certificates\ curl\ software-properties-commoncurl-fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add- add-apt-repository\ "deb https://download.docker.com/linux/$(. / etc/os-release; echo "$ID")\ $(lsb_release-cs)\ stable "apt-get update & & apt-get install-y docker-ce=$ (apt-cache madison docker-ce | grep 17.03 | head-1 | awk'{print $3}')

Apt-cache madison docker-ce can view the available versions.

! Separate settings for the apt-get proxy.

Apt-get is sometimes inaccessible, even with a system agent. You can try to set up the proxy for apt separately, as follows:

$nano / etc/apt/apt.confAcquire::http::Proxy "http://192.168.199.99:9999";Acquire::https::Proxy" http://192.168.199.99:9999";

Or add the-E parameter to the execution (instructing apt to use environment variables), as follows (see: https://askubuntu.com/questions/257290/configure-proxy-for-apt):

Sudo-E https_proxy=192.168.199.99:9999 apt install docker-ce=17.03.2~ce-0~ubuntu-xenial

Do not update, or docker will be updated to the latest version. You can try the latest version, and it can be run.

! Refer to the format below to set the independent agent for docker and clear all system proxies (because Kubernetes wants to access the local service).

Edit the configuration file for the docker service, and then restart / etc/systemd/system/docker.service.

Describe https://my.oschina.net/u/2306127/blog/1627620 in detail.

Run sudo gedit / lib/systemd/system/docker.service, and add the following settings:

Environment= "HTTP_PROXY= http://192.168.199.99:9999/" Environment=" HTTPS_PROXY= http://192.168.199.99:9999/" Environment= "NO_PROXY=localhost,127.0.0.0/8"

However, you still encounter extremely slow situations during installation, so you can only try more.

2. Pull the image

The original docker image of Kubernetes is on gcloud, and you need to register, log in, and use gcloud tools to get it, even if you use an agent. Using docker can lead to inexplicable failure (error message ambiguity). You can pull a copy of the Kubernetes image from DockerHub and change the name of docker (you can install it later through kubeadm's configuration file, so you should be able to specify the installation source).

2.1 create an automatic pull script

The script is as follows. If you need other container images, you can add them and change the version number to what you need.

Note: each version of kubernetes depends on a different version, and 1.9.3 applies below.

Echo "= =" echo "Set proxy to http://192.168.199.99:9999..."echo"export http_proxy= http://192.168.199.99:9999export https_proxy= http://192.168.199.99:9999echo" = = "echo"echo" Pulling Docker Images from mirrorgooglecontainers... "echo" = > kube-apiserver: "docker pull mirrorgooglecontainers/kube-apiserver-amd64:v1.9.3docker tag mirrorgooglecontainers/kube-apiserver-amd64:v1.9.3 gcr.io/google_containers / kube-apiserver-amd64:v1.9.3echo "= > kube-controller-manager:" docker pull mirrorgooglecontainers/kube-controller-manager-amd64:v1.9.3docker tag mirrorgooglecontainers/kube-controller-manager-amd64:v1.9.3 gcr.io/google_containers/kube-controller-manager-amd64:v1.9.3echo "= = > kube-scheduler:" docker pull mirrorgooglecontainers/kube-scheduler-amd64:v1.9.3 docker tag mirrorgooglecontainers/kube-scheduler-amd64:v1.9.3 gcr.io/google_containers/kube- Scheduler-amd64:v1.9.3echo "= > kube-proxy:" docker pull mirrorgooglecontainers/kube-proxy-amd64:v1.9.3docker tag mirrorgooglecontainers/kube-proxy-amd64:v1.9.3 gcr.io/google_containers/kube-proxy-amd64:v1.9.3echo "= = > k8s-dns-sidecar:" docker pull mirrorgooglecontainers/k8s-dns-sidecar-amd64:1.14.8docker tag mirrorgooglecontainers/k8s-dns-sidecar-amd64:1.14.8 gcr.io/google_containers/k8s-dns-sidecar-amd64: 1.14.8echo "= > k8s-dns-kube-dns:" docker pull mirrorgooglecontainers/k8s-dns-kube-dns-amd64:1.14.8docker tag mirrorgooglecontainers/k8s-dns-kube-dns-amd64:1.14.8 gcr.io/google_containers/k8s-dns-kube-dns-amd64:1.14.8echo "= = > k8s-dns-dnsmasq-nanny:" docker pull mirrorgooglecontainers/k8s-dns-dnsmasq-nanny-amd64:1.14.8docker tag mirrorgooglecontainers/k8s-dns-dnsmasq-nanny-amd64:1.14 .8 gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.8echo "= > etcd:" docker pull mirrorgooglecontainers/etcd-amd64:3.1.11docker tag mirrorgooglecontainers/etcd-amd64:3.1.11 gcr.io/google_containers/etcd-amd64:3.1.11echo "= > pause:" docker pull mirrorgooglecontainers/pause-amd64:3.0docker tag mirrorgooglecontainers/pause-amd64:3.0 gcr.io/google_containers/pause-amd64:3.0echo finished.echo "More update Please visit: https://hub.docker.com/r/mirrorgooglecontainers"echo "" 2.2 run the image acquisition script

It is recommended that you save the above to getkubeimage.sh, and then run it. As follows:

Gedit getkubeimages.sh# copy and paste the contents of the script above. Sudo chmod + x getkubeimages.shsudo. / getkubeimages.sh3, Kubeadm installation 3.1 install kubeadm first

Kubeadm is a command line tool for kubernetes installation and maintenance.

Echo "add Kubernetes installation source authentication key:" sudo curl-sSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key addecho "add Kubernetes installation source:" sudo echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" > / etc/apt/sources.list.d/kubernetes.listecho "Update system package list:" sudo apt updateecho "View the available version of Kubernetes:" apt-cache madison kubeadmecho "installation Kubeadm 1.9.3: "apt-get install-y kubeadm=1.9.3-00 stories! Fixed the error in the kubelet startup parameters.

For kubeadm version 1.9.3. Add the following to the / etc/systemd/system/kubelet.service.d/10-kubeadm.conf file.

[Service] Environment= "KUBELET_CGROUP_ARGS=--cgroup-driver=systemd" Environment= "KUBELET_EXTRA_ARGS=--fail-swap-on=false"

And add KUBELET_CGROUP_ARGS to the startup parameter (which is missing in this version of the installation file).

The final / etc/systemd/system/kubelet.service.d/10-kubeadm.conf file is as follows:

[Service] Environment= "KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf-kubeconfig=/etc/kubernetes/kubelet.conf" Environment= "KUBELET_SYSTEM_PODS_ARGS=--pod-manifest-path=/etc/kubernetes/manifests-allow-privileged=true" Environment= "KUBELET_NETWORK_ARGS=--network-plugin=cni-cni-conf-dir=/etc/cni/net.d-cni-bin-dir=/opt/cni/bin" Environment= "KUBELET_DNS_ARGS=--cluster-dns=10.96 .0.10-cluster-domain=cluster.local "Environment=" KUBELET_AUTHZ_ARGS=--authorization-mode=Webhook-client-ca-file=/etc/kubernetes/pki/ca.crt "Environment=" KUBELET_CADVISOR_ARGS=--cadvisor-port=0 "Environment=" KUBELET_CERTIFICATE_ARGS=--rotate-certificates=true-cert-dir=/var/lib/kubelet/pki "Environment=" KUBELET_CGROUP_ARGS=--cgroup-driver=systemd "Environment=" KUBELET_EXTRA_ARGS=--fail-swap-on=false "ExecStart=ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ ARGS $KUBELET_SYSTEM_PODS_ARGS $KUBELET_NETWORK_ARGS $KUBELET_DNS_ARGS $KUBELET_AUTHZ_ARGS $KUBELET_CADVISOR_ARGS $KUBELET_CERTIFICATE_ARGS $KUBELET_CGROUP_ARGS $KUBELET_EXTRA_ARGS! Then restart the kubelet service, run: systemctl daemon-reloadsystemctl restart kubelet Note: modify the kubelet service configuration file on each node reference above, otherwise the kubelet startup fails.

You can use sudo systemctl status kubelet to view the status of the kubelet service.

3.2 run kubeadm init to start the cluster creation kubeadm init-- kubernetes-version=v1.9.3-- pod-network-cidr=192.168.0.0/16

Or:

Kubeadm init-- kubernetes-version=v1.9.3-pod-network-cidr 10.244.0.0swap 16

Using CoreDNS, add:

Kubeadm init-- kubernetes-version=v1.9.3-pod-network-cidr 10.244.0.0It 16-- feature-gates CoreDNS=true

Refer to https://my.oschina.net/u/2306127/blog/1627651 for specific commands to install using Kubeadm.

Add nod

Remember the add node command that returned the value above and execute it on other nodes.

If you forget, get the above command again and run:

Kubeadm token create-print-join-command-ttl 0

You can use kubeadm token-h to view token operations.

Regenerate the CA authorization code:

# openssl x509-pubkey-in / etc/kubernetes/pki/ca.crt | openssl rsa-pubin-outform der 2 > / dev/null | openssl dgst-sha256-hex | sed's / ^. * /'

See discussion: https://github.com/kubernetes/kubeadm/issues/26

3. 3 install flannel network driver

Follow the prompts after completion above:

Sudo kubectl apply-f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml kids! There is a pending hang problem when installing the flannel network (the podCidr parameter is missing, even if it is specified when kubeadm is running): # View the log: kubectl get pods-- namespace kube-systemkubectl-- the kube-flannel-ds-bvxd2 above namespace kube-system logs kube-flannel-ds-bvxd2# is the pod name of flannel. Enter according to the name returned by get pods. # Edit / etc/kubernetes/manifests/kube-controller-manager.yaml in the command section, add:-- allocate-node-cidrs=true- cluster-cidr=10.244.0.0/16 and then system restart kubelet.

Detailed reference for the solution to the problem: https://github.com/coreos/flannel/issues/728

3.4 the master node is enabled

By default, the master node does not turn on the task node function, but can be opened by command, so that the master node and task node can be deployed to a single machine.

Kubectl taint nodes-all node-role.kubernetes.io/master-3.5 joins the cluster node

Use the kubeadm join command to add other task nodes to the master cluster.

Recovery of kubeadm join-token 8dc9d8.df09161bed020a12 192.168.199.106 discovery-token-ca-cert-hash sha256:16exxx3.6 discovery-token-ca-cert-hash sha256:16exxx3.6 installation failure

If the installation fails, use kubeadm reset to reset the installation environment. Most of the time, you need to restart the operating system to run kubeadm init, because some system network service ports are already occupied and cannot be simply cleared.

Note: if you change the proxy settings, you need to close the console window and then reopen it. Otherwise, the "Settings"-> "Network"-> "Agent" settings dialog box of the Ubuntu system changes the global environment variables http_proxy and https_proxy, which are not cleared and are still valid, resulting in the local service not being connected. This is also why some services such as apt/node.js/github (proxy Settings reference https://my.oschina.net/u/2306127/blog/869727) do not use these environment variables, which are still invalid despite operating system settings.

4. Kubernetes installation success information

Kubernetes is installed successfully, and the output information is as follows:

Root@kube:/home/supermap# kubeadm init-- kubernetes-version v1.9.3-- pod-network-cidr=10.244.0.0/16 [init] Using Kubernetes version: v1.9.3 [init] Using Authorization modes: [Node RBAC] [preflight] Running pre-flight checks. [WARNING FileExisting-crictl]: crictl not found in system path [preflight] Starting the kubelet service [certificates] Generated ca certificate and key. [certificates] Generated apiserver certificate and key. [certificates] apiserver serving cert is signed for DNS names [kube kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.199.111] [certificates] Generated apiserver-kubelet-client certificate and key. [certificates] Generated sa key and public key. [certificates] Generated front-proxy-ca Certificate and key. [certificates] Generated front-proxy-client certificate and key. [certificates] Valid certificates and keys now exist in "/ etc/kubernetes/pki" [kubeconfig] Wrote KubeConfig file to disk: "admin.conf" [kubeconfig] Wrote KubeConfig file to disk: "kubelet.conf" [kubeconfig] Wrote KubeConfig file to disk: "controller-manager.conf" [kubeconfig] Wrote KubeConfig file to disk: "scheduler.conf" [controlplane] Wrote Static Pod manifest for component kube-apiserver to "/ etc/kubernetes/manifests/kube- Apiserver.yaml "[controlplane] Wrote Static Pod manifest for component kube-controller-manager to" / etc/kubernetes/manifests/kube-controller-manager.yaml "[controlplane] Wrote Static Pod manifest for component kube-scheduler to" / etc/kubernetes/manifests/kube-scheduler.yaml "[etcd] Wrote Static Pod manifest for a local etcd instance to" / etc/kubernetes/manifests/etcd.yaml "[init] Waiting for the kubelet to boot up the controlplane as Static Pods from directory" / etc/kubernetes/manifests ".init] This might take A minute or longer if the control plane images have to be pulled. [apiclient] All control plane components are healthy after 33.501916 seconds [uploadconfig] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace [markmaster] Will mark node kube as master by adding a label and a taint [markmaster] Master kube tainted and labelled with key/value: node-role.kubernetes.io/master= "[bootstraptoken] Using token: 8b2ed3.149a349e4b775985 [bootstraptoken] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long Term certificate credentials [bootstraptoken] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstraptoken] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [bootstraptoken] Creating the "cluster-info" ConfigMap in the "kube-public" namespace [addons] Applied essential addon: kube-dns [addons] Applied essential addon: kube-proxyYour Kubernetes master has initialized fulfilled to start using your cluster You need to run the following as a regular user: mkdir-p $HOME/.kube sudo cp-I / etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id-u): $(id-g) $HOME/.kube/configYou should now deploy a podnetwork to the cluster.Run "kubectl apply-f [podnetwork] .yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/You can now join any number of machines by running the following on Each nodeas root: kubeadm join-- token 8b2ed3.149a349e4b775985 192.168.199.111pur6443-- discovery-token-ca-cert-hash sha256:ab69621f2117f2b283df725859724efc71c37a20f6da519237ca1dad5a72d9b2

Follow-up actions, perform:

Mkdir-p $HOME/.kube sudo cp-I / etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id-u): $(id-g) $HOME/.kube/config

Then run kubectl get ns to get the information about the namespace (namespaces). The output is as follows:

Supermap@kube:~$ kubectl get nsNAME STATUS AGEdefault Active 22mkube-public Active 22mkube-system Active 22m

Add this machine as a task node:

Kubectl taint nodes-all node-role.kubernetes.io/master-

View the node status:

Supermap@kube:~$ kubectl get nodeNAME STATUS ROLES AGE VERSIONkube NotReady master 28m v1.9.3

Now, the machine runs both the master and a task node.

View node details:

# display the pods list. Kubectl get pods-- namespace=kube-system-o wide# gets the details of the pod, where kube-dns-6f4fd4bdf-895jh is the node name. Kubectl get-o json-- namespace=kube-system pods/kube-dns-6f4fd4bdf-895jh5, install dashboard and helm

The following images are required for subsequent installation of dns, dashboard and helm, pull down the agent on the shelf, or find the pull on the hub.docker.com and make the name in the docker tag, otherwise the kubectl get pods will be in the pending status all the time.

Docker pull quay.io/coreos/flannel:v0.10.0-amd64docker pull gcr.io/google_containers/k8s-dns-kube-dns-amd64:1.14.7docker pull k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3docker pull gcr.io/kubernetes-helm/tiller:v2.8.1

Reference:

Kubernetes Dashboard panel service installation, https://my.oschina.net/u/2306127/blog/1606599

Dashboard installation authorization login details reference, https://my.oschina.net/u/2306127/blog/1630861

Helm package Management Service installation, https://my.oschina.net/u/2306127/blog/1619818

Map the port of dashboard so that it can be accessed by the outside:

Kubectl port-forward kubernetes-dashboard-7798c48646-wkgk4 8443-- namespace=kube-system &

Note that kubernetes-dashboard-7798c48646-wkgk4 is the pod name of the dashboard listed by kubectl get pods-- namespace=kube-system-o wide.

Dashboard still has some problems with licensing on systems installed using kubeadm. For more information, please see: https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/#deploying-the-dashboard-ui

Now you can open a browser and type http://localhost:8443 to see how kubernetes is running.

The great task has been completed.

After reading the above, have you mastered the practical process of installing Kubernetes in kubeadm? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.