Shulou(Shulou.com)06/03 Report--

This article describes how to deploy a highly available k8s cluster through Kubespray, which is version 1.12.5.

1. Deployment manual

Code repository: https://github.com/kubernetes-sigs/kubespray

Reference document: https://kubespray.io/#/

2. K8s master machine configuration

3. K8s cluster installation step step1: set secret-free login between hosts

Since kubespray relies on ansible,ansible for inter-host access through ssh protocol, you need to set secret-free login between hosts before deployment, as follows:

Ssh-keygen-t rsascp ~ / .ssh/id_rsa.pub root@IP:/root/.sshssh root@IPcat / root/.ssh/id_rsa.pub > > / root/.ssh/authorized_keysstep2: download kubespray

Note: do not use the code from the master branch of the github repository, I am using tag v2.8.3 for deployment

Wget https://github.com/kubernetes-sigs/kubespray/archive/v2.8.3.tar.gztar-xvf v2.8.3cd kubespray-v2.8.33.1 replace Mirror

Most of the Kubernetes installations use foreign images, which cannot be obtained due to firewalls, so you need to create your own image repository and upload these images to the image repository.

3.1.1 create an image repository

The component we selected for the image warehouse is Harbor. For more information on installation steps, please see:

Https://github.com/goharbor/harbor/blob/master/docs/installation_guide.md

3.1.2 sort out the images that need to be used in k8s cluster deployment

In the file roles/download/defaults/main.yml file, you can see the full list of images used. Note that some images are not used for the time being because the feature is not used. We mainly use the following images:

3.1.3 download and upload the required images to the private image repository

The list of images used is as follows. Here I applied for a foreign Ali CVM, downloaded the required images on the host and uploaded them to the private image warehouse.

For example, when operating a certain image, you need to execute the following command:

Docker pull gcr.io/google_containers/kubernetes-dashboard-amd64:v1.10.0docker tag gcr.io/google_containers/kubernetes-dashboard-amd64:v1.10.0 106.14.219.69:5000/google_containers/kubernetes-dashboard-amd64:v1.10.0docker push 106.14.219.69:5000/google_containers/kubernetes-dashboard-amd64:v1.10.03.1.4 changes mirror address and modifies Docker configuration

Add the following configuration to the inventory/testcluster/group_vars/k8s-cluster/k8s-cluster.yml file:

# kubernetes image repo definekube_image_repo: "10.0.0.183:5000/google_containers" # # modified by: robbin# comment: modify the image repository of the component to private image repository address etcd_image_repo: "10.0.0.183:5000/coreos/etcd" coredns_image_repo: "10.0.0.183:5000/coredns" calicoctl_image_repo: "10.0.0.183:5000/calico/" Ctl "calico_node_image_repo:" 10.0.0.183:5000/calico/node "calico_cni_image_repo:" 10.0.0.183:5000/calico/cni "calico_policy_image_repo:" 10.0.0.183:5000/calico/kube-controllers "hyperkube_image_repo:" {{kube_image_repo}} / hyperkube- {{image_arch}} "pod_infra_image_repo:" {{kube_image_ Repo}} / pause- {{image_arch}} "dnsautoscaler_image_repo:" {{kube_image_repo}} / cluster-proportional-autoscaler- {{image_arch}} "dashboard_image_repo:" {{kube_image_repo}} / kubernetes-dashboard- {{image_arch}} "

Since the https certificate is not configured in our private image repository, the following configuration needs to be added to the inventory/testcluster/group_vars/all/docker.yml file:

Docker_insecure_registries:- 10.0.0.183Docker 50003.2 Docker installation source changes and perform file preprocessing 3.2.1 Docker installation source changes

Since docker is installed from the official Docker source by default, which is very slow, here we replace it with a domestic Ali source and add the following configuration to the inventory/testcluster/group_vars/k8s-cluster/k8s-cluster.yml file:

# CentOS/RedHat docker-ce repodocker_rh_repo_base_url:' docker_rh_repo_gpgkey: 'dockerproject_rh_repo_base_url:' dockerproject_rh_repo_gpgkey:' https://mirrors.aliyun.com/docker-engine/yum/gpg'

3.2.2 executable file preprocessing

In addition, due to the need to download some executable files from google and github, we cannot download them directly on the server due to firewall. We can download these executable files in advance and upload them to the specified server path.

The executable download address can be found in the roles/download/defaults/main.yml file, and the download path is as follows:

Kubeadm_download_url: "https://storage.googleapis.com/kubernetes-release/release/v1.12.5/bin/linux/amd64/kubeadm"hyperkube_download_url:" https://storage.googleapis.com/kubernetes-release/release/v1.12.5/bin/linux/amd64/hyperkube"cni_download_url: "https://github.com/containernetworking/plugins/releases/download/v0.6.0/cni-plugins-amd64-v0.6.0.tgz""

Next, modify the file permissions and upload them to the / tmp/releases directory of each server

Chmod 755 cni-plugins-amd64-v0.6.0.tgz hyperkube kubeadmscp cni-plugins-amd64-v0.6.0.tgz hyperkube kubeadm root@node1:/tmp/releases

3.3 component list

Components required for K8s

List of optional plug-ins

3.4 DNS scheme

K8s service discovery relies on DNS and involves two types of networks: host network and container network, so Kubespray provides two configurations for management

3.4.1 dns_mode

Dns_mode is mainly used for domain name resolution in the cluster. There are the following types. Our technical selection is coredns. Note: if you select a certain dns_mode, you may need to download and install multiple container images, and the version of the image may be different.

3.4.2 resolvconf_mode

Resolvconf_mode is mainly used to solve how to use k8s dns when the container is deployed in host network mode. Here we use docker_dns.

Resolvconf_mode: docker_dns3.5 Network plug-in Select 3.5.1 kube-proxy

Kube-proxy can choose ipvs or iptables. Here we choose ipvs mode. For the difference between the two, please refer to Huawei Cloud's Service performance optimization practice (https://zhuanlan.zhihu.com/p/37230013)) in K8S large-scale scenarios.

3.5.2 list of network plug-ins

The list of network plug-ins is as follows. Our technology selection is calico. Note: selecting a network plug-in may require one or more container images, and the image version may be different.

3.6High availability solution step4: install and deploy # Install dependencies from ``environments.txt`sudo pip install-r requirements.txt# Copy `inventory/ sample`as `inventory/ mycluster`cp-rfp inventory/sample inventory/mycluster# Update Ansible inventory file with inventory builderdeclare-an IPS= (10.10.1.3 10.10.1.4 10.10.1.5) CONFIG_FILE=inventory/mycluster/hosts.ini python3 contrib/inventory_builder/inventory.py ${IPS [@]} # Review and change parameters under `inventory/mycluster/group_ vars`cat inventory/mycluster/group_vars/all/all.ymlcat inventory/mycluster/group_vars/k8s-cluster/k8s-cluster.yml# Deploy Kubespray with Ansible Playbook-run the playbook as root# The option `- b` is required As for example writing SSL keys in / etc/,# installing packages and interacting with various systemd daemons.# Without-b the playbook will fail to runaway ansibleplay book-I inventory/mycluster/hosts.ini-- become--become-user=root cluster.yml

After the deployment is complete, you can log in to the host where the k8s-master is located, execute the following command, and you can see that each component is normal.

Kubectl cluster-infokubectl get nodekubectl get pods-all-namespaces

Reference documentation:

Https://github.com/kubernetes-sigs/kubespray/blob/master/docs/getting-started.md

Https://xdatk.github.io/2018/04/16/kubespray2/

Https://jicki.me/kubernetes/docker/2018/12/21/k8s-1.13.1-kubespray/

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.