Shulou(Shulou.com)06/03 Report--

Write at the front

In the previous article, kubernetes series tutorials (8) Pod health check mechanism introduced the Pod health check mechanism in kubernetes, and introduced two kinds of health check probes in kubernetes through actual combat: livenessProbe survival check, readinessProbe readiness check, survival check is used to check the availability of applications, and readiness check is used to check whether the container is ready to accept traffic. Health check includes three detection methods: exec command line probe, tcpSocket port detection, and httpGet request detection. It is suitable for health examination in different scenarios. Next, we will introduce the storage management of the kubernetes series tutorials pod.

According to the course of development, kubernetes storage management involves Volume,PV (PersistentVolume) and PVC (PersistentVolumeClaims), and StorageClass,Volume is the first proposed storage volume, which mainly solves the dependency between container and data storage, and abstracts the underlying drivers to support different storage types. Using Volume needs to know the details of the underlying storage, so it is proposed that PV,Persistent Volume is a storage unit defined by the K8s administrator, and the application uses PersistentVolumeClaims declaration to call PV storage, further abstracting the underlying storage; with the increase in the number of PV, administrators need to constantly define the number of PV, derived from the dynamic generation of PV,StorageClass through StorageClass and the storage capacity declared in PVC, which will call the underlying provider to generate PV. This article introduces the use of Volume, and the next article introduces PV,PVC and StorageClass.

Volume storage volumes are independent of containers, and the backend and different storage drivers interface with PV PersistentVolume persistent storage volumes. Similar to node, it is a cluster resource defined by administrators and docked with different storage PVC PersistentVolumeClaims persistent storage declarations. Similar to pod, as the consumer StorageClass dynamic storage type of PV, it is divided into static and dynamic types. By defining the storage type in PVC, the required PV1 is created automatically. Kubernetes Storage Management 1.1 Storage Overview

The data in the kubernetes container is temporary, that is, the container data will be lost after restart or crash. In addition, there is a need for shared storage among containers, so the abstraction of volume storage is provided in kubernetes. The volume backend can support a variety of different plugin drivers, define a storage through .spec.volumes, then call it in the container .spec.containers.volumeMounts, and finally present it as a directory inside the container.

Kubernetes built-in can support a variety of different driver types, generally can be divided into four types: 1. Public / private cloud driver interfaces, such as awsElasticBlockStore implementation and aws EBS integration, 2. Open source storage driver interfaces, such as ceph rbd, implement docking with ceph rb block storage, 3. Local temporary storage, such as hostPath,4. Kubernetes object API driver interface, implements other object calls, such as configmap. Each storage supports different drivers, as described below:

Public / private cloud driver interface awsElasticBlockStore AWS EBS cloud disk azureDisk Microsoft azure cloud disk azureFile Microsoft NAS storage gcePersistentDisk google cloud disk cinder openstack cinder cloud disk vsphereVolume VMware VMFS storage scaleIO EMC distributed storage open source storage driver interface ceph rbd ceph block storage cephfs ceph file storage nfs nfs file iscsiflexvolumecsi community standardized drive flocker local temporary storage hostpath host file emptyDir temporary directory kubernetes object API driver interface configMap calls configmap object, injection configuration file secrets calls secrets object Injecting ciphertext configuration file persistentVolumeClaim through pvc call storage downloadAPI download URLprojected1.2 emptyDir temporary storage

EmptyDir is a kind of temporary storage. When pod is created, it will apply for a temporary directory for the container on the node node. With the lifecycle of the container, such as container deletion, the temporary storage space defined by emptyDir will also be deleted. If the container is unexpected, crash will not be affected. At the same time, if the container is migrated, the data on it will be lost. EmptyDir is generally used for testing or caching scenarios.

Define an emptyDir storage size of 1G Mount it to the / data directory of redis [root@node-1 happylau] # cat emptydir-redis.yamlapiVersion: v1kind: Podmetadata: name: emptydir-redis labels: volume: emptydir annotations: kubernetes.io/storage: emptyDirspec: containers:-name: emptydir-redis image: redis:latest imagePullPolicy: IfNotPresent ports:-name: redis-6379-port protocol: TCP containerPort: 6379 volumeMounts: # Mount the defined driver emptydir-redis to the container's / data directory Define a storage by name association-name: emptydir-redis mountPath: / data volumes: # and the driver type is emptyDir Size 1G-name: emptydir-redis emptyDir: sizeLimit: 1Gi generates redis pod And view the details of describe pod [root@node-1 happylau] # kubectl apply-f emptydir-redis.yaml pod/emptydir-redis created execute kubectl describe pods emptydir-redis to view the storage mount information of the container Containers: emptydir-redis: Container ID: docker://dddd9f3d0e395d784c08b712631d2b0c259bfdb30b0c655a0fc8021492f1ecf9 Image: redis:latest Image ID: docker-pullable://redis@sha256:cb379e1a076fcd3d3f09e10d7b47ca631fb98fb33149ab559fa02c1b11436345 Port: 6379/TCP Host Port: 0 / TCP State: Running Started: Tue 01 Oct 2019 11:04:30 + 0800 Ready: True Restart Count: 0 Environment: Mounts: # Mount Information Mount emptydir-redis to the / data directory, and the rw read and write status / data from emptydir-redis (rw) / var/run/secrets/kubernetes.io/serviceaccount from default-token-5qwmc (ro) Conditions: Type Status Initialized True Ready True ContainersReady True PodScheduled True Volumes: # defines a storage of type EmptyDir The size is 1Gi emptydir-redis: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: 1Gi default-token-5qwmc: Type: Secret (a volume populated by a Secret) SecretName: default-token-5qwmc Optional: false writes data to redis to obtain the ip address of pod [root@node-1 happylau] # kubectl get pods emptydir-redis-o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESemptydir-redis 1 Running 1 17m 10.244.1.27 node-2 installation client redis-cli [root@node-1 ~] # yum install redis writes two key10.244.1.27:6379 > set volume emptydirOK10.244.1.27:6379 > set username happylauliuOK10.244.1.27:6379 > get volume "emptydir" 10 to redis. 244.1.27 get username 6379 > get username "happylauliu" login to pod to install a tool to view the process procps The process is generally 1. As shown in the figure below, the redis-server process can be directly kill. After the process is kill, kubelet will automatically restart the process to login container [root@node-1 ~] # kubectl exec-it emptydir-redis / bin/bash to install the software root@emptydir-redis:/data# apt-get update. Apt-get install procps can view the process through top. The process number is usually 1root@emptydir-redis:/data# kill 1.

After an abnormal restart of pod, log in to redis again and look at the data content in redis and find that the data has not been lost. [root@node-1 ~] # redis-cli-h 10.244.1.2710.244.1.27 get volume "emptydir" 10.244.1.27 get username "happylauliu" emptyDir is actually a directory created on the host, mounted to the container in the form of bind mount, and followed the container's life cycle [root@node-2 ~] # docker container list | grep redise0e9a6b0ed77 01a52b3b5cd1 "docker-entrypoint.s..." 20 minutes ago Up 20 minutes k8s_emptydir-redis_emptydir-redis_default_4baadb25-1e62-4cf5-9724-821d04dcdd44_2dfef32905fe5 k8s.gcr.io/pause:3.1 "/ pause" 45 minutes ago Up 45 minutes k8s_POD_emptydir-redis_default_4baadb25-1e62-4cf5-9724-821d04dcdd44_0

Docker container inspect e0e9a6b0ed77 views the storage content as shown below:

View the information of the directory:

[root@node-2] # ls-l / var/lib/kubelet/pods/4baadb25-1e62-4cf5-9724-821d04dcdd44/volumes/kubernetes.io~empty-dir/emptydir-redis Total consumption 4 RW root@node-2 Ruki-1 polkitd input 8 October 8 14:55 dump.rdbPod deleted Volume information is also deleted [root@node-1 ~] # kubectl delete pods emptydir-redis pod "emptydir-redis" deleted [root @ node-1 ~] # ssh node-2Last login: Tue Oct 8 15:15:41 2019 from 10.254.100.101 [root @ node-2 ~] # ls-l / var/lib/kubelet/pods/4baadb25-1e62-4cf5-9724-821d04dcdd44/volumes/kubernetes.io~empty-dir/emptydir-redisls: unable to access / var/lib/kubelet/pods/ 4baadb25-1e62-4cf5-9724-821d04dcdd44/volumes/kubernetes.io~empty-dir/emptydir-redis: there is no such file or directory

Summary: emptyDir is a temporary storage defined on host. It is mounted to the container in the form of bind mount. The container restart data will be retained, and the volume will be deleted if the container is deleted.

1.3 hostPath host storage

Similar to emptyDir, hostpath supports mounting directories or files of node nodes into containers for stand-alone testing scenarios. It is also suitable for scenarios where some container businesses need to access host directories, such as monitoring system access / proc and / sys directories, and log system access / var/lib/docker directories. Support setting different type types

Directory local directory DirectoryOrCreate directory, if does not exist, permissions set to 755, owner and group settings are consistent with kubelet File files exist locally, FileOrCreate files are created if they do not exist, permissions are set to 644 Owner and group settings are consistent with kubelet Socket local existence Socket file CharDevice local Char character device BlockDevice local existing Block block device mounts the local / mnt directory to the container [root@node-1 happylau] # cat hostpath-demo.yamlapiVersion: v1kind: Podmetadata: name: hostpath-demo labels: storage: hostpath annotations: kubernetes.io/storage: hostpathspec: containers:-name: nginx image: nginx:latest imagePullPolicy: IfNotPresent ports: -name: nginx-http-port protocol: TCP containerPort: 80 volumeMounts: # Mount to nginx's web site directory-name: hostpath-demo mountPath: / usr/share/nginx/html volumes: # order a hostPath local storage-name: hostpath-demo hostPath: type: DirectoryOrCreate path: / mnt/data generate nginx container and web site data [root@node-1 happylau] # kubectl apply-f hostpath -demo.yaml pod/hostpath-demo created gets the data of the node node where the pod is located [root@node-1 happylau] # kubectl get pods hostpath-demo-o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATEShostpath-demo 1 Plus 1 Running 0 31s 10.244.2.24 node-3 generates web site data [root@node-1 happylau] # ssh node- 3Last login: Tue Oct 8 22:49:14 2019 from 10.254.100.101 [root @ node-3 ~] # echo "hostPath test page" > / mnt/data/index.html [root@node-3 ~] # curl http://10.244.2.24hostPath test page to see how storage is mounted in the container Mount to a container in the form of bind mount

Simulate the failure of container restart. The data in volume after container restart is retained according to # docker layer kill process [root@node-3 ~] # docker container list | grep hostpath39a7e21afebb f949e7d76d63 "nginx-g 'daemon of …" 11 minutes ago Up 11 minutes k8s_nginx_hostpath-demo_default_6da41e3d-8585-4997-bf90-255ca0948030_0490f50108e41 k8s.gcr.io/pause:3.1 "/ pause" 11 minutes ago Up 11 minutes k8s_POD_hostpath-demo_default_6da41e3d-8585-4997-bf90-255ca0948030_0 [root@node- 3 ~] # docker container kill 39a7e21afebb39a7e21afebb39a7e21afebb [root @ node-3 ~] # exit log out # to get the address of pod According to RESTART, the container has been restarted once. The test data still retains [root@node-1 happylau] # kubectl get pods-o wide hostpath-demo NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATEShostpath-demo 1 wide hostpath-demo NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATEShostpath-demo 1 Running 1 12m 10.244.2.24 node-3 [root@node-1 happylau] # curl http://10.244.2.24hostPath test page

Summary: hostPath is similar to emptyDir to provide temporary storage. HostPath is suitable for scenarios where containers need to access host directories or files, but it is not a good solution for data persistence.

1.4 NFS storage docking

NFS is the NAS storage to achieve Network File System network file sharing, kubernetes and NFS docking to achieve storage sharing, when the container deletion does not affect storage and can achieve cross-machine storage sharing, this paper uses to build a NFS storage to achieve kubernetes docking.

Prepare a nfs server share Install the nfs service [root@node-1 ~] # yum install nfs-utils-y from the / mnt/data directory share of node-1 to configure the nfs share Create a directory in advance [root@node-1 ~] # cat / etc/exports/mnt/data 10.254.100.0 take 24 (rw) restart and verify that [root@node-1 ~] # systemctl restart NFS [root @ node-1 ~] # showmount-e node-1Export list for node-1:/mnt/data 10.254.100.0/24kubernets uses nfs driver docking [root@node-1 happylau] # cat nfs-demo.yaml apiVersion: v1kind: Podmetadata: name : nfs-demo labels: storage: nfs annotations: kubernetes.io/storage: nfsspec: containers:-name: nginx image: nginx:latest imagePullPolicy: IfNotPresent ports:-name: nginx-http-port protocol: TCP containerPort: 80 volumeMounts: # Mount to nfs directory-name: nfs-demo mountPath: / usr/share/nginx/html volumes: # define a nfs-driven storage-name: nfs -demo nfs: server: 10.254.100.101 path: / mnt/data generate pod Prompt the error message in events when using kubectl get pods Failed to mount Events: Type Reason Age From Message-Normal Scheduled 40s default-scheduler Successfully assigned default/nfs-demo to node-2 Warning FailedMount 39s kubelet Node-2 MountVolume.SetUp failed for volume "nfs-demo": mount failed: exit status 32Mounting command: systemd-runMounting arguments:-- description=Kubernetes transient mount for / var/lib/kubelet/pods/78bf6a81-082d-4d6c-a163-75241bf21cde/volumes/kubernetes.io~nfs/nfs-demo-- scope-- mount-t nfs 10.254.100.101:/mnt/data / var/lib/kubelet/pods/78bf6a81-082d-4d6c-a163-75241bf21cde/volumes/kubernetes.io~nfs/nfs-demoOutput: Running scope as unit run-29843.scope.mount: wrong fs type Bad option, bad superblock on 10.254.100.101:/mnt/data, missing codepage or helper program, or other error (for several filesystems (e.g. Nfs, cifs) you might need a / sbin/mount. Helper program) learned from the above steps that when the host mounts nfs, it prompts that there is no mount.nfs command, so you need to install nfs-utils, the client software of nfs, on all node nodes. Take node-2 as an example. Other nodes are similar to [root@node-1 happylau] # ssh node-2Last login: Tue Oct 8 15:22:04 2019 from 10.254.100.101 [root @ node-2 ~] # yum install nfs-utils-y test site data [root@node-1 happylau] # kubectl get pods nfs-demo-o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESnfs-demo 1 Running 0 4m41s 10.244.1.28 node-2 [root@node-1 happylau] # echo "nfs test age" > / mnt/data/index.html [root@node-1 happylau] # curl http://10.244.1.28nfs test age View the data shared by nfs after deleting pod The original data still retains [root@node-1 happylau] # kubectl delete pods nfs-demo pod "nfs-demo" deleted [root @ node-1 happylau] # mount.nfs node-1:/mnt/data/ / media/ [root @ node-1 happylau] # ls-l / media/ Total usage 4muri RWMurray Rafe-1 root root 13 October 8 23:26 index.html1.5 TKE use volume storage

TKE supports specified storage volumes such as Deployments,DaemonSets,StatefulSets when creating Workload, and supports temporary directory emptyDir, host path hostPath, NFS disk, pvc, cloud disk, configmap,secrets. Here, take Tencent Cloud CFS as an example (create storage in CFS in advance to ensure that CFS and container host are in the same VPC network).

Create a storage volume and mount Tencent Cloud CFS storage using NFS

Storage is used in Pod, and the storage volume is called by volume-nfs-demo name

The contents of the generated yaml file are as follows: apiVersion: apps/v1beta2kind: Deploymentmetadata: annotations: deployment.kubernetes.io/revision: "1" description: demo creationTimestamp: "2019-10-08T15:45:18Z" generation: 1 labels: k8s-app: the-volume-demo qcloud-app: the-volume-demo name: the-volume-demo namespace: default resourceVersion: "618380753" selfLink: / apis/apps/v1beta2/namespaces/default/deployments/the-volume- Demo uid: a0fc4600-e9e2-11e9-b3f4-decf0ef369cfspec: minReadySeconds: 10 progressDeadlineSeconds: 600 replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: k8s-app: the-volume-demo qcloud-app: the-volume-demo strategy: rollingUpdate: maxSurge: 1 maxUnavailable: 0 type: RollingUpdate template: metadata: creationTimestamp: null labels: k8s-app: the-volume-demo qcloud-app: the-volume-demo spec: Containers:-image: nginx:latest imagePullPolicy: Always name: nginx-demo resources: limits: cpu: 500m memory: 1Gi requests: cpu: 250m memory: 256Mi securityContext: privileged: false terminationMessagePath: / dev/termination-log terminationMessagePolicy: File volumeMounts: # Mount To pod-mountPath: / usr/share/nginx/html name: volume-nfs-demo dnsPolicy: ClusterFirst imagePullSecrets:-name: qcloudregistrykey-name: tencenthubkey restartPolicy: Always schedulerName: default-scheduler securityContext: {} terminationGracePeriodSeconds: 30 volumes: # CFS storage-name: volume-nfs-demo nfs: path: / server: 10.66.200.7 write at the end

This paper introduces the use of the most basic volume in kubernetes storage, introduces that volume supports many different drivers, introduces the docking of emptyDir,hostPath,nfs drivers with practical cases, and introduces the use of volume function under TKE. Because volume needs to know the details of the underlying storage, it is not easy to be widely used. Later, it is derived to PV. The administrator defines the PV implementation to interface with the underlying storage, and users use PV through PVC. In the next section, we will introduce the use of PV/PVC and StorageClass.

reference

Volume Management: https://kubernetes.io/docs/concepts/storage/volumes/

Using volume: https://kubernetes.io/docs/tasks/configure-pod-container/configure-volume-storage/ in pod

When your talent can't support your ambition, you should calm down and study.

Return to the kubernetes series tutorial directory

* * if you think the article is helpful to you, please subscribe to the column and share it with friends in need.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.