Volume of Kubernetes storage 01/16 Update SLTechnology News&Howtos

Volume of Kubernetes storage

2025-01-16 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Servers >

Shulou(Shulou.com)06/02 Report--

In a series of virtualization solutions, data persistence is an issue of great concern to us, as is the case with dokcer, and so is Kubernetes. In Kubernetes, however, there is a concept of data volumes.

A brief introduction to Volume

We often say: containers and Pod are very short-lived! The implication is that both the container and the Pod have a short life cycle and are frequently destroyed and created. When the container is destroyed, the data stored in the internal file system of the container will be erased.

The life cycle of Volume is independent of containers. Containers in Pod may be destroyed and restarted, but Volume will be retained.

Kubernetes Volume mainly solves the following two problems:

1) data persistence: usually, after the container is running, the files written to its file system are temporary. When the container crashes, kubelet will constantly restart the container. When the number of restarts is reached, the container is still unavailable, then the container will be kill off and a new container will be rebuilt. At this point, the newly running container does not have the data in the original container, because the container is created by the image

2) data sharing: between containers running in the same Pod, there is often a need to share files / shared folders

Basically, a data volume is just a directory or file that can be accessed by Pod. How this directory comes from depends on the type of data volume. Two containers in the same Pod can mount a data volume to different directories.

Volume provides an abstraction of various backend. When using Volume to read and write data, the container does not need to care whether the data is stored in the file system of the local node or on the cloud disk. To it, all types of Volume are just a directory.

II. Brief introduction to emptyDir1 of Volume) emptyDir

EmptyDir is the most basic Volume type. As its name suggests, an emptyDir Volume is an empty directory on Host.

EmptyDir Volume is persistent for containers, but not for Pod. When the Pod is deleted from the node, the contents of the Volume are also deleted. However, if only the container is destroyed and the Pod is still there, the Volume is not affected. Similar to the docker manager volume approach in docker data persistence!

2) emptyDir uses the example [root@master yaml] # vim emptyDir.yaml apiVersion: v1kind: Podmetadata: name: producer-consumer # to define the name of the Pod spec: containers:-image: busybox name: producer # define the name of the container volumeMounts:-mountPath: / producer_dir # specify the path within the container name: shared-volume # indicates that the Shared-volume is mounted to the container args: # when the container is running Perform the following write operation-/ bin/sh-- c-echo "hello k8s" > / producer_dir/hello Sleep 30000-image: busybox name: consumer # defines the name of the container volumeMounts:-mountPath: / consumer_dir name: shared-volume # same as the previous container args:-/ bin/sh-- c-cat / consumer_dir/hello Sleep 30000 volumes:-name: shared-volume # defines the name of the data volume, which must be consistent with the name of the data volume mounted above emptyDir: {} # define a data volume of type emptyDir The name is shared-volume [root@master yaml] # kubectl apply-f emptyDir.yam # to generate the required Pod resources [root@master yaml] # kubectl exec-it producer-consumer-c producer / bin/sh# enter the first container for authentication / # cat / producer_dir/hello hello k8s [root@master yaml] # kubectl exec-it producer-consumer-c consumer / bin/sh# for second container authentication / # cat / consumer_dir/hello hello k8s

You can see that the contents of the directories specified by the two containers in this pod are the same, and the local directory needs to be further verified.

[root@master yaml] # kubectl get pod-o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESproducer-consumer 2bind 2 Running 0 7m58s 10.244.2.2 node02 # you can see that this pod is a [root@node02 ~] # docker ps running on node02 | grep busybox # because there are many containers Filter 4fbd734e1763 busybox "/ bin/sh-c 'cat / co based on the image name used." 8 minutes ago Up 8 minutes k8s_consumer_producer-consumer_default_003a002d-caec-4202-a020-1ae8d6ff7eba_0b441c2ff2217 busybox "/ bin/sh-c 'echo\" h … " 8 minutes ago Up 8 minutes k8s_producer_producer-consumer_default_003a002d-caec-4202-a020-1ae8d6ff7eba_0 [root@node02 ~] # docker inspect 4fbd734e1763 # View the details of the container based on its ID # find the Mounts field As follows: "Mounts": [{"Type": "bind", "Source": "/ var/lib/kubelet/pods/003a002d-caec-4202-a020-1ae8d6ff7eba/volumes/kubernetes.io~empty-dir/shared-volume" # what is specified here is the docker host local directory "Destination": "/ consumer_dir", # container directory "Mode": "", "RW": true, "Propagation": "rprivate"} [root@node02 ~] # docker inspect b441c2ff2217 "Mounts": [{"Type": "bind", "Source": "/ var/lib/kubelet/pods/003a002d-caec-4202-a020-1ae8d6ff7eba/volumes/kubernetes.io~empty-dir/shared-volume", "Destination": "/ producer_dir", "Mode": "" "RW": true, "Propagation": "rprivate"}, # you can see that the source directories of the two containers are the same What is mounted is the same directory local to docker host [root@node02 ~] # cd / var/lib/kubelet/pods/003a002d-caec-4202-a020-1ae8d6ff7eba/volumes/kubernetes.io~empty-dir/shared-volume [root@node02 shared-volume] # cat hello hello k8s# verification content

Because it is a Kubernetes cluster environment, it is troublesome to delete a container. Delete the pod directly to see if the local data in docker host exists!

[root@master yaml] # kubectl delete-f emptyDir.yaml#master node deletes pod [root@node02 ~] # cd / var/lib/kubelet/pods/003a002d-caec-4202-a020-1ae8d6ff7eba/volumes/kubernetes.io~empty-dir/shared-volume-bash: cd: / var/lib/kubelet/pods/003a002d-caec-4202-a020-1ae8d6ff7eba/volumes/kubernetes.io~empty-dir/shared-volume: there is no file or directory # node02 to verify and found that the directory has disappeared 3) emptyDir summary

EmptyDir is a temporary directory created on Docker Host, and its advantage is that it can easily provide shared storage for containers in Pod without additional configuration. But it doesn't have persistence, and if Pod doesn't exist, emptyDir will be gone. Based on this feature, emptyDir is particularly suitable for scenarios where containers in Pod need temporary shared storage space!

Simply put, if the container is deleted, the data still exists; if the Pod is deleted, the data will not exist!

III. HostPath of Volume

The role of hostPath Volume is to mount a directory that already exists in the Docker Host file system to the container of Pod. Most applications will not use hostPath Volume because it actually increases the coupling between Pod and nodes and limits the use of Pod. However, applications that need access to Kubernetes or Docker internal data (configuration files and binary libraries) need to use hostPath. Similar to the bind mount approach in docker data persistence!

Of course, you can also create it. Just be lazy and use the YAML file that comes with the Kubernetes cluster for introduction!

[root@master yaml] # kubectl edit-- namespace=kube-system pod kube-apiserver-master # View the yaml file of the apiserver component

As shown in the figure:

If the Pod is destroyed, the directory corresponding to the hostPath will also be retained, so from this point of view, hostPath is more persistent than emptyDir. But once Host crashes, hostPath becomes inaccessible.

Due to the lack of use scenarios, the above two ways will not be described in detail here!

IV. Persistent Volume of Volume

Overview of Persistent Volume

There is a static binding relationship between the normal Volume and the Pod that uses it. In the file that defines the Pod, the Volume it uses is also defined. Volume is an accessory to Pod, and we cannot create a separate Volume because it is not a separate K8S resource object.

Persistent Volume for short PV is a K8S resource object, so we can create a separate PV. It does not have a direct relationship with Pod, but through Persistent Volume Claim, referred to as PVC to achieve dynamic binding. PVC is specified in the Pod definition, and then PVC will automatically bind the appropriate PV to Pod for use according to the requirements of Pod.

Now that we have the concept of PV, then the concept of PVC (PersistentVolumeClaim) also has to say that PVC represents a user's request for storage and an application for PV persistence space. K8s clusters may have multiple PV, and you need to constantly create multiple PV for different applications.

As shown in the figure:

1) PV is a storage resource in a cluster, which is usually created and managed by the cluster administrator

2) StorageClass is used to classify PV. If configured correctly, Storage can also dynamically create PV according to the request of PVC.

3) PVC is a request to use this resource, which is usually made by the application and specifies the corresponding StorageClass and the space size of the requirement

4) as a kind of data volume, PVC can be mounted and used in Pod.

The management process of PV and PVC is as follows: 1) divide a separate directory on the host for PV use and define its available size; 2) create PVC as a resource object to facilitate the application of PV storage space; 3) add data volumes to Pod, and data volumes are associated with PVC;4) Pod contains containers, and containers mount data volumes

Let's learn more about Persistent Volume through a case study.

Case realization process:

1) the underlying layer is stored in NFS, and then 1G of capacity is divided under the directory of NFS for PV scheduling.

2) create a PVC to apply for storage space for PV

3) create a Pod and use the storage space requested by PVC to persist the data

1) build NFS storage

This case creates NFS storage directly on the master node!

The explanation of the main fields in the above yaml file:

1) accessModes (access mode)

ReadWriteOnce: Mount to a single node read-write; ReadWriteMany: mount to multiple nodes read-write; ReadOnlyMany: mount to multiple nodes read-only

2) persistentVolumeReclaimPolicy (recycling strategy of PV)

Recycle: automatically clears the data in PV and automatically reclaims; Retain: manual recycling is required; Delete: delete cloud storage resources (cloud storage only) [root@master ~] # kubectl apply-f test-pv.yaml [root@master ~] # kubectl get pv # View the status of PV NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGEtest-pv 1Gi RWO Recycle Available nfs 21s# Note that the status of its PV must be Available before it can be used normally 3) create a PVC resource pair For example, [root@master ~] # vim test-pvc.yaml apiVersion: v1kind: PersistentVolumeClaimmetadata: name: test-pvcspec: accessModes: # define access mode Must be consistent with the access mode defined by PV-ReadWriteOnce resources: requests: storage: 1Gi # Direct request I use maximum capacity storageClassName: nfs # the name defined must be consistent with the name defined by PV [root@master ~] # kubectl apply-f test-pvc.yaml [root@master ~] # kubectl get pvc # View the status NAME STATUS VOLUME CAPACITY of PVC ACCESS MODES STORAGECLASS AGEtest-pvc Bound test-pv 1Gi RWO nfs 102s [root@master ~] # kubectl get pv # check the status of PV NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGEtest-pv 1Gi RWO Recycle Bound default/test-pvc nfs 14m# note that the status of both PV and PVC is Bound Indicates that the association between PV and PVC is successful

If there are many similar PV,PVC in K8s cluster, when applying for space from PV, they will consider not only the name and access control mode, but also the size of your application space, which will be allocated to the PV of the most appropriate size!

Common states are:

1) Available-- > idle state, not bound to PVC

2) Bound-- > bind to PVC

3) Released-- > PVC is deleted and the resource is not utilized

4) Failed-- > automatic collection failed

4) create a Pod [root@master ~] # vim test-pod.yamlapiVersion: v1kind: Podmetadata: name: test-podspec: containers:-name: test-pod image: busybox args:-/ bin/sh-- c-sleep 300000 volumeMounts:-mountPath: / testdata # define the directory in the container name: volumedata # make sure it matches the name of the volume volumes: -name: volumedata # define the name of the volume persistentVolumeClaim: claimName: test-pvc # specify the PVC name corresponding to the logical volume [root@master ~] # kubectl apply-f test-pod.yaml [root@master ~] # kubectl get pod # View the status of pod NAME READY STATUS RESTARTS AGEtest-pod 0ram 1 ContainerCreating 0 6m26s# notice that its status is ContainerCreating Indicates that the container is being created, but it is not normal to view it for such a long time.

When the pod status is abnormal, we can generally use the following three ways to troubleshoot:

1) use "kubectl describe pod pod name" to view the details of pod

2) use "kubectl logs pod name" to view the log information of pod

3) use "cat / var/log/messages | grep kubelet" to view the kubelet system log of the corresponding node

This time, we use the first way to troubleshoot!

The information in the last piece of [root@master ~] # kubectl describe pod test-pod # is as follows: mount.nfs: mounting 192.168.1.4:/nfsdata/test-pv failed, reason given by server: No such file or directory# according to the message prompt, the specified local directory to be mounted does not exist [root@master ~] # mkdir-p / nfsdata/test-pv#. After the creation of the directory, it is recommended to check the nodes running by the container generated by pod. Restart the kubelet service on the node. After the restart is complete, check the status of Pod again [root@master ~] # kubectl get pod / / check the status of pod again NAME READY STATUS RESTARTS AGEtest-pod 1 play 1 Running 0 32m

If Pod keeps restarting during the process of creating Pod, it is mainly because:

1) the swap swap partition is not closed, which results in abnormal operation of the cluster.

2) the memory of the node node is insufficient, and the Daozhou cluster is not running properly.

Both of the above situations will cause Pod to restart constantly, which can be viewed using the "free-h" command!

5) Test the effect of data persistence [root@master] # kubectl exec-it test-pod / bin/sh/ # echo "test pv pvc" > / testdata/test.txt# into the container Create a file for testing [root@master ~] # cat / nfsdata/test-pv/test.txt test pv pvc# confirm that the file exists locally [root@master ~] # kubectl delete-f test-pod.yaml # delete pod [root@master ~] # cat / nfsdata/test-pv/test.txt test pv pvc# again and find that the test file of pod still exists [root@master ~] # kubectl delete-f test-pvc.yaml # Delete PVC [root@master ~] # cat / nfsdata/test-pv/tescat: / nfsdata/test-pv/tes: there is no such file or directory # check again and find that the test file for pod is missing Because the summary of PVC is deleted: because when we create the resource object pv, the recycling strategy is to clear the data in PV and then collect it automatically, while the resource object PV is applied for by PVC, so whether it is the container or pod, their destruction will not affect the data under the local directory of nfs used for data persistence, but once the PVC is deleted Then the local data will no longer exist with the destruction of PVC, that is to say, data persistence is achieved by using data volumes like PV, and the life cycle of data persistence is consistent with the life cycle of PVC. 5. Application of mysql to data persistence

You may not really understand the role of Persistent volume through step 4, so let's verify the role of Persistent volume by creating a mysql container!

1) build NFS shared storage

This case creates NFS storage directly on the master node!

[root@master ~] # yum-y install nfs-utils rpcbind [root@master ~] # vim / etc/exports/nfsdata * (rw,sync No_root_squash) [root@master ~] # systemctl start nfs-server [root@master ~] # systemctl start rpcbind [root@master ~] # showmount-eExport list for master:/nfsdata * 2) create a PV resource object [root@master ~] # vim mysql-pv.yaml apiVersion: v1kind: PersistentVolumemetadata: name: mysql-pvspec: capacity: storage: 1Gi accessModes:-ReadWriteOnce persistentVolumeReclaimPolicy: Retain # Note that the specified recovery policy is manual Receive storageClassName: nfs nfs: path: / nfsdata/mysql-pv server: 192.168.1.4 [root@master] # kubectl apply-f mysql-pv.yaml [root@master] # kubectl get pvNAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGEtest-pv 1Gi RWO Retain Available nfs 15s [root@master] # mkdir- P / nfsdata/mysql-pv 3) create PVC resource object [root@master ~] # vim mysql-pvc.yaml apiVersion: v1kind: PersistentVolumeClaimmetadata: name: mysql-pvcspec: accessModes:-ReadWriteOnce resources: requests: storage: 1Gi storageClassName: nfs [root@master ~] # kubectl apply-f mysql-pvc.yaml [root@master ~] # kubectl get pvcNAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGEmysql-pvc Bound mysql-pv 1Gi RWO Nfs 13s [root@master ~] # kubectl get pvNAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGEmysql-pv 1Gi RWO Retain Bound default/mysql-pvc nfs 8m14s4) create a pod resource [root@master ~] # vim mysql-pod.yamlapiVersion: extensions/v1beta1kind: Deploymentmetadata: name: mysql-podspec: Selector: # set the tag selector that gives the equivalent matchLabels: app: mysql template: metadata: labels: app: mysql spec: containers:-image: mysql:5.6 name: mysql env: # set the environment variable Password of the database root user-name: MYSQL_ROOT_PASSWORD value: 123.com volumeMounts:-name: mysql-storage mountPath: / var/lib/mysql # this directory is the directory where the database stores the data (specified is the directory in the container) volumes:-name: mysql-storage persistentVolumeClaim: claimName: mysql -pvc [root@master ~] # kubectl apply-f mysql-pod.yaml [root@master ~] # kubectl get podNAME READY STATUS RESTARTS AGEmysql-pod-6cc889468b-gq4qz 1 Running 1 Running 0 3s5) Test data persistence [root@master ~] # kubectl exec-it mysql-pod-6cc889468b-gq4qz-- mysql- u root-p123.com# directly log in to the pod running mysql database Mysql# inserts data to test mysql > create database lzj Mysql > use lzj;mysql > create table my_id (id int (4)); mysql > insert my_id values (9527); mysql > select * from my_id +-+ | id | +-+ | 9527 | +-+ [root@master ~] # ls / nfsdata/mysql-pv/auto.cnf ibdata1 ib_logfile0 ib_logfile1 lzj mysql performance_schema# View the directory of NFS corresponding to pod There is indeed data [root@master ~] # kubectl get pod- o wide # View pod details NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESmysql-pod-6cc889468b-gq4qz 1 Running 0 23m 10.244.2.6 node02 # View pod is running on the node02 node # Simulation node02 downtime, step omitted Turn it off or hang it up! [root@node01 ~] # systemctl restart kubelet# restarts node01's kubelet service # then wait patiently for the transfer of pod It may take about 5 minutes for ^ C [root@master] # kubectl get pod- o wide # to see that pod has been running on node01 NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESmysql-pod-6cc889468b-gdf7k 1 Running 0 66s 10.244.1. 6 node01 mysql-pod-6cc889468b-gq4qz 1 Terminating 0 32m 10.244.2.6 node02 [root@master ~] # kubectl exec-it mysql-pod-6cc889468b-gdf7k-- mysql- u root-p123.com# login again to mysql running in pod (note: name of pod) mysql > select * from lzj.my_id # whether the data exists again +-+ | id | +-+ | 9527 | +-+ 1 row in set (sec) # the data still exists

-this is the end of this time, thank you for reading-

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.

*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.