Shulou(Shulou.com)05/31 Report--

This article will explain in detail how to use LXCFS to improve the visibility of container resources in Kubernetes. The content of the article is of high quality, so the editor will share it with you for reference. I hope you will have some understanding of the relevant knowledge after reading this article.

The following describes how to solve the problem that legacy applications cannot recognize container resource limitations in Docker and Kubernetes environments.

Linuxs uses Cgroup to restrict the resources of the container, but the / proc directory of procfs on the host is still mounted by default inside the container, which contains resource information such as meminfo, cpuinfo,stat, uptime and so on. Some monitoring tools such as free/top or legacy applications also rely on the contents of the above files for resource configuration and usage. When they run in the container, they will read out the resource status of the host, causing errors and inconvenience.

Introduction to LXCFS

A common practice in the community is to use lxcfs to provide visibility of resources in containers. Lxcfs is an open source FUSE (user mode file system) implementation to support LXC containers, it can also support Docker containers.

LXCFS provides the following procfs files in the container through the user-mode file system.

/ proc/cpuinfo/proc/diskstats/proc/meminfo/proc/stat/proc/swaps/proc/uptime

The schematic diagram of LXCFS is as follows

For example, after the / var/lib/lxcfs/proc/memoinfo file of the host is mounted to the / proc/meminfo location of the Docker container. When the process in the container reads the contents of the corresponding file, the FUSE implementation of LXCFS reads the correct memory limit from the corresponding Cgroup of the container. Thus, the application can get the correct resource constraint setting.

Use of LXCFS in Docker environment

Note:

In this paper, CentOS 7.4 is used as the test environment, and FUSE module support has been enabled.

Due to the highly tailored operating system, Docker for Mac/Minikube and other development environments can not support FUSE, and run LXCFS for testing.

Install the RPM package for lxcfs

Wget https://copr-be.cloud.fedoraproject.org/results/ganto/lxd/epel-7-x86_64/00486278-lxcfs/lxcfs-2.0.5-3.el7.centos.x86_64.rpmyum install lxcfs-2.0.5-3.el7.centos.x86_64.rpm

Start lxcfs

Lxcfs / var/lib/lxcfs &

test

$docker run-it-m 256m\-v / var/lib/lxcfs/proc/cpuinfo:/proc/cpuinfo:rw\-v / var/lib/lxcfs/proc/diskstats:/proc/diskstats:rw\-v / var/lib/lxcfs/proc/meminfo:/proc/meminfo:rw\-v / var/lib/lxcfs/proc/stat:/proc/stat:rw\-v / var/lib/lxcfs/proc/swaps:/proc/swaps : rw\-v / var/lib/lxcfs/proc/uptime:/proc/uptime:rw\ ubuntu:16.04 / bin/bash root@f4a2a01e61cd:/# free total used free shared buff/cache availableMem: 262144 708 261436 2364 0 261436Swap: 262144

We can see that the memory of total is 256MB, and the configuration is in effect.

Kubernetes practice of lxcfs

Some students have asked how to use lxcfs in a Kubernetes cluster environment, and we will give you an example method for reference.

First, we will install and start lxcfs on the cluster node, and we will run the lxcfs FUSE file system using Kubernetes, container and DaemonSet.

All sample code in this article can be obtained from Github at the following address

Git clone https://github.com/denverdino/lxcfs-initializercd lxcfs-initializer

The manifest file is as follows

ApiVersion: apps/v1beta2kind: DaemonSetmetadata: name: lxcfs labels: app: lxcfsspec: selector: matchLabels: app: lxcfs template: metadata: labels: app: lxcfsspec: hostPID: true tolerations:-key: node-role.kubernetes.io/master effect: NoSchedule containers:-name: lxcfs image: registry.cn-hangzhou.aliyuncs.com/denverdino/lxcfs:2.0.8 ImagePullPolicy: Always securityContext: privileged: true volumeMounts:-name: rootfs mountPath: / host volumes:-name: rootfs hostPath: path: /

Note: since lxcfs FUSE needs to share the PID namespace of the system and requires privileged mode, we have configured the corresponding container startup parameters.

Isn't it easy to automatically install and deploy lxcfs on all cluster nodes with the following command? : -)

Kubectl create-f lxcfs-daemonset.yaml

So how do you use lxcfs in Kubernetes? As above, we can add the definition of volume (file volume) and volumeMounts (file volume mount) to the definition of Pod for the files under / proc. However, this makes the K8S application deployment files become more complex, is there any way to let the system automatically complete the mounting of the corresponding files?

Kubernetes provides an Initializer extension mechanism that can be used to intercept and inject resource creation, and we can use it to gracefully automate the mount of lxcfs files.

Note: Aliyun Kubernetes cluster has been enabled to support Initializer by default. If you are testing on a self-built cluster, please see the document to enable the corresponding function.

The manifest file is as follows

ApiVersion: rbac.authorization.k8s.io/v1kind: ClusterRolemetadata: name: lxcfs-initializer-default namespace: defaultrules:- apiGroups: ["*"] resources: ["deployments"] verbs: ["initialize", "patch", "watch" "list"]-apiVersion: v1kind: ServiceAccountmetadata: name: lxcfs-initializer-service-account namespace: default---kind: ClusterRoleBindingapiVersion: rbac.authorization.k8s.io/v1metadata: name: lxcfs-initializer-role-bindingsubjects:- kind: ServiceAccount name: lxcfs-initializer-service-account namespace: defaultroleRef: kind: ClusterRole name: lxcfs-initializer-default apiGroup: rbac.authorization.k8s.io---apiVersion: apps/v1beta1kind: Deploymentmetadata: initializers: pending: [] labels: app: Lxcfs-initializer name: lxcfs-initializerspec: replicas: 1 template: metadata: labels: app: lxcfs-initializer name: lxcfs-initializerspec: serviceAccountName: lxcfs-initializer-service-account containers:-name: lxcfs-initializer image: registry.cn-hangzhou.aliyuncs.com/denverdino/lxcfs-initializer:0.0.2 imagePullPolicy: Always args:-"- annotation=initializer .kubernetes.io / lxcfs "-"-require-annotation=true "--apiVersion: admissionregistration.k8s.io/v1alpha1kind: InitializerConfigurationmetadata: name: lxcfs.initializerinitializers:-name: lxcfs.initializer.kubernetes.io rules:-apiGroups: -" * "apiVersions: -" * "resources:-deployments

Note: this is a typical Initializer deployment description. First, we create a service account lxcfs-initializer-service-account and grant it permissions to find, change, and so on "deployments" resources. Then we deploy an Initializer named "lxcfs-initializer", and use the above SA to start a container to handle the creation of "deployments" resources. If the deployment contains a comment with initializer.kubernetes.io/lxcfs as true, the container in the application will be mounted with files.

We can execute the following command, and after the deployment is complete, we can play happily.

Kubectl apply-f lxcfs-initializer.yaml

Let's deploy a simple Apache application, allocate 256MB memory to it, and declare the following annotation "initializer.kubernetes.io/lxcfs": "true"

The manifest file is as follows

ApiVersion: apps/v1beta1kind: Deploymentmetadata: annotations: "initializer.kubernetes.io/lxcfs": "true" labels: app: web name: replicas: 1 template: metadata: labels: app: web name: webspec: containers:-name: web image: httpd:2 imagePullPolicy: Always resources: requests: memory "256Mi" cpu: "500m" limits: memory: "256Mi" cpu: "500m"

We can deploy and test in the following ways

$kubectl create-f web.yaml deployment "web" created$ kubectl get podNAME READY STATUS RESTARTS AGEweb-7f6bc6797c-rb9sk 1 Running 0 32s $kubectl exec web-7f6bc6797c-rb9sk free total used free shared buffers cachedMem: 262144 2876 259268 2292 0 304 buffers/cache + buffers/cache: 2572 259572Swap: 0 000

We can see that the total memory returned by the free command is the container resource capacity we set.

We can check the configuration of the above Pod, and sure enough, all the relevant procfs files have been mounted correctly.

$kubectl describe pod web-7f6bc6797c-rb9sk... Mounts: / proc/cpuinfo from lxcfs-proc-cpuinfo (rw) / proc/diskstats from lxcfs-proc-diskstats (rw) / proc/meminfo from lxcfs-proc-meminfo (rw) / proc/stat from lxcfs-proc-stat (rw). So much for sharing about how to use LXCFS to improve the visibility of container resources in Kubernetes. I hope the above content can be helpful to you and learn more. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.