Shulou(Shulou.com)06/01 Report--

The editor will share with you how to set up regular cleaning and rewinding of container instance logs in cloud computing. I hope you will gain something after reading this article. Let's discuss it together.

Introduction

Kubernetes does not provide a native solution for log collection of container instances. You can use the kubectl logs command to view the running log of the container instance. The basic principle of the kubectl logs command is that the container runtime outputs logs from standard output and standard error to disk by default. Save to the host directory: / var/lib/docker/containers/container_id/ directory. When the user invokes the kubectl logs command, kubelet reads the data in the corresponding log file, sends the data back to master, and then master returns it to the user. So as to realize the user's view of the log.

Tencent Cloud CCS uses the kubectl logs command to view the log of the corresponding container instance on the console, and provides the feature to view the log of a specific time period, which greatly facilitates users to locate and track the programs in the container instance. However, because the container instance log is saved locally, when a large number of logs are printed in the program, it is easy to cause a large amount of disk space on the host to be occupied. After the log service is online for a period of time, it is found that when users encounter this situation, they usually clean the logs manually. Let's consider whether there is a relatively simple way to clean and roll back the logs on a regular basis in the cluster node.

Regular log cleaning and rollback through logrotate service

Logrotate is a very useful tool that automatically truncates (or rotates) logs, compresses, and deletes old log files. For example, you can set up logrotate to rotate the / var/log/foo log file every 30 days and delete logs for more than 6 months. After configuration, the operation of logrotate is fully automated without any further human intervention.

However, if you follow the previous deployment pattern, you need to manually install and configure the corresponding logrotate tools on each node. Through the ability of Kubernetes CCS orchestration, logrotate can be deployed to each node through services in Kubernetes, so that it can be deployed to all nodes only once. And through the container way to ensure the consistency of the logrotate configuration.

The specific implementation plan is shown in the following figure:

The specific implementation of the scheme is to create a DaemonSet resource implementation in the Kubernetes cluster. The DaemonSet resource deploys a container instance of logrotate on each Node node, and sets the log log directory of the mapped host in the container instance, thus realizing regular log cleaning and rollback.

An example of creating a DaemonSet is as follows:

ApiVersion: extensions/v1beta1kind: DaemonSetmetadata: name: logrotatespec: template: metadata: labels: app: logging id: logrotate name: logrotatespec: containers:-name: logrotate-es image: blacklabelops/logrotate securityContext: privileged: true volumeMounts:-name: containers mountPath: / var/lib/docker/containers-name: varlog MountPath: / var/log/docker-name: logs mountPath: / logs env:-name: LOGS_DIRECTORIES value: "/ var/lib/docker/containers / var/log/docker"-name: LOGROTATE_INTERVAL value: "hourly"-name: LOGROTATE_OLDDIR value: "/ logs" volumes: -hostPath: path: / var/lib/docker/containers name: containers-hostPath: path: / var/log/docker name: varlog-hostPath: path: / var/log/containers/ name: logs

Using this yaml file, you can deploy directly in Kubernetes.

# kubectl create-f logrotate_ds.yamldaemonset "logrotate" >

In the example yaml file, the logrotate service rolls back the log on a regular basis (1 hour), and cleans up the log after it rolls back more than 5 copies. If necessary, you can modify the corresponding parameters and set different rollback rules and cleanup rules. For more information on the parameters, please see https://github.com/blacklabelops/logrotate.

Rollback and cleanup by modifying the dockerd parameter

Because of the log collection of Kubernetes, the underlying layer is implemented through docker. On the other hand, docker provides some log rollback and cleaning functions. You can roll back and clean up the log by adding the log-opts () parameter to the startup parameters of dockerd, where the max-size parameter sets the maximum value of a copy of the log and max-file sets the maximum number of copies of the log. If you exceed this number, the log will be deleted.

The specific modification process includes three steps:

1. Create / etc/dockerd/daemon.json

{"log-driver": "json-file", "log-opts": {"max-size": "10m", "max-file": "3"}}

Parameter description: rollback is performed if a single container log is set to exceed 10m, and cleaned up if the number of copies of rollback exceeds 3.

2. Modify the dockerd service configuration file

In the / etc/systemd/system/multi-user.target.wants/dockerd.serviced file

Add dockerd startup parameter-config-file=/etc/docker/daemon.json

3. Restart the dockerd service

Systemctl daemon-reloadservice dockerd restart has read this article, I believe you have a certain understanding of "how to set up regular cleaning and rollback of container instance logs in cloud computing". If you want to know more about it, welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.