Shulou(Shulou.com)06/01 Report--

This article mainly explains the "implementation method of automatic monitoring of Docker containers". The explanation content in this article is simple and clear, and it is easy to learn and understand. Please follow the ideas of Xiaobian and go deeper to study and learn the "implementation method of automatic monitoring of Docker containers" together.

abstr

In recent years, container technology has been mature and applied. Docker, as a representative of container technology, is currently developing rapidly, and various applications based on Docker are also becoming popular. At the same time, Docker has also brought impact to the traditional operation and maintenance system. In the process of building an operation and maintenance platform, we also need to face and solve container-related problems.

Docker operation and maintenance is a system, and monitoring system as an important part of the operation and maintenance system, in the Docker operation and maintenance process needs to be considered. This article introduces an automatic monitoring implementation method for Docker containers, aiming to provide relevant solutions for the establishment of Docker operation and maintenance system.

container

When it comes to containers, some people first think of LXC (Linux Container). It is a kernel virtualization technology, a virtualization of resources at the operating system level. Before Docker came along, there were already companies using LXC technology. The use of container technology has greatly improved resource utilization and reduced costs.

Direct use of LXC is slightly more complex, enterprises embrace container technology has a certain threshold, it can be said that the emergence of Docker has changed this situation. Docker encapsulates the complexity of the underlying technology of containers, greatly reducing the complexity of using them, thus lowering the threshold for using container technology. Docker provides some basic specifications and interfaces, and users can easily play with container technology as long as they are familiar with Docker's interfaces. It can be said that Docker has greatly accelerated the popularity of container technology and is even regarded as an industry container specification.

Monitoring of containers

Containers differ from ordinary virtual machines in the degree of virtualization and in the means of monitoring. A virtual machine can be treated as a physical machine, and although containers can also be treated as virtual machines, this does not conform to the concept of container use. In the implementation of monitoring, we prefer to think of containers as a series of process trees on the host.

In the process of implementing mainstream monitoring systems, agent modules are generally deployed on target machines, and data collection is done through agent modules. According to the concept of container use, it is generally not recommended to bundle agents in container images. Of course, this does not mean that data cannot be collected. According to the virtualization technology characteristics of containers, it is completely feasible to collect data from containers on the host machine of containers, and it can be more efficient.

Of course, if the container is treated as a virtual machine, it is also a method to deploy an agent module to collect monitoring data, but this is not recommended. We can see that some Docker monitoring solutions that have emerged in the industry, such as Docker Stats, CAdvisor, Scout, etc., are also monitoring containers on the host. The monitoring scheme proposed in this paper will also start from the host computer.

Common problems with container monitoring

With the application of Docker, there are many monitoring tools in the industry, and these tools can actually monitor Docker containers. Using these tools to build a monitoring system to use is also basically able to solve some needs. But analyzing these monitoring tools, there are two main problems.

1. Degree of integration with operation and maintenance system

These tools are basically independent and difficult to integrate with other systems in the operation and maintenance system. In today's continuous development of operation and maintenance automation, more attention is often paid to the integration of the entire system. Therefore, a better modeling idea is needed to facilitate the data connection between systems.

2. hierarchy of monitoring

The monitoring of these tools generally stays at the level of a single container, such as the monitoring of the container's CPU, disk IO, etc. However, most application design architectures have certain node fault tolerance capabilities, and the problems of a single node often cannot reflect the real problems of the application. So monitoring needs to cover more layers.

Modelling container monitoring solutions

Here we propose a modeling monitoring scheme as a whole. This solution is conducive to getting through with the CMDB system based on operation and maintenance, and at the same time, it can take into account more levels of monitoring.

Monitoring system generally involves: data acquisition, data storage, data analysis and alarm, data display and other parts. This article will describe a modeling monitoring method, mainly proposed the following five models:

1 Monitoring object model

Here we will model the monitored objects using a product tree structure. The monitoring objects are divided into four categories: products, applications, clusters and nodes.

○ Product: Generally a high-level concept, a product can generally be independently exported and provide services to the outside world.

○ Application: It is composed of modules under the product, and multiple applications jointly form a product.

Cluster: It is the form of existence of the application. The same application will generally deploy multiple clusters according to the environment, region, etc.

Node: Resources that carry services within the cluster, including the servers, virtual machines, containers, etc. mentioned above.

3 Data models

Used to define the monitoring data format. The model includes data items and indicator items. A data item generally contains one or more indicator items. The data in the data model comes from the corresponding collector.

This model represents the CPU trend graph and is plotted according to the two metrics usr, sys. Examples are as follows:

The basic functions of each module are briefly described as follows:

○ agent: node monitoring data collection

○ master: agent's control center, responsible for sending monitoring item configuration to agent.

Monitor: Receive monitoring data collected by agent and store it in Kafka message queue.

Analyzer: Subscribe to Kafka alignment messages, analyze and process data, store and alarm. (In the actual implementation process, the module can be appropriately expanded and split according to the situation)

○ web: various management of monitoring models, display of views.

○ kafka: message queue, cache collection data, a total of other modules subscribe to use.

DB/HBase: Store model configuration, monitoring data, etc.

This architecture is a common monitoring model architecture, and it is relatively easy to get through with the operation and maintenance system. This model can be used in our implementation of container monitoring.

Container Monitoring Data Collection

Data acquisition is the most significant difference between Docker monitoring and general monitoring system implementation. Because inside Docker containers, agent modules without data collection will not be able to rely directly on agents for collection.

1. node data

On the container host, we can obtain a lot of basic data of the container. There are usually several ways.

Docker command

docker stats This method is relatively simple, but the data is not comprehensive, we can see the following effects.

2. data acquisition

Cluster data is calculated from the original data on each node. It is a kind of aggregation operation. Generally, there will be sum, avg and other operation scenarios.

3. Application and product data

Similarly, application and product data can be calculated from child node data.

Automation of monitoring

Due to the nature of containers, destruction, creation, etc. of containers is a common scenario. After a container is started, how does the monitoring system detect it and what data models need to be collected? These problems need to be solved in the monitoring automation process.

1. Self-discovery of containers

Container new creation, stop, or destroy is perceived on the host. It can generally be obtained from the following directories. Due to different Docker installation configurations or differences in file systems used by Docker, some directories may be inconsistent, but the actual acquisition strategies are similar.

Once a container is associated with a cluster, item configuration can be monitored automatically. After the configuration is delivered to the agent on the container host through the master, you can start collecting and reporting data on the container, so as to automatically monitor the container.

summary

This paper presents a model-based container monitoring scheme. By modeling the monitoring object and monitoring process, the whole monitoring scene is driven based on the model, and the main implementation method of the scheme is described.

Compared with existing container monitoring implementations, this solution has better flexibility and scalability. Through the improvement and extension of the model, Docker container monitoring can be easily integrated into the existing monitoring and operation system.

The monitoring system itself is a very complex system. The scheme described in this paper has not been fully developed in many details, and there may be some limitations and ill-conceived places in the establishment of the model, which need to be gradually improved. I hope this paper can provide some references for readers in the process of developing monitoring system and constructing operation and maintenance system.

Thank you for reading, the above is the content of "Docker container automatic monitoring implementation method", after the study of this article, I believe that everyone has a deeper understanding of Docker container automatic monitoring implementation method, the specific use situation still needs to be verified by practice. Here is, Xiaobian will push more articles related to knowledge points for everyone, welcome to pay attention!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.