Shulou(Shulou.com)05/31 Report--

This article mainly explains "how to build a Docker container monitoring system". The content of the article is simple and clear, and it is easy to learn and understand. Please follow the editor's ideas to study and learn "how to build a Docker container monitoring system".

As a very excellent container management technology, Docker is favored by a large number of enterprises. There are more and more scenarios in which Docker containers are used to deploy services and applications in the production environment. Therefore, in the face of the increasingly large number of docker service group applications, how to be targeted and effective monitoring has also become the work requirements of enterprise operation and maintenance personnel.

There are many solutions for container information collection and monitoring, including docker's own docker stats command, Scout, Data Dog, Prometheus and so on. This is to build and share two classic container open source monitoring combination solutions Cadvisor+InfluxDB+Grafana and Cadvisor+Prometheus+Grafana.

Cadvisor+InfluxDB+Grafana1.1 、 Cadvisor

Cadvisor is a monitoring tool used by Google to monitor resource information of a single node. Cadvisor provides basic query interface and http interface, which is convenient for other components such as Grafana, Prometheus and so on to grab data. Cadvisor can real-time monitor resources and containers on Docker hosts and collect performance data, including CpU usage, memory usage, network throughput and file system usage. Cadvisor is developed in the go language and uses Liunx's Cagroups to obtain the resource usage information of the container.

Google is also used as a resource monitoring tool for a single node by default in Kubernetes, and Advisor components are installed on each node by default.

Cadvisor product features:

The monitoring data at both host and container levels can be displayed.

Historical change data can be displayed.

Google's open source products.

The monitoring indicators are complete.

Easy to deploy, with an official docker image.

By default, only 1 minute of data is saved locally, which can be inherited from third-party storage such as InfluxDB.

Because Cadvisor provides a slightly crude interface, needs to jump between different pages, and can only monitor one Host, people can't help but question its usefulness. But one of the highlights of Cadvisor is that he can give the monitored data everywhere to third-party tools for further processing.

We can position Cadvisor as a monitoring data collector, collecting and exporting data is its strength, rather than displaying data.

1.2 、 InfluxDB

InfluxDB is an open source non-relational sequential database developed by InfluxData. It is written by Go and focuses on "performance" query and storage of sequential data. InfluxDB is widely used in storage system monitoring data, loT industry real-time data and other scenarios. The same type of database products include Elasticsearch, Graphite and so on. ←

InfluxDB application scenarios: back-end storage of performance monitoring, application metrics, Internet of things sensor data and real-time analysis.

Main functions of nfluxDB:

Based on time series, support time-related functions (such as maximum, minimum, summation, etc.)

Measurability: you can calculate large amounts of data in real time

Event-based: it supports arbitrary event data

The main features of InfluxDB:

Unstructured (modeless): can be any number of columns

Support expansion

Support a series of functions such as min,max,sum, count,mean, median, etc., to facilitate statistics

Native HTTP support, built-in HTTP API

Powerful SQL-like syntax

1.3 、 Grafana

Grafana is a visual panel (Dashboard) tool with very beautiful icons and layout display functions, full-featured measurement dashboards and compilers, and supports the main features of Graphite, zabbix, InfluxDB, Prometheus and Grafana

Flexible and rich graphical options

Can mix a variety of styles

Support for day and night mode

Support for multiple data sources

1.4. Monitoring and building architecture

Monitoring component architecture diagram:

Tip: InfluxDB is used for data storage, Cadvisor user data acquisition, Grafana for data display.

Monitor component architecture deployment scenarios:

Create a custom network monitor (custom network name) for later containers to join this network

Create InfluxDB containers, create data users, databases

Create a Cadvisor container

Create a Grafana container and configure grafana

1.5 、 Start deploying [root@flag ~] # iptables-F [root@flag ~] # setenforce 0setenforce: SELinux is disabled [root@flag ~] # systemctl stop firewalld install docker-ce configuration image acceleration slightly download image component docker pull tutum/influxdbdocker pull google/cadvisordocker pull grafana/grafana [root@flag ~] # docker imagesREPOSITORY TAG IMAGE ID CREATED SIZEgrafana/grafana latest 651ff2dc930f 2 weeks ago 187MBgoogle/cadvisor latest eb1210707573 2 years ago 69.6MBtutum/influxdb latest c061e5808198 4 years ago 290MB create a custom network

In order to add the three containers of Cadvisor+InfluxDB+Grafana created later to the self-defined network, it is easy to manage.

Solution and management, so create a new custom network. (let the three containers uniformly manage and communicate with each other)

[root@flag ~] # docker network create monitor0533cc85ed9f3f9d6865e091120aa2ed051cbde09d8daf342dab456b6d7fc0ca [root@flag ~] # docker network lsNETWORK ID NAME DRIVER SCOPE8aecbd2a0b19 bridge bridge localfc1c55c5fa26 host host local0533cc85ed9f monitor bridge local8714055457ff none null local create InfluxDB container

Launch the container. The tutum/influxdb image does not have a specified version. The latest version of InfluxDB will be pull by default:

Parameter description:

-d: run this container in the background

-- name: the shipping container assigns the name influxdb

-- net: add the container to the new network monitor

-p: mapped port, port 8083 is the influxdb background control port, port 8086 is the data port of infuxdb

Tutum/influxdb: runs through this container, and defaults to the docker official warehouse pull.

Check to see if the influxdb container starts:

[root@flag ~] # docker ps-lCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESa6f6543140ed tutum/influxdb "/ run.sh" >

Visit the influxdb console http://192.168.200.111:8083

As you can see from the figure above, common operation templates are provided in the Query Templates options, according to which you can manage influxdb. Let's create the database and database users.

CREATE USER "root" WITH PASSWORD '890' WITH ALL PRIVILEGES

Create Cadvisor database cadvisor and user root, which can be defined by users and database at will, which can be used for later grafana configuration:

CREATE DATABASE "cadvisor"

Create Cadvisor container [root@flag ~] # docker run-d-- volume=/:/rootfs:ro-- volume=/var/run:/var/run:rw-- volume=/sys:/sys:ro-- volume=/var/lib/docker/:/var/lib/docker:ro-- net monitor-- publish=8080:8080-- name=cadvisor google/cadvisor-storage_driver=influxdb-storage_driver_db=cadvisor-storage_driver_host=influxdb:8086effa70d8cd00189b6ea51ae8e187d29df574fcd6194e5deda2ab5cd218430a55

Parameter description:

-d: run the secondary container in the background

-- name: launches the container assignment name Cadvisor

-- net: add the container to the new network monitor

-p: map port 8080

-- mout: bind the photo directory of the host to the container. These directories are the directory files and monitoring contents that Cadvisor needs to collect.

-storage_driver: you need to specify the storage driver, data host and database name of the Cadvisor

Google/Cadvisor: use the Cadvisor image to run the container. By default, the image will be pull in the official docker repository.

View Cadvisor Container

[root@flag] # docker ps-lCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESeffa70d8cd00 google/cadvisor "/ usr/bin/cadvisor-…" 6 minutes ago Up 6 minutes 0.0.0.014 8080-> 8080/tcp cadvisor

Through the http://192.168.200.10:8080 shorts access test, the first visit to this page is a bit slow

As you can see from the picture above, Cadvisor also has a basic graphical interface display function, which is mainly used for data collection here.

Create grafana container [root@flag ~] # docker run-d-- name grafana-- net monitor-p 3000 name grafana

View the running results:

[root@flag ~] # docker ps-lCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES6a4a527897bc grafana/grafana "/ run.sh" 8 seconds ago Up 6 seconds 0.0.0.0 lCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES6a4a527897bc grafana/grafana 3000-> 3000/tcp grafana

Access grafana through the http://192.168.200.10:3000 port. The default account password (admin/admin) needs to be updated for the first time.

Add the data source Add data source, as shown below:

Create a new Dashboard, as shown below:

Brief introduction of Cadvisor+Prometheus+Grafana2.1 and Cadvisor products

Cadvisor is an open source visualization tool for displaying and analyzing the running state of containers by Google. By running Cadvisor on the host, the user can easily get the running statistics of the container on the current host and show it to the user in the form of a chart.

2.2.deploy Cadvisor

Deploy Cadvisor containers on monitored hosts

[root@10 ~] # docker rm-f $(docker ps-aq) 6a4a527897bceffa70d8cd00a6f6543140ed [root@10 ~] # docker run-d\-- volume=/:/rootfs:ro\-- volume=/var/run:/var/run:ro\-- volume=/sys:/sys:ro\-- volume=/var/lib/docker/:/var/lib/docker:ro\-- volume=/dev/disk/:/dev/disk:ro\-- publish=8080:8080\-- detach=true\-- name=cadvisor\ google/cadvisor:lateste7493105e8a02b82f1ec67517a2ee28124d2b6f2659d4c9524361ca0cdc953502.3, visit Cadvisor page

Visit the http://192.168.200.10 cadvisor page to see the data received by the phone.

2.4.Productivity of Prometheus

Prometheus is an open source system monitoring and alerting toolkit originally built on SoundCloud. Since its inception in 2012, many companies and organizations have adopted Prometheus, which has a very active developer and user community. It is now an independent open source project that can be maintained independently of any company. To emphasize this point and clarify the governance structure of the project, Prometheus joined Cloud Native Computing Foundation in 2016 as the second managed project after Kubernetes.

The main feature of Prometheus is ∶.

Multi-dimensional data model-composed of time series data identified by index key-value pairs

PromQL, a flexible query language

Does not rely on distributed storage; a single server node is autonomous

Pull time series data through pull model in HTTP way

Support pushing time series data through intermediate gateways

Discover the target service object through service discovery or static configuration

Support a wide variety of charts and interface displays

2.5. Deploy Prometheus [root@10 ~] # docker imagesREPOSITORY TAG IMAGE ID CREATED SIZEprom/prometheus latest 0bc82119c95b 2 weeks ago 169MBgrafana/grafana latest 651ff2dc930f 2 weeks ago 187MBgoogle/cadvisor latest eb1210707573 2 years ago 69.6MB

Prepare the configuration first

[root@10 ~] # vi / tmp/prometheus.yml # my global configglobal: scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute. Evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute. # scrape_timeout is set to the global default (10s). # Alertmanager configurationalerting: alertmanagers:-static_configs:-targets: #-alertmanager:9093# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.rule_files: #-"first_rules.yml" #-"second_rules.yml" # A scrape configuration containing exactly one endpoint to scrape:# Here it's Prometheus itself.scrape_configs: # The job name is added as a label `job= `to any timeseries scraped from this config. -job_name: 'prometheus' # metrics_path defaults to' / metrics' # scheme defaults to 'http'. Static_configs:-targets: ['localhost:9090']-job_name:' docker' # defines a group called docker static_configs:-targets: ['192.168.200.10localhost:9090' 8080'] # fill in one or more host addresses of cadvisor separated by commas

Run the container

[root@10 ~] # docker run-d\-- name=prometheus-p 9090pur9090\-v / tmp/prometheus.yml:/etc/prometheus/prometheus.yml\-v / etc/localtime:/etc/localrime\ prom/prometheus2.6, visit the Prometheus page

Http://192.168.200.10:9090

See that the docker group status up is normal

Data can be found in the query items.

2.7. deploy Grafana [root@10 ~] # docker run-d\-- name=grafana\-p 3000 name=grafana\ grafana/grafana2.8, and configure Grafana

Access to http://192.168.200.10:3000 default account admin password admin first login password needs to be changed

Configure the data source

Import template

Select the corresponding data source, click Import, and you can see the data of the monitored host

Prepare the test container

Thank you for reading, the above is the content of "how to build a Docker container monitoring system". After the study of this article, I believe you have a deeper understanding of how to build a Docker container monitoring system, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.