Shulou(Shulou.com)06/01 Report--

This article focuses on "how to deploy docker swarm cluster monitoring", interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how to deploy docker swarm cluster monitoring.

Premise

Docker

Preface

Now Docker Swarm has completely lost to K8S, but now K8S is still very complex and more difficult to use than Docker Swarm. If it is a small team and requires container choreography, it is appropriate to use Docker Swarm.

At present, there is a problem in Docker Swarm that has not been solved. If the business needs to know the user's request IP, then Docker Swarm cannot meet the requirements. The service currently deployed in Docker Swarm cannot get the user's request IP.

You can take a look at this ISSUE- > Unable to retrieve user's IP address in docker swarm mode.

Overall thinking

The overall idea is to use Influxdb+Grafana+cadvisor, in which cadvisor is responsible for data collection, each node deploys a cadvisor service, Influxdb is responsible for data storage, Grafana is responsible for data visualization.

Demo environment host IPmaster (manager) 192.168.1.60node1 (worker) 192.168.1.61node2 (worker) 192.168.1.62

I am here to demonstrate the master node as a node for monitoring data storage and visualization services, usually with a worker node to do this.

Initialize Docker Swarm

Initialize the cluster on the master machine and run

Docker swarm init-- advertise-addr {MASTER-IP}

[root@master] # docker swarm init-- advertise-addr 192.168.1.60Swarm initialized: current node (138n5rwjz83y8goyzepp1cdo7) is now a manager.To add a worker to this swarm, run the following command: docker swarm join--token SWMTKN-1-67je7chylnpyt0s4k1ee63rhxgh0qijiah9gadvcr7i6uab909-535nf6qu6v7b8dscc0plghr9j 192.168.1.60:2377To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions

Run the prompted command on the node node to join the cluster

Docker swarm join-- token SWMTKN-1-67je7chylnpyt0s4k1ee63rhxgh0qijiah9gadvcr7i6uab909-535nf6qu6v7b8dscc0plghr9j 192.168.1.60 67je7chylnpyt0s4k1ee63rhxgh0qijiah9gadvcr7i6uab909 2377

After the manager node initializes the cluster, there will be such a prompt. This command is just an example. The actual command needs to be run according to the actual situation after initializing the cluster.

[root@node1] # docker swarm join-- token SWMTKN-1-67je7chylnpyt0s4k1ee63rhxgh0qijiah9gadvcr7i6uab909-535nf6qu6v7b8dscc0plghr9j 192.168.1.60:2377This node joined a swarm as a worker. [root@node2] # docker swarm join-- token SWMTKN-1-67je7chylnpyt0s4k1ee63rhxgh0qijiah9gadvcr7i6uab909-535nf6qu6v7b8dscc0plghr9j 192.168.1.60:2377This node joined a swarm as a worker. View the current node node on the master machine

Docker node ls

[root@master] # docker node lsID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION138n5rwjz83y8goyzepp1cdo7 * master Ready Active Leader 18.09.8q03by75rqur63lx36cmordf11 node1 Ready Active 18.09 .86shdf5ej4b5u7x877bg9nyjk3 node2 Ready Active

So far, the cluster has been built, and then the service is deployed.

Deploy monitoring services in Docker Swarm

Docker stack deploy-c docker-compose-monitor.yml monitor

[root@master ~] # docker stack deploy-c docker-compose-monitor.yml monitorCreating network monitor_defaultCreating service monitor_influxCreating service monitor_grafanaCreating service monitor_cadvisor

Docker-compose-monitor.yml file content

Version: '3'services: influx: image: influxdb volumes:-influx:/var/lib/influxdb deploy: replicas: 1 placement: constraints:-node.role = = manager grafana: image: grafana/grafana ports:-0.0.0. 0. 0. 0. 0. 0. Constraints:-node.role = = manager cadvisor: image: google/cadvisor hostname:'{{.Node.Hostname}} 'command:-logtostderr-docker_only-storage_driver=influxdb-storage_driver_db=cadvisor-storage_driver_host=influx:8086 volumes:-/: / rootfs:ro-/ var/run:/var/run:rw-/ sys:/sys:ro-/ var/lib/docker/:/var / lib/docker:ro depends_on:-influx deploy: mode: globalvolumes: influx: driver: local grafana: driver: local

Download docker-compose-monitor.yml

View the deployment of the service

Docker service ls

[root@master ~] # docker service lsID NAME MODE REPLICAS IMAGE PORTSqth5tssf2sm1 monitor_cadvisor global 3 google/cadvisor:latest p2vbxe7ic175 monitor_grafana replicated 1 grafana/grafana:latest 1 grafana/grafana:latest *: 80-> 3000/tcpvon1rpeqq7vj Monitor_influx replicated 1/1 influxdb:latest

So far, the service has been deployed, with each of the three machines deploying a cadvisor, grafana and influxdb on the master node

Configure the influxdb database for cadvisor

Check the services on the master machine

Docker ps

[root@master ~] # docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES55965fdf13a3 grafana/grafana:latest "/ run.sh" 3 hours ago Up 3 hours 3000/tcp monitor_grafana.1.l9uh0ov7ltk7q2yollmk4x1v90bf544c7d81c google/cadvisor:latest "/ usr/bin/cadvisor-…" 3 hours ago Up 3 hours 8080/tcp monitor_cadvisor.138n5rwjz83y8goyzepp1cdo7.l53vufoivp0oe8tyy14nh0jof3ce050f0483e influxdb:latest "/ entrypoint.sh infl …" 3 hours ago Up 3 hours 8086/tcp monitor_ roomx.1.vraeh8ektium1j1jd27qvq1au [root @ master ~] #

You can see that it is as expected. Next, take a closer look at the log of the cadvisor container.

Docker logs-f 0bf544c7d81c

[root@master ~] # docker logs-f 0bf544c7d81cW0209 09 0bf544c7d81cW0209 32 manager.go:349] Could not configure a source for OOM detection Disabling OOM events: open / dev/kmsg: no such file or directoryE0209 09dev/kmsg 33 memory.go:94 15.783705 1 memory.go:94] failed to write stats to influxDb-{"error": "database not found:\" cadvisor\ ""} E0209 09 144V 15.818661 1 memory.go:94] failed to write stats to influxDb-{"error": "database not found:\" cadvisor\ "" E0209 09 09 vehicle 3516.009312 1 memory.go:94] Failed to write stats to influxDb-{"error": "database not found:\" cadvisor\ ""} E0209 09 failed to write stats to influxDb:\ "error": "database not found:\" cadvisor\ "} E0209 09:\" cadvisor\ ""} E0209 09:\ "cadvisor\"} E0209 09:38:16 .215684 1 memory.go:94] failed to write stats to influxDb-{"error": "database not found:\" cadvisor\ ""} E0209 09 09 cadvisor 39 virtual 16.305772 1 memory.go:94] failed to write stats to influxDb-{"error": "database not found:\" cadvisor\ ""}

You can see that errors have been reported all the time, because there is no database such as cadvisor in the current influx container. Next, we go to the influx container and create the corresponding cadvisor database, and execute the following command on the master machine.

Docker exec `docker ps | grep-I influx | awk'{print $1} '`influx-execute' CREATE DATABASE cadvisor'

Of course, it can also be implemented step by step.

Find influxdb's container.

Enter the influxdb container and log in to influx

Create a database

There will be no demonstration here.

Configure grafana

So far, the data has been collected and stored in influxdb. Next, configure grafana to visualize the data.

Because the port configured for grafna in the docker-compose-monitor.yml file is 80, you can access grafana by directly accessing the IP of the master machine and open 192.168.1.60 in the browser.

Grafana

The default account is admin

The default password is admin

You will be prompted to change your password after logging in for the first time, and it doesn't matter if the new password is set to admin.

Start setting up the data source after a successful login

Configure the data source

Open the left menu bar to enter the data source configuration page

Add a new data source, I have added it here, so there will be an influxdb data source display.

Select a data source of type influxdb

Fill in the information corresponding to influxdb, and fill in influx in Name. Because a grafana template will be used later, it is called influx name here, and it is not fixed for URL to fill in http://influx:8086,. The container of influxdb in this docker-compose-monitor.yml file is called influx, and the port is opened to 8086 (default), so enter influx:8086 here.

So far, the content related to the data source has been configured.

Configure grafana View template

The template is used here only to demonstrate the effect. If the style of the template is not satisfied, you can study the grafana to adjust itself.

First, open grafana's dashboard market to download the template https://grafana.com/grafana/dashboards/4637/reviews.

Select the dashboard menu, select import to import

Open dashboard and you can already see the contents of the dashboard template.

Summary

A basic Docker Swarm cluster monitoring is completed.

There are more advanced ones and maybe a blog will be updated later to tell it. For example, when a certain value (CPU) reaches a certain threshold, send a nail or slack message to alarm.

As long as you understand the train of thought, there is basically no problem with practical operation.

At this point, I believe you have a deeper understanding of "how to deploy docker swarm cluster monitoring". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.