Shulou(Shulou.com)06/02 Report--

This article introduces how to use docker to achieve log monitoring, the content is very detailed, interested friends can use for reference, I hope it can be helpful to you.

Log processing mechanism

Let's first take a look at the mechanism of docker log processing. When you start a container, it is actually a child process of docker daemon. Docker daemon can get the standard output of the process in your container. After getting the standard output, it will handle it through its own LogDriver module. LogDriver supports many ways, such as writing to a local file (the default), sending it to syslog, and so on.

By default, docker stores the standard output of mobile applications in a json.log file in a format similar to the following:

{"log": "root@c835298de6dd:/# ls\ r\ n", "stream": "stdout", "time": "xxoo.155863426Z"} {"log": "bin boot dev\ u0009etc home lib\ u0009lib64 media mnt opt\ u0009proc root run sbin selinux\ u0009srv sys tmp usr var\ r\ n"}

Store one row at a time as a json data. This log storage method of docker can be configured. Specific parameters can be configured through log-driver when running the run startup container. For more information, please see log-driver.

Docker uses json-file driver as the log driver by default, and gelf is the log driver we need to use. When there are too many containers, or when docker is deployed in a similar swarm cluster, various logs are stored in various json.log files. When looking for problems or doing related statistics, scattered logs are very unfriendly to us. We need a tool that can centrally manage docker logs, which is graylog.

Graylog

Docker natively supports graylog protocol and sends logs directly to graylog (via gelf protocol)

Graylog officially provides support for deploying itself to docker.

Graylog officially provides dockerfile for us to quickly deploy the log system on docker. In the address of this docker hub, docker-compose.yml is also provided to quickly deploy the entire graylog stack, including MongoDB and elasticsearch, without the need to deploy separately.

Https://hub.docker.com/r/graylog/graylog

Graylog deployment

Create a directory to deploy graylog. This article assumes that the directory is / root/graylog, and all of the following operations are done in / root/graylog.

Initialize directories and configuration files

# create data directory mkdir-p. / graylog/data# create configuration file directory mkdir-p. / graylog/configcd. / graylog/config# download the officially recommended configuration file wget https://raw.githubusercontent.com/Graylog2/graylog-docker/2.5/config/graylog.conf# log configuration file wget https://raw.githubusercontent.com/Graylog2/graylog-docker/2.5/config/log4j2.xml directly

Modify the root_timezone in the downloaded graylog.conf to GMT+0800 China time zone

Root_timezone=Etc/GMT-8

Create a new docker-compose.yml for docker-compose to quickly start the completion service. It should be noted that due to the large amount of docker-compose content, we have stored it in the current sibling directory in the form of an attachment.

Start the entire service

Docker-compose up

If there is no problem, you will see the terminal output message of graylog webserver started. When you visit http://{server}:9000, you will see the web interface of graylog. Log in to the backend with the username admin and password admin, and the deployment is complete.

Complete docker-compose file

Version: '2'services: mongodb: image: mongo:3 volumes:-/ usr/share/zoneinfo/Asia/Shanghai:/etc/localtime-mongo_data:/data/db elasticsearch: image: elasticsearch:6.6.2 volumes:-/ usr/share/zoneinfo/Asia/Shanghai:/etc/localtime-es_data:/usr/share/elasticsearch/data# here you need to add a jvm.options file locally And specify the garbage collector as G1GC Otherwise, it cannot be started successfully-/ Users/zhangxufeng/xufeng.zhang/docker/conf-compose/graylog/graylog/jvm.options:/usr/share/elasticsearch/config/jvm.options environment:-http.host=0.0.0.0-transport.host=localhost-network.host=0.0.0.0-xpack.security.enabled=false-xpack.watcher.enabled=false-xpack.monitoring.enabled=false-xpack.security.audit .enabled = false-xpack.ml.enabled=false-xpack.graph.enabled=false-ES_JAVA_OPTS=-Xms512m-Xmx512m-XX:+UseG1GC ulimits: memlock: soft:-1 hard:-1 mem_limit: 512m graylog: image: graylog/graylog:2.5 volumes:-/ usr/share/zoneinfo/Asia/Shanghai:/etc/localtime-graylog_journal:/usr/share/graylog/data / journal -. / graylog/config:/usr/share/graylog/data/config environment:-GRAYLOG_PASSWORD_SECRET=admin_zxfwy1314_-GRAYLOG_ROOT_PASSWORD_SHA2=8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918-GRAYLOG_WEB_ENDPOINT_URI= http://127.0.0.1:9000/api links:-mongodb:mongo-elasticsearch depends_on:-mongodb-elasticsearch ports:-9000 virtual 9000-514: Mongo_data: driver: local es_data: driver: local graylog_journal: driver: local

Graylog system configuration

If docker is started from the command line, you can add the following parameters to the run command:

Docker run-log-driver=gelf-log-opt gelf-address=udp:// {graylog server address}: 12201-log-opt tag=

Example:

Docker run-d-- log-driver=gelf-- log-opt gelf-address=udp://localhost:12201-- log-opt tag= "{{.imageName}} / {{.Name}} / {{.ID}}" busybox sh-c 'while true; do echo "Hello, this is A"; sleep 10; done;'

If you use the docker-compose command, you can add relevant configurations to docker-compose.yml. Take the NGINX container as an example:

Version: '2'services: nginx: image: nginx:latest ports:-"80:80" logging: driver: "gelf" options: gelf-address: "udp://localhost:12201" tag: front-nginx

Log collection of graylog is accomplished by defining input objects. In the graylogweb management interface, enter the input object configuration as follows, and select GELF UDP protocol to create a new input device:

On how to use docker to achieve log monitoring is shared here, I hope the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.