Shulou(Shulou.com)06/01 Report--

This article mainly introduces "how to use logstash as a docker log driver to collect logs". In daily operation, I believe many people have doubts about how to use logstash as a docker log driver to collect logs. Xiaobian consulted all kinds of information and sorted out simple and easy-to-use methods of operation. I hope it will be helpful to answer the questions of "how to use logstash as a docker log driver to collect logs". Next, please follow the editor to study!

Preface

Logstash is an open source data collector for unified processing of logs, which belongs to L in ELK and is widely used in the field of log collection.

Docker default log driver is json-file, each container will generate a / var/lib/docker/containers/containerID/containerID-json.log locally, and the log driver supports extension. This chapter mainly describes the use of logstash to collect docker logs.

Docker does not have a logstash driver, but you can collect logs of gelf drivers through logstash's gelf input plug-in.

Premise

Docker

Understand logstash configuration

Docker-compose

Prepare the configuration file

Docker-compose.yml

Version: '3.7'x-logging: & default-logging driver: gelf options: gelf-address: "udp://localhost:12201" mode: non-blocking max-buffer-size: 4m tag: "kafeidou. {{.Name}" # configure the tag of the container to kafeidou. For prefix, container name is suffix, docker-compose will add copy suffix to container. > for example, logstash_1services: logstash: ports:-12201:12201/udp image: docker.elastic.co/logstash/logstash:7.5.1 volumes: -. / logstash.yml:/usr/share/logstash/config/logstash.yml-/ var/log/logstash:/var/log/logstash -. / logstash.conf:/usr/share/logstash/pipeline/logstash.conf logstash-worker: image: docker.elastic.co / logstash/logstash:7.5.1 depends_on:-logstash logging: driver: "gelf" options: gelf-address: "udp://localhost:12201"

Logstash.yml

Http.host: "0.0.0.0"

Logstash.conf

Input {gelf {use_udp = > true port_tcp = > 12202}} output {file {path = > "/ var/log/logstash/% {+ yyyy-MM-dd-HH} /% {container_name} .log"}}

Because logstash needs to have write permission in the configured directory, you need to prepare the directory to store log and give it permission.

Create a directory

Mkdir / var/log/logstash

Give permission, here for experimental demonstration, directly authorize 777

Chmod-R 777 / var/log/logstash

Execute the command in the directory of the docker-compose.yml,logstash.conf and logstash.yml files:

Docker-compose up-d

[root@master logstash] # docker-compose up-dWARNING: The Docker Engine you're using is running in swarm mode.Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.To deploy your application across the swarm, use `docker stack deploy`.Starting logstash_logstash_1... DoneStarting logstash_logstash-worker_1... Done

Logstash starts slowly, and the effect of my experiment is about 90 seconds, so it is more recommended to use fluentd to collect logs

Check the log directory and you should have the corresponding container log file:

[root@master logstash] # ls / var/log/logstash/2020-02-16 [root@master logstash] # ls / var/log/logstash/2020-02-16/logstash_logstash-worker_1.log

You can also download my files directly:

Docker-compose.yml

Logstash.conf

Logstash.yml

Summary of technology selection is more recommended fluentd, why?

Fluentd is more lightweight and flexible, and currently belongs to CNCF, activity and reliability please climb to the next level.

Why introduce logstash to collect docker logs?

If a company or business is already using the ELK technology stack, it may not be necessary to introduce a fluentd, but continue to use logstash to get through the docker log. Here is mainly a sharing, so that students in this situation can have one more choice.

At this point, the study on "how to use logstash as a docker log driver to collect logs" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.