Shulou(Shulou.com)06/01 Report--

This article mainly introduces how to install and use Filebeat in Linux. It is very detailed and has a certain reference value. Friends who are interested must read it!

Filebeat is a lightweight delivery program for forwarding and centralizing log data. As an agent installation on the server, Filebeat monitors the log files or locations you specify, collects log events, and forwards them to Elasticsearch or Logstash for indexing.

I. Architecture Diagram

This experiment is based on the previous articles, and we need to build the basic environment based on the previous articles.

Second, install Filebeat

Download and install Filebeat

Wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.0.1-x86_64.rpmyum install. / filebeat-6.0.1-x86_64.rpm

Modify Filebeat configuration file

Vim / etc/filebeat/filebeat.yml # main configuration file\-type: log # document type paths:\-/ var/log/httpd/access.log* # where to read the data # output in elasticsearch or logstash choose one of the two output.elasticsearch: # to export the data to Elasticsearch. Choose either hosts or the following logstash: ["localhost:9200"] output.logstash: # to transfer data to logstash, to configure logstash to receive hosts using beats: ["172.18.68.14 logstash 5044"]

Start Filebeat

Systemctl start filebeat III. Configure Filebeat

Configure Logstash to receive data collected from Filebeat

Vim / etc/logstash/conf.d/test.confinput {beats {port = > 5044 # snooping 5044 is used to receive data from Filebeat} filter {grok {match = > {"message" = > "% {COMBINEDAPACHELOG}" # matching HTTP logs} remove_field = > "message" # does not show the original information Only after matching}} output {elasticsearch {hosts = > ["http://172.18.68.11:9200","http://172.18.68.12:9200","http://172.18.68.13:9200"] # Cluster IPindex = >" logstash-% {+ YYYY.MM.dd} "action = >" index "document_type = >" apache_logs "}} is displayed

Start Logstash

/ usr/share/logstash/bin/logstash-f / etc/logstash/conf.d/test.conf IV. Simulated log access

Use the curl command to simulate customer access and generate access logs

Curl 127.0.0.1curl 172.18.68.51curl 172.18.68.52curl 172.18.68.53 V. Verification information

Clear the old data from the previous experiment (type the deletion in the dialog box when deleting), and then you can see the data collected by filebeat that is filtered by Logtash and sent to Elasticsearch.

Expansion

With the gradual upgrade of the ELK log system, now it can collect the logs of each node based on Filebeat, filter and trim the data by Logstash, and finally build index, word segmentation and search engine in ELasticsearch. Now you can view it in a browser based on Elasticsearch's Head view, but Head can't effectively analyze and present the data just because it can be viewed simply. If you want to analyze the data and show it well, you need to use Kibana,Kibana to explain it in the next article, and the architecture diagram is shown here.

The above is all the contents of the article "how to install and use Filebeat in Linux". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.