Shulou(Shulou.com)06/03 Report--

Filebeat has a lot of built-in modules that can be used directly to collect logs. The supported modules are as follows:

[root@ELK-chaofeng07 logstash]# filebeat modules listEnabled:mysqlDisabled:apache2auditdelasticsearchhaproxyicingaiiskafkakibanalogstashmongodbnginxosquerypostgresqlredissuricatasystemtraefik

As long as it is the filebeat that appears above, it is supported, but you need to set it before you can use it, such as:

filebeat modules enable mysql

So this time let's look at how to use mysql module to collect mysql slow query logs and error logs

1. First, make filebeat support mysql module

filebeat modules enable mysql

2. Enable output to elasticsearch and kibana in the configuration file/etc/filebeat/filebeat.yml.

After the settings are complete, save it. In addition, your kibana and elasticsearch clusters need to be up and running to move forward.

3. Start preparing the predefined environment

filebeat setup -e

If the error is reported, it needs to be repaired according to the situation.

The requested URL/etc/filebeat/modules.d/mysql.yml was not found on this server.

5. After the above is completed, it is almost the same. At this time, we can start the filebeat service.

If you start with the following error, it means that there are multiple output sources in your filebeat.yml configuration file. In this case, you need to comment out all other outputs and leave only one source output to elasticsearch.

Exiting: Template loading requested but the Elasticsearch output is not configured/enabled

You will see this index in elasticsearch cluster:

7. The next step is to go to the kibana interface. Here, you don't need to create an index in advance. Just click on the dashboard interface and select mysql. For example:

Finally, you use a few slow queries and insert some error logs into the log file of the execution to see the effect:

That's how it's done.

This practice is my reference to the official website of the document to do, the details link is: www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-mysql.html#mysql-settings

Summary: I don't think the above method can be widely used because it lacks logstash filtering step, so we can't use aggregation and other graphics very well. In addition, there is no broker, which will cause a lot of output to elasticsearch and cause blocking. My solution is to use logstash filter plus built-in custom module is better.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.