How to use filter in LogStash
LogStash filter how to use, many novices are not very clear about this, in order to help you solve this problem, the following small series will explain in detail for everyone, there are people who need this can learn, I hope you can gain something.
grok
Custom grok format
In the conf file folder sibling directory, usually in the patterns folder, create your own pattern file, such as the extra file.
# contents of ./ patterns/postfix:
POSTFIX_QUEUEID [0-9A-F]{10,11}
For example, for log format: Jan 1 06:25:43 mailserver14 postfix/cleanup[21403]: BEF25A72965: message-id= "./ patterns"
match => [ "message", "%{SYSLOGBASE} %{POSTFIX_QUEUEID:queue_id}: %{GREEDYDATA:syslog_message}" ]
}
}
The results are:
correct matching
2. Some of the more magical Grok patterns
For example: GREEDYDATA .*
Note: If I don't want to match everything, use Greedidata directly.
Example: Match, 52:awfasdf0r8b123e:222g1g16:885579b2:9afr
Add %{GREEDYDATA:sth} to the grok of filter to match the above string. If you want more detailed matching, write your own regular, matching each item.
Practical examples 1. Want to handle java log information, including normal info and error stack information how to deal with?
Thinking:
In filter, use multiline to group log information; then use grok to split it.
Things that need attention may encounter newline problems. What to do at this time?
The processing at this time is such that using GREEDYDATA is invalid because it is only a match.* And... is not matching newline, at this time, GreedyData should be replaced with:
(? (.|\ r| * ** Did reading the above help you? If you still want to have further understanding of related knowledge or read more related articles, please pay attention to the industry information channel, thank you for your support.
