Shulou(Shulou.com)06/01 Report--

This article is about how to implement an automated operation and maintenance system in Logstash. The editor thinks it is very practical, so I share it with you. I hope you can get something after reading this article. Let's take a look at it.

1. Nginx log configuration

Edit / etc/nginx/nginx.conf add the following

Log_format json'{"@ timestamp": "$time_iso8601",''"host": "$host",''"scheme": "$scheme",''"server_addr": "$server_addr",''"client_ip": "$remote_addr",''"server_protocol": "$server_protocol",''"method": "$request_method" '' "query_string": "$query_string", "body_bytes_sent": $body_bytes_sent,''"bytes_sent": $bytes_sent,''"request_length": $request_length,''"request_time": $request_time,''"upstream_time": "$upstream_response_time", "upstream_host": "$upstream_addr" "upstream_status": "$upstream_status", "server_name": "$server_name", "url": "$uri", "request_url": "$request_uri", "http_x_forwarded_for": "$http_x_forwarded_for", "referer": "$http_referer", "agent": "$http_user_agent" '' "status": $status}' Access_log / home/nginx/log/access.log json;2.2 rsyslog configuration

Use rsyslog to push the log to Logstash, edit / etc/rsyslog.d/nginx.conf, enter the following, and remember to restart the rsyslog service when you are finished.

ModLoad imfile$Modload mmjsonparseaction (type= "mmjsonparse") template (name= "nginx-json" type= "list") {property (name= "$! all-json")} $InputFileName / home/nginx/log/access.log$InputFileTag nginx-access$InputFileStateFile nginx-accessfile$InputFileSeverity info$InputFileFacility local7 $InputRunFileMonitor$InputFilePersistStateInterval 1if $syslogtag startswith 'nginx' then @ @ 10.160.209.10 Server configuration 3.1 Logstash configuration

Create a new Logstash configuration named metric_nginx.conf with the following content

Input {syslog {host = > "10.160.209.10" port = > "514" codec = > "json"}} filter {json {source = > "msg"} mutate {convert = > ["request_time" Float]} mutate {remove_field = > ["msg"] remove_field = > ["@ version"] remove_field = > ["port"] remove_field = > ["facility"] remove_field = > ["priority"] remove_field = > ["severity"] remove_field = > ["severity_label"] remove_field = > ["facility_label" ]} metrics {meter = > "error.% {status}" # set up counters according to different status values add_tag = > "metric" ignore_older_than = > 10}} output {if "metric" in [tags] {if [error.504] [rate_1m] > 0.0 {# alarm stdout {when it reaches the set threshold Codec = > line {format = > "alarm:% {[error.504] [rate_1m]}" # Analog alarm}}

Then use the following command to start the Logstash service

Logstash-f metric_nginx.conf

For the definition of the [rate_1m] counter above, please refer to: https://www.elastic.co/guide/en/logstash/current/plugins-filters-metrics.html#_literal_meter_literal_values

4. Operation and maintenance Automation Model based on Logstash Model

After introducing the above example, I believe you can see that the role of Logstash is much more than that. After you are familiar with the log processing process of Logstsh, you can achieve a set of automatic operation and maintenance platform based on log driver. As shown in the figure below, the logs of various services such as Mon and OSD can be connected to Logtash. After Logstash processing, with the help of different output plug-ins, you can selectively carry out Alarm alarms, trigger Ansible playbook operations, or store them in message middleware such as redis and kafka to open data channels with other business systems. Especially when your operation and maintenance staff are familiar with all kinds of log anomalies in Ceph, you can handle some online failures automatically.

The above is how to implement an automated operation and maintenance system in Logstash. The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.