Shulou(Shulou.com)05/31 Report--

This article introduces the relevant knowledge of "what is the method of Nginx log analysis and performance troubleshooting". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

If you need to get the processing time of the request, you need to add $request_time to the nginx log. Here is my log_format.

Nginx.conf

Log_format main'$remote_addr-$remote_user [$time_local] "$request"'

'$status $body_bytes_sent $request_body "$http_referer''

'"$http_user_agent"$http_x_forwarded_for"$request_time"'

Restart nginx after modification, and when you check the nginx log, you can see the time taken by nginx to process the request, which is basically the time spent by the backend, so you can get slow requests according to this field.

Here are some of the commands I used.

Get the number of pv

$cat / usr/local/nginx/logs/access.log | wc-l

Get the number of ip

$cat / usr/local/nginx/logs/access.log | awk'{print $1}'| sort-K1-r | uniq | wc-l

Get the most time-consuming request time, url, and time-consuming. You can modify the following numbers to get more. If you don't add them, you can get all of them.

$cat / usr/local/class/logs/access.log | awk'{print $4 head 7 camera NF}'| awk-F'"'{print $1 camera 2 camera 3}'| sort-K3-rn | head-10

To get the number of requests at a certain time, you can remove seconds to get minute data, minutes to get hour data, and so on.

$cat / usr/local/class/logs/access.log | grep 2017, 13, 28, and 55 | wc-l

Get the number of requests per minute, output it into a csv file, and then open it with excel to generate a bar chart

$cat / usr/local/class/logs/access.log | awk'{print substr ($4Jet 14jue 5)}'| uniq-c | awk'{print $2 "," $1}'> access.csv

The above figure is generated by excel, and you can also use the command line tool gnuplot to generate png. I also tried it. There is no problem. It is convenient to get the report directly in the form of programming. It is very convenient to remove the manual operation part, but one thing is that when there is a lot of x-axis data, it can not automatically dilute the data like excel, so I still like to use excel to generate.

In fact, there are only a few commands to be used:

Cat: entering file contents

Grep: filtering text

'sort by sort':

Uniq': de-weighting

'awk': text processing

Command combination, a single command can be used multiple times, to achieve the effect of multiple filtering, the output of the previous command is the input of the latter command, streaming, as long as you learn this command, how complex things seem to become extremely simple.

The above are all commands. Let's introduce a direct output of html, which is actually using go-access to analyze nginx logs.

Cat / usr/local/nginx/logs/access.log | docker run-- rm-I diyan/goaccess-- time-format='%H:%M:%S'-- date-format='%d/%b/%Y'-- log-format='%h% ^ [% DV% t% ^] "% r"% s% b "% R"% u"'> index.html

Go-access runs as a docker container. As long as you install docker, you can run it directly. It is very convenient without installation.

The above script, together with the daily log division of the log, and then configure the auto-run script in crontab to generate nginx reports every day. The situation of the website is clear, of course, there are shortcomings here, because it is not real-time.

To count real-time data, you can use ngxtop to view it, and it's easy to install.

$pip install ngxtop

If running, advance to the nginx directory, and then run,-c specify the configuration file,-t refresh rate, in seconds

$cd / usr/local/nginx

$ngxtop-c conf/nginx.conf-t 1

But this real-time way also requires ssh remote login, which is not very convenient, and you can also use lua for real-time statistics, and then write an interface to display the data, which can be used through lua-nginx-module,nginx/tengine. If you install openresty directly, it is convenient, embedded with lua, and there is no need to recompile nginx.

This is the end of the content of "what is the method of Nginx log analysis and performance troubleshooting". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.