Shulou(Shulou.com)06/02 Report--

Nginx: PV, UV, independent IP

People who do websites know that they usually have to inquire about the visit data of websites such as PV and UV. Of course, if the website has done CDN, the local log of nginx is meaningless. Here is the statistics on the log access data of nginx website.

Concept:

UV (Unique Visitor): a unique visitor who treats each separate Internet computer (based on cookie) as a visitor, the number of visitors to your website in a day (00:00-24:00). Visits to the same cookie are counted only once in a day PV (Page View): the number of visits, that is, page views or clicks, is recorded every time a user visits the site. User visits to the same page multiple times, cumulative statistics of visit value independent IP:00:00-24:00 the same IP address is only calculated once, friends who do website optimization are most concerned about this.

First, declare the environment. Nginx version 1.7 runs this time, and the back-end Tomcat runs a dynamic interactive program (user authentication is required. If it is a static page, the cache value cannot be caught, and $http_cookie is null). That's it.

Nginx log file configuration

Http {include mime.types; default_type application/octet-stream; log_format main'$remote_addr-[$time_local] "$request"'- $status "User_Cookie:$guid"; # User_Cookie is a log display character, $guid is a variable, and the details are defined below. You can also write $http_cookie in the log format to display the complete cookie content.

Sendfile on; keepalive_timeout 65; upstream backserver {ip_hash; server 1.1.2.2 server 8080; server 1.1.2.3 server 8080;} server {listen 80; server_name localhost; # if ($http_cookie ~ * "(. *) $") matches all content if ($http_cookie ~ * "CSID= ([A-Z0-9] *)") {set $guid $1 } # matches only CSID character information, here is a regular expression

Access_log logs/host.access.log main; location ~ * ^ (. *) ${# limit_req zone=allips burst=1 nodelay; proxy_pass http://backserver; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header REMOTE-HOST $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; client_max_body_size 8m } error_page 500 502 503 504 / 50x.htl; location = / 50x.html {root html;}}

Note: the value in $http_cookie is a cookie value, separated by ";"

Log output format

192.168.40.2-[02/Nov/2016:15:44:35 + 0800] "GET / wcm/app/main/refresh.jsp?r=1478072325778 HTTP/1.1"-200 "User_Cookie:7F00000122A5597C46607B1C0A7EC016"

192.168.40.2-[02/Nov/2016:15:44:35 + 0800] "GET / webpic/W0201611/W020161102/W020161102566715167404.jpg HTTP/1.1"-200 "User_Cookie:7F00000122A5597C46607B1C0A7EC016"

119.255.31.109-[02/Nov/2016:15:44:36 + 0800] "GET / wcm/app/main/refresh.jsp?r=1478072510132 HTTP/1.1"-200 "User_Cookie:7F000001237921BE9237838AEC65704D"

119.255.31.109-[02/Nov/2016:15:44:36 + 0800] "GET / wcm/app/message/message_query_service.jsp?READFLAG=0&MSGTYPES=1%2C2%2C3 HTTP/1.1"-200 "User_Cookie:7F000001237921BE9237838AEC65704D"

192.168.40.2-[02/Nov/2016:15:44:37 + 0800] "GET / wcm/app/message/message_query_service.jsp?READFLAG=0&MSGTYPES=1%2C2%2C3 HTTP/1.1"-200 "User_Cookie:7F00000123D3BF2345115EAAC21F71E0"

192.168.40.2-[02/Nov/2016:15:44:37 + 0800] "GET / wcm/app/message/message_query_service.jsp?READFLAG=0&MSGTYPES=1%2C2%2C3 HTTP/1.1"-200 "User_Cookie:7F00000123EF73896DF98EDA9950944E"

192.168.40.2-[02/Nov/2016:15:44:37 + 0800] "GET / wcm/app/message/message_query_service.jsp?READFLAG=0&MSGTYPES=1%2C2%2C3 HTTP/1.1"-200 "User_Cookie:7F00000123FE0F9C397E1A8F0C4F044B"

192.168.40.2-[02/Nov/2016:15:44:37 + 0800] "GET / wcm/app/main/refresh.jsp?r=1478072511427 HTTP/1.1"-200 "User_Cookie:7F00000123A465B7EA1DE0AF0AE671B7"

119.255.31.109-[02/Nov/2016:15:44:38 + 0800] "GET / wcm/app/message/message_query_service.jsp?READFLAG=0&MSGTYPES=1%2C2%2C3 HTTP/1.1"-200 "User_Cookie:7F00000123D89B11302DF80AE773C900"

PV statistics

You can count the number of visits to a single link address:

[root@localhost logs] # grep index.shtml host.access.log | wc-l

Total PV:

[root@localhost logs] # awk'{print $6} 'host.access.log | wc-l

Independent IP

[root@localhost logs] # awk'{print $1} 'host.access.log | sort-r | uniq-c | wc-l

UV statistics

[root@localhost logs] # awk'{print $10} 'host.access.log | sort-r | uniq-c | wc-l

Cookie Test Page

For species cookie, you can use the following html code to edit and add the cookie that needs to be planted

# index.html / / to facilitate testing, the page is refreshed every 10 seconds. The test.test.com domain test lists the cookie for that domain below

[xss_clean] = "guid=A1UD8E5512451111111111"; / / cookie, append [xss_clean] = "city=beijing"; / / cookie, append [xss_clean] ([xss_clean]); / / list existing ones

The above is the whole content of this article, I hope it will be helpful to your study, and I also hope that you will support it.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.