Shulou(Shulou.com)06/03 Report--

Log segmentation

As the number of visits to the website increases, the single log file for Apache becomes larger and larger by default.

The log file takes up a lot of disk space to view related information. It is not convenient to split the log file. Apache comes with rotatelogs split tool to implement third-party tool cronolog split rotatelogs split configuration the log file of the website is transferred to rotatelogs split processing configuration format is ErrorLog 4 | absolute path of the rotatelogs command-I log file path / website name-error% Y%m%d.log 86400 "- CustomLog" | rotatelogs command path-I log File path / website name-access% Y%m%d.log 86400 "combined matching instance [root@localhost ~] # yum install httpd-y / / install http service loaded plug-in: fastestmirror LangpacksLoading mirror speeds from cached hostfile * base: centos.ustc.edu.cn * extras: centos.ustc.edu.cn...// omits part of the content. Installed: httpd.x86_64 0 2.4.6-90.el7.centos. / / omit part of the content. Over! [root@localhost bin] # cd / usr/sbin/ enter the sbin directory [root@localhost sbin] # ls rotat* / / check whether a command is generated under the directory rotatelogs / / after installing the http service, generate the rotatelogs command [root@localhost sbin] # cd / var/log/httpd/ enter the http service log directory [root@localhost httpd] # ls / / check whether the log file [root@localhost httpd] # / / does not generate the directory file [root@localhost httpd] # vim / etc/httpd/conf/httpd.conf / / enter the http configuration file after the installation is completed Make general configuration. / / omit part of the content. # prevent Apache from glomming onto all bound IP addresses.#Listen 192.168.144.133 Listen 80 / / modify IPv4 to local address # Listen 80 / / comment out IPv6 snooping. / / omit part of the content. # If your host doesn't have a registered DNS name Enter its IP address here.#ServerName www.kgc.com:80 / / enable domain name service # # Deny access to the entirety of your server's filesystem. You must...// omitted part of the content.: wq [root@localhost httpd] # systemctl stop firewalld.service / / turn off the firewall [root@localhost httpd] # setenforce 0 / turn off the enhanced security function [root@localhost httpd] # systemctl start httpd.service / / start the http service [root@localhost httpd] # ls / / check whether the log file access_log error_log is generated / / after the service is enabled, the log file [root@localhost httpd] # vim conf/httpd.conf / / is successfully generated to change the master configuration file. Do log segmentation. / / omit part of the content. # container, that host's errors will be logged there and not here.##ErrorLog "logs/error_log" / / copy this entry and paste the entry below ErrorLog "| / usr/sbin/rotatelogs-l logs/www.kgc.com.error_%Y%m%log 86400" / / change entry, reference the rotatelogs command to create a split log entry name Create a file every 86400 seconds (a day) # LogLevel: Control the number of messages logged to the error_log....// omits part of the content. # (Combined Logfile Format) you can use the following directive. # # CustomLog "logs/access_log" combined / / copy this entry, paste the entry below CustomLog "| / usr/sbin/rotatelogs-l logs/www.kgc.com.access_%Y%m%dlog 86400" combined / / change the entry, quote the rotatelogs command, and create a split log entry name Create a file every 86400 seconds. / / omit part of the content.: wq [root@localhost httpd] # systemctl stop httpd.service / / stop http service [root@localhost httpd] # systemctl start httpd.service / / start http service [root@localhost httpd] # cd. [root@localhost log] # cd ~ [root@localhost ~] # cd / var/log/httpd/ re-enter the site file [root@ Localhost httpd] # lsaccess_log error_log www.kgc.com.error_20191105log / / display the split log file [root@localhost httpd] # date / / View time on Tuesday, November 5, 2019, 19:27:50 CST [root@localhost httpd] # date-s 11-06-19 / / set the time one day after Wednesday, 06 November 2019, 00:00:00 CST [root@localhost httpd] # Systemctl stop httpd.service / / close the http service [root@localhost httpd] # systemctl start httpd.service / / start the service [root@localhost httpd] # cd ~ / / return to the home directory [root@localhost ~] # cd / var/log/httpd/ re-store today's log directory [root@localhost httpd] # ls / / View access_log error_log www.kgc.com.error_20191105log www.kgc .com.error _ 20191106log / / generate new log files third-party tools cronolog source code compilation installation cronolog tools cronolog installation package download configuration website log files are transferred to cronolog split processing configuration format: ErrorLog "| absolute path of the cronolog command log file path / website name-error_% Y%m%d.log'CustomLog" | absolute path of the cronolog command | log file path / site name% Y%m%d.log "combined configuration instance

Restart a Linux virtual machine and reinstall the http service

[root@localhost ~] # yum install httpd-y / / the installation service has loaded the plug-in: fastestmirror, langpacksLoading mirror speeds from cached hostfile * base: centos.ustc.edu.cn...// omits part of the content. Installed: httpd.x86_64 0root@localhost 2.4.6-90.el7.centos... / / omit some contents. [root@localhost ~] # mount.cifs / / 192.168.100.10/lamp-c7 / mnt/ / remote mount directory Password for root@//192.168.100.10/lamp-c7: [root@localhost ~] # cd / mnt/ enter the mount directory [root@localhost Mnt] # ls / / check whether apr-1.6.2.tar.gz cronolog-1.6.2-14.el7.x86_64.rpm httpd-2.4.29.tar.bz2 mysql-5.6.26.tar.gzapr-util-1.6.0.tar.gz Discuz_X2.5_SC_UTF8.zip LAMP-php5.6.txt nginx-1.12.0.tar.gzawstats- is mounted successfully 7.6.tar.gz error.png miao.jpg php-5.6.11.tar.bz2 [root@localhost mnt] # rpm-ivh cronolog-1.6.2-14.el7.x86_64.rpm / / installation package warning: cronolog-1.6.2-14.el7.x86_64.rpm: header V3 RSA/SHA256 Signature Key ID 352c64e5: NOKEY in preparation. # # [100%] upgrading / installing... 1:cronolog-1.6.2-14.el7 # # # [root@localhost mnt] # cd / usr/sbin/ enter the directory [root@localhost sbin] # ls cronolog* / / check whether the cronolog command cronolog / / successfully installed [root@localhost sbin] # vim / etc/httpd/conf/httpd.conf / / enter the editing configuration file . / / omit part of the content. # prevent Apache from glomming onto all bound IP addresses.#Listen 192.168.144.133 Dynamic Shared Object 80 / / set the listening address # Listen 80 # Dynamic Shared Object (DSO) Support...// omit part of the content. # If your host doesn't have a registered DNS name Enter its IP address here.#ServerName www.kgc.com:80 / / set the domain name # # Deny access to the entirety of your server's filesystem. You must...// omits part of the content. # container, that host's errors will be logged there and not here.#ErrorLog "/ usr/sbin/cronolog logs/www.kgc.com.error_%Y%m%dlog" / / call the cronolog command to create an error log file # # LogLevel: Control the number of messages logged to the error_log....// omits part of the content. # (Combined Logfile Format) log. # CustomLog "| / usr/sbin/cronolog logs/www.kgc.com.access_%Y%m%dlog" combined / / call the cronolog command to create an access log file. / / omit part of the contents.: wq [root@localhost sbin] # systemctl stop firewalld.service / / turn off the firewall [root@localhost sbin] # setenforce 0 / / turn off the enhanced security feature [root@localhost Sbin] # systemctl start httpd.service / / enable the http service [root@localhost sbin] # cd / var/log/httpd/ enter the log file directory [root@localhost httpd] # ls / / View www.kgc.com.error_20191105log / / generate a log partition file [root@localhost httpd] # date / / View the current time, Tuesday, November 5, 20119, 20:11:57 CST [root@localhost httpd] # date-s 11-06-19 / / set the time to the following day, Wednesday, November 6, 2019, at 00:00:00 CST [root@localhost httpd] # systemctl stop httpd.service / / stop the http service [root@localhost httpd] # systemctl start httpd.service / / start the http service [root@localhost httpd] # cd ~ / / return to the home directory [root@localhost] # cd / var/log/httpd/ re-enter the log directory [root@localhost httpd] # ls / / View www.kgc.com.error_20191105log www.kgc.com.error_ 20191106log / / generate new log files AWStats log analysis system introduction AWStats log analysis system an open source log analysis system developed by Perl language can be used to analyze Apache, The access log information of Samba, Vsftpd, IIS and other servers is combined with scheduled task services such as crond Log content can be regularly analyzed and deployed AWStats log analysis system restarts a Linux virtual machine, installs http and DNS services, and makes general configuration [root@localhost ~] # yum install bind httpd-y loaded plug-ins: fastestmirror, langpacksLoading mirror speeds from cached hostfile * base: mirrors.163.com...// omits some of the contents. Installed: bind.x86_64 32root@localhost 9.11.4-9.P2.el7 httpd.x86_64 090.el7.centos 2.4.6-90.el7.centos. / / omit part of the content. [root@localhost ~] # vim / etc/named.conf. / / omit part of the content. Options {listen-on port 53 {any;}; listen-on-v6 port 53 {:: 1;} Directory "/ var/named"; dump-file "/ var/named/data/cache_dump.db"; statistics-file "/ var/named/data/named_stats.txt"; memstatistics-file "/ var/named/data/named_mem_stats.txt"; recursing-file "/ var/named/data/named.recursing"; secroots-file "/ var/named/data/named.secroots" Allow-query {any;};. / / omit part of the content... [root@localhost ~] # vim / etc/named.rfc1912.zones...// omit part of the content... zone "kgc.com" IN {type master; file "kgc.com.zone"; allow-update {none;};} ... omit part of the content. [root@localhost named] # cp-p named.localhost kgc.com.zone [root@localhost named] # vim kgc.com.zone $TTL 1D @ IN SOA @ rname.invalid. (0; serial 1D; refresh 1H; retry 1W; expire 3H) Minimum NS @ A 127.0.0.1www IN A 192.168.144.133:wq [root@localhost named] # vim / etc/httpd/conf/httpd.conf...// omitted part... # prevent Apache from glomming onto all bound IP addresses.#Listen 192.168.144.133:80#Listen 80th # Dynamic Shared Object (DSO) Support...// omitted part... # If your host doesn't have a registered DNS name Enter its IP address here.#ServerName www.kgc.com:80## Deny access to the entirety of your server's filesystem. You must...// omits part of the content.: wq [root@localhost named] # systemctl stop firewalld.service [root@localhost named] # setenforce 0 [root@localhost named] # systemctl start named [root@localhost named] # systemctl start httpd.service tests whether the website is accessible in the client

Install the AWStats package [root@localhost named] # cd / etc/httpd/ [root@localhost httpd] # mount.cifs / / 192.168.100.10/lamp-c7 / mnt/ / Mount the software storage directory in the mnt directory Password for root@//192.168.100.10/lamp-c7: / / directly enter [root@localhost httpd] # cd / mnt/ / / View the hanging directory [root@localhost mnt] # lsapr-1.6.2.tar.gz cronolog-1.6.2-14.el7.x86_64.rpm httpd-2.4.29.tar.bz2 mysql-5.6.26.tar.gzapr-util-1.6.0.tar.gz Discuz_X2.5_SC_UTF8.zip LAMP-php5.6.txt Nginx-1.12.0.tar.gzawstats-7.6.tar.gz error.png miao.jpg php-5.6.11.tar.bz2 [root@localhost mnt] # tar zxvf awstats-7.6.tar.gz-C / opt/ decompress the awstats tool in the opt directory awstats-7.6/ awstats-7.6/tools/awstats-7.6/ Tools/awstats_buildstaticpages.pl...// omits part of the content. [root@localhost mnt] # cd / opt/ enter the opt directory [root@localhost opt] # lsawstats-7.6 rh [root@localhost opt] # mv awstats-7.6/ / usr/local/awstats / / move the extracted files to / usr/ [root@localhost opt] # cd / usr/local/ enter the local directory [root@localhost local] # lsawstats bin etc games include lib lib64 libexec sbin share src [root@localhost local] # cd awstats/ in the local/ directory [root@localhost awstats] # lsdocs README.md tools wwwroot [root@localhost awstats] # cd tools/ enter the tool store Put in the directory [root@localhost tools] # ls / / View the executable tool awstats_buildstaticpages.pl awstats_updateall.pl httpd_conf nginx xsltawstats_configure.pl dolibarr logresolvemerge.pl urlaliasbuilder.plawstats_exportlib.pl geoip_generator.pl maillogconvert.pl webmin to create a configuration file for the site to be counted [root@localhost tools] #. / awstats_ Configure.pl / / Executive configuration tool-AWStats awstats_configure 1.0 (build 20140126) (c) Laurent Destailleur-This tool will help you to configure AWStats to analyze statistics forone web server. You can try to use it to let it do all that is possible...// omits some contents... Enter full config file path of your Web server.Example: / etc/httpd/httpd.confExample: / usr/local/apache2/conf/httpd.confExample: C:\ Program files\ apache group\ apache\ conf\ httpd.confConfig file path ('none' to skip web server setup): > / etc/httpd/conf/httpd.conf / / enter the path to the http profile. / / omit part of the content.-> Need to create a new config file? Do you want me to build a new AWStats config/profilefile (required if first install) [y Do you want me to build a new AWStats config/profilefile N]? Y / / enter y confirmation-> Define config file name to createWhat is the name of your web site or profile analysis? Example: www.mysite.comExample: demoYour web site Virtual server or profile name: > www.kgc.com / / enter the web page domain name-> Define config file pathIn which directory do you plan to store your config file (s)? Default: / etc/awstats Directory path to store config file (s) (Enter for default): > / / enter directly By default, you can store the configuration file path. / / omit part of the content. Or if you have several config files and prefer having only one command:/usr/local/awstats/tools/awstats_updateall.pl nowPress ENTER to continue... A SIMPLE config file has been created: / etc/awstats/awstats.www.kgc.com.confYou should have a look inside to check and change manually main parameters.You can then manually update your statistics for 'www.kgc.com' with command: > perl awstats.pl-update-config=www.kgc.comYou can also read your statistics for' www.kgc.com' with URL: > http://localhost/awstats/awstats.pl?config=www.kgc.com / / gives the analysis access URL Press ENTER to Finish... [root@localhost tools] # cd / etc/httpd/conf / / enter the http configuration file directory [root@localhost conf] # vim httpd.conf / / Edit configuration file. / / omit part of the content. # This is to permit URL access to scripts/files in AWStats directory.# / / automatically generate configuration file Options None AllowOverride None # Order allow Deny / / comment out the access permission setting # Allow from all Require all granted / / add entries that everyone can access: wq modifies the site statistics configuration file [root@localhost etc] # cd / etc/awstats/ enter the configuration awstats configuration file directory [root@localhost awstats] # ls / / View awstats.www.kgc. Com.conf / / awstats configuration file [root@localhost awstats] # ls/ var/log/httpd/ View http service log file name access_log error_ log [root @ localhost awstats] # vim awstats.www.kgc.com.conf / / enter the awstats configuration file. / / omit part of the content. # Example: "/ pathtotools/logresolvemerge.pl *. Log |" # LogFile= "/ var / log/httpd/access_log "/ / found secondary entry Change the name of the log file to be analyzed and # Enter the log file type you want to analyze....// can omit part of the content. # Example: "C:/awstats_data_dir" # Default: "." (means same directory as awstats.pl) # DirData= "/ var/lib/awstats" / / find this entry Check the location of the analyzed data # Relative or absolute web URL of your awstats cgi-bin directory....// omitted part of the content.: wq [root@localhost awstats] # cd / var/lib/ enter the lib directory [root@localhost lib] # mkdir awstats / / create the awstats analysis data storage directory [root@localhost lib] # ls / / View the directory is Failed to create AccountsService chrony flatpak initramfs misc PackageKit rpm-state systemd yumalsa color fprint ipa-client mlocate plymouth rsyslog tpmalternatives colord games iscsi net-snmp polkit-1 samba tunedauthconfig dav gdm libvirt NetworkManager postfix selinux udisks2awstats dbus geoclue lldpad nfs pulse setroubleshoot upowerbluetooth successfully Dhclient gssproxy logrotate ntp rpcbind sss vmwarecertmonger dnsmasq hyperv machines os-prober rpm stateless xkb [root@localhost lib] # systemctl restart httpd.service / / restart the http service

Test the awstats tool in the client

The awstats visited at this time did not do statistical analysis. We need to execute the update data tool to perform statistics [root@localhost lib] # cd / usr/local/awstats/ enter the awstats tool catalog [root@localhost awstats] # lsdocs README.md tools wwwroot [root@localhost awstats] # cd tools/ enter the tool catalog [root@localhost tools] # lsawstats_buildstaticpages.pl awstats_updateall.pl httpd_conf nginx xsltawstats_configure.pl dolibarr Logresolvemerge.pl urlaliasbuilder.plawstats_exportlib.pl geoip_generator.pl maillogconvert.pl webmin [root@localhost tools] #. / awstats_updateall.pl now / / execution tool for update Don't forget to add nowRunning'"/ usr/local/awstats/wwwroot/cgi-bin/awstats.pl"-update-config=www.kgc.com-configdir= "/ etc/awstats"'to update config www.kgc.comCreate/Update database for config "/ etc/awstats/awstats.www.kgc.com.conf" by AWStats version 7.6 (build 20161204) From data in log file "/ var/log/httpd/access_log"... Phase 1: First bypass old records Searching new record...Searching new records from beginning of log file...Phase 2: Now process new records (Flush history on disk after 20000 hosts)... Jumped lines in file: 0Parsed lines in file: 48 Found 0 dropped records, Found 0 comments, Found 0 blank records, Found 0 corrupted records, / / successfully updated Found 0 old records, Found 48 new qualified records. Revisit awstats on the client

Set up periodic scheduled tasks and automatically count access information And optimize the access path [root@localhost tools] # crontab-e / / set periodic scheduled task * / 5 * / usr/local/awstats/tools/./awstats_updateall.pl now / / set to execute every 5 minutes ~: wq [root@localhost tools] # systemctl start crond / / start periodic scheduled task [root@localhost tools] # cd / var/www/html/ / / enter the http site directory [root@localhost html] # vim aws.html / / Edit the page / / Edit the page content: wq / / Save the exit to test whether the optimized access path can be used in the client

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.