In addition to Weibo, there is also WeChat
Please pay attention
WeChat public account
Shulou
2025-02-24 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Servers >
Share
Shulou(Shulou.com)06/02 Report--
Preface
Log files are extremely important tools for Linux system security. For some reason, I found that many operation and maintenance students have CRON scripts running on their servers, such as slicing Nginx logs every day. Everyone seems to have forgotten Logrotate and scrambled to invent their own wheels, which is really frustrating. It's like lying next to a ready-made sexy beauty, but everyone is busy entertaining themselves, sin! The logrotate program is a log file management tool. Used to split log files, delete old log files, and create new log files, play the role of "dump". Disk space can be saved.
The following is a summary of the logrotate log rotation operation:
1) profile introduction
The logrotate tool is installed by default on the Linux system, and its default configuration file is:
/ etc/logrotate.conf
/ etc/logrotate.d/
Logrotate.conf is the main configuration file. Logrotate.d is a directory in which all files are actively read into / etc/logrotate.conf.
In addition, if no details are set in the file in / etc/logrotate.d/, the settings in the file / etc/logrotate.conf will be used as the default value.
Logrotate runs based on CRON, its script is / etc/cron.daily/logrotate, and log rotation is done automatically by the system.
When actually running, Logrotate invokes the configuration file / etc/logrotate.conf.
You can place a custom configuration file in the / etc/logrotate.d directory to override the default value of Logrotate.
[root@huanqiu_web1 ~] # cat / etc/cron.daily/logrotate #! / bin/sh / usr/sbin/logrotate / etc/logrotate.conf > / dev/null 2 > & 1EXITVALUE=$?if [$EXITVALUE! = 0]; then / usr/bin/logger-t logrotate "ALERT exited abnormally with [$EXITVALUE]" fiexit 0
If you can't wait for cron to automatically perform log rotation, you need to add the-f parameter if you want to force the log to be cut manually. However, you'd better use the Debug option to verify (- d parameter) before formal execution, which is also important for debugging.
# / usr/sbin/logrotate-f / etc/logrotate.d/nginx# / usr/sbin/logrotate-d-f / etc/logrotate.d/nginx
Logrotate command format:
Logrotate [OPTION...]-d,-- debug: debug mode to test the configuration file for errors. -f,-- force: forcibly dump files. -m,-- mail=command: after compressing the log, send the log to the specified mailbox. -s,-- state=statefile: uses the specified status file. -v,-- verbose: shows the dump process.
Operate according to log cutting settings and display details
[root@huanqiu_web1] # / usr/sbin/logrotate-v / etc/logrotate.conf [root@huanqiu_web1 ~] # / usr/sbin/logrotate-v / etc/logrotate.d/php
It is executed according to the log cutting settings, and the details are displayed, but no specific operation is performed, debug mode
[root@huanqiu_web1] # / usr/sbin/logrotate-d / etc/logrotate.conf [root@huanqiu_web1 ~] # / usr/sbin/logrotate-d / etc/logrotate.d/nginx
Check the specific implementation of each log file
[root@fangfull_web1 ~] # cat / var/lib/logrotate.status
2) introduction to cutting
For example, use Syslog / var/log/message to make a simple explanation:
After performing the rotate for the first time, the original messages becomes messages.1 and an empty messages is created for the system to store logs
After the second execution, the messages.1 becomes messages.2 and the messages becomes messages.1, creating an empty messages to store the log!
If you set to keep only three logs (that is, rotate 3 times), then on the third execution, the messages.3 file will be deleted and replaced by a newer save log later! That is, the latest logs will be saved.
How many times the log is rotated is determined by the dateext parameter in the configuration file.
Take a look at logrotate.conf configuration:
The setting under # cat / etc/logrotate.conf# is "the default value of logrotate". If other files set other values, # will use the settings of other files as the main weekly / / default to perform rotate rotation once a week rotate 4 / / how many log files to keep (several times). Four are reserved by default. Is to specify the number of rotations before the log file is deleted. 0 means no backup create / / automatically creates a new log file, and the new log file has the same permissions as the original file; because the log has been renamed, it is important to create a new one to continue to store the previous log dateext / /. The cut log file ends in the current date format, such as xxx.log-20131216. If the log file is commented out, it is incremented by number, that is, whether the xxx.log-1 format compress / / is compressed and dumped by gzip, such as xxx.log-20131216.gz. If you don't need compression Comment out the line include / etc/logrotate.d# loads all the files in the / etc/logrotate.d/ directory / var/log/wtmp {/ / only for the parameter set by / var/log/wtmp monthly / / cut once a month Instead of the default weekly minsize 1m / / file size exceeding 1m, create 0664 root utmp / / specifies the permissions for newly created log files and the user and group rotate 1 / / keeps only one log.} # this wtmp records the time when the user logs in to the system and the system restart # because of the parameters of minsize So it doesn't have to be carried out once a month. Depends on the size of the file.
From the settings of this file, we can know that / etc/logrotate.d is actually the directory planned by / etc/logrotate.conf. Although all configurations can be written into / etc/logrotate.conf, this file is really too complex, especially when using a lot of services on the system, it seems unreasonable for each service to modify the settings of / etc/logrotate.conf.
So, if there is a separate directory, then each service that wants to cut the log can be a separate file and placed in / etc/logrotate.d/
Other important parameters indicate
-
Compress uses gzip compression to dump logs nocompress does not do gzip compression processing copytruncate is used to backup and truncate the current log files that are still open. It is a way to copy and then empty. There is a time difference between copy and emptying, and some log data may be lost. Nocopytruncate backup log files only specify the attributes to create new files during create mode owner group rotation. For example, when create 0777 nobody nobodynocreate does not create new log files, delaycompress and compress are used together, the dumped log files will not be compressed until the next dump, nodelaycompress overwrite delaycompress option, and the dump will be compressed at the same time. Missingok if the log is lost, the error message when scrolling the next log errors address storage is sent to the specified Email address ifempty even if the log file is empty. This is the default option for logrotate. When notifempty log file is empty, no rotation is performed mail address sends the dumped log file to the specified E-mail address nomail dump does not send log file olddir directory dump log file is placed in the specified directory, must be in the same file system noolddir dump log file and current log file in the same directory sharedscripts runs postrotate script The function is to execute the script once after all the logs are rotated. If this is not configured, then after each log rotation, the instructions that the script prerotate needs to execute before the logrotate dump is executed, such as modifying the properties of the file, must be executed independently by postrotate after the logrotate dump, such as kill-HUP a service! Must be independent daily specified dump cycle daily weekly specified dump cycle weekly monthly specified dump cycle monthly rotate count specified number of dumps before log file deletion, 0 means no backup, 5 refers to keeping 5 backup dateext using the current date as the naming format dateformat.% s with dateext, immediately following the next line, defining the file name after the cut, which must be used with dateext Only the four parameters of% Y% m% d% s, size (or minsize), log-size log files, are dumped when they reach the specified size. Log-size can specify bytes (default) and KB (sizek) or MB (sizem). Dump when the log file > = log-size. The following is a legal format: (unit case of other formats has not been tried) size = 5 or size 5 (> = 5 bytes is dumped) size = 100k or size 100ksize = 100m or size 100m
Small example: the following configuration for cutting nginx logs
[root@master-server ~] # vim / etc/logrotate.d/nginx / usr/local/nginx/logs/*.log {dailyrotate 7 missingoknotifemptydateextsharedscriptspostrotate if [- f / usr/local/nginx/logs/nginx.pid]; then kill-USR1 `cat / usr/local/nginx/logs/ nginx.pid` fiendscript}
-share an example of nginx log cutting processing script that has been used before-
1) logrotate log segmentation configuration:
[root@bastion-IDC ~ # vim / etc/logrotate.d/nginx/data/nginx_logs/*.access_log {nocompress daily copytruncate create ifempty olddir / data/nginx_logs/days rotate 0}
2) Log segmentation script
[root@bastion-IDC ~ # vim / usr _ vf _ nginx _ time=$ _ Nginx_logs/days# uniformly converts the file names of the dump log files in the directory for i in $(ls. / | grep "^\ (. *\)\. [[: digit:]] $") domv ${I}. / $(echo ${I} | sed-n's / ^\ (. *\)\.\ ([: digit:]]\) $/\ 1p')-$(echo $log) done# to compress and store the dumped log files And delete the original dump log file, save only the compressed log file. To save storage space for i in $(ls. / | grep "^\ (. *\)\ ([[: digit:] -]\ +\) $") dotar jcvf ${I} .bz2. / ${I} rm-rf. / ${I} done# only retains the compressed dump log file find / data/nginx_logs/days/*-name "* .bz2"-mtime 7-type f-exec rm-rf {}\
3) crontab timing execution
[root@bastion-IDC ~ # crontab-e#logrotate0 0 * / bin/bash-x / usr/local/sbin/logrotate-nginx.sh > / dev/null 2 > & 1
Execute the script manually and test it:
[root@bastion-IDC ~ # / bin/bash-x / usr/local/sbin/logrotate-nginx.sh [root@bastion-IDC ~ # cd / data/nginx_logs/days [root@bastion-IDC days# lshuantest.access_log-2017-01-18.bz2
-- an example of php script cutting
[root@huanqiu_web1 ~] # cat / etc/logrotate.d/php / Data/logs/php/*log {daily rotate 365missingok notifempty compress dateext sharedscripts postrotate if [- f / Data/app/php5.6.26/var/run/php-fpm.pid] Then kill-USR1 `cat / Data/app/php5.6.26/var/run/php- fpm.pid`fi endscript postrotate / bin/chmod 644 / Data/logs/php/*gz endscript} [root@huanqiu_web1 ~] # ll / Data/app/php5.6.26/var/run/php-fpm.pid-rw-r--r-- 1 root root 4 Dec 28 17:03 / Data/app/php5.6.26/var/run/php-fpm.pid [root@huanqiu_web1 ~ ] # cd / Data/logs/ Sep [root @ huanqiu_web1 php] # lltotal 25676 nobody nobody Sep 7 2015 error.log-20150907.gz-1 root root 0 Jun 1 2016 error.log-rw-r--r-- 1 nobody nobody 182 Aug 30 2015 error.log-20150830.gz-rw-r--r-- 1 nobody nobody 371 Sep 1 2015 error.log-20150901.gz-rw-r--r--
-- an example of nginx log cutting--
[root@huanqiu_web1 ~] # cat / etc/logrotate.d/nginx / Data/logs/nginx/*/*log {daily rotate 365missingok notifempty compress dateext sharedscripts postrotate / etc/init.d/nginx reload endscript} [root@huanqiu_web1 ~] # ll / Data/logs/nginx/www.huanqiu.com/.-rw-r--r-- 1 root root 1652 Jan 1 00:00 error.log-20170101.gz-rw-r--r-- 1 root Root 1289 Jan 2 00:00 error.log-20170102.gz-rw-r--r-- 1 root root 1633 Jan 3 00:00 error.log-20170103.gz-rw-r--r-- 1 root root 3239 Jan 4 00:00 error.log-20170104.gz
-- an example of Syslog cutting
[root@huanqiu_web1 ~] # cat / etc/logrotate.d/syslog/var/log/cron/var/log/maillog/var/log/messages/var/log/secure/var/log/spooler {sharedscripts postrotate / bin/kill-HUP `cat / var/run/syslogd.pid 2 > / dev/ null`2 > / dev/null | | true endscript} [root@huanqiu_web1 ~] # ll / var/log/messages*-rw- 1 root root 34248975 Jan 19 18:42 / var/log / messages-rw- 1 root root 51772994 Dec 25 03:11 / var/log/messages-20161225-rw- 1 root root 51800210 Jan 1 03:05 / var/log/messages-20170101-rw- 1 root root 51981366 Jan 8 03:36 / var/log/messages-20170108-rw- 1 root root 51843025 Jan 15 03:40 / var/log/messages-20170115 [root@huanqiu_web1 ~] # ll / var/log / cron*-rw- 1 root root 2155681 Jan 19 18:43 / var/log/cron-rw- 1 root root 2932618 Dec 25 03:11 / var/log/cron-20161225-rw- 1 root root 2939305 Jan 1 03:06 / var/log/cron-20170101-rw- 1 root root 2951820 Jan 8 03:37 / var/log/cron-20170108-rw- 1 root root 3203992 Jan 15 03:41 / var/log/cron-20170115 [root@huanqiu_web1 ~] # ll / var/log/secure*-rw- 1 root root 275343 Jan 19 18:36 / var/log/secure-rw- 1 root root 2111936 Dec 25 03:06 / var/log/secure-20161225-rw- 1 root root 2772744 Jan 1 02:57 / var/log/secure-20170101-rw- 1 root root 1115543 Jan 8 03 26 / var/log/secure-20170108-rw- 1 root root 731599 Jan 15 03:40 / var/log/secure-20170115 [root@huanqiu_web1 ~] # ll / var/log/spooler*-rw- 1 root root 0 Jan 15 03:41 / var/log/spooler-rw- 1 root root 0 Dec 18 03:21 / var/log/spooler-20161225-rw- 1 root root 0 Dec 25 03: 11 / var/log/spooler-20170101-rw- 1 root root 0 Jan 1 03:06 / var/log/spooler-20170108-rw- 1 root root 0 Jan 8 03:37 / var/log/spooler-20170115
-- an example of tomcat log cutting--
[root@huanqiu-backup ~] # cat/ etc/logrotate.d/tomcat/Data/app/tomcat-7-huanqiu/logs/catalina.out {rotate 14dailycopytruncatecompressnotifemptymissingok} [root@huanqiu-backup ~] # ll / Data/app/tomcat-7-huanqiu/logs/catalina.*-rw-r--r--. 1 root root 0 Jan 19 19:11 / Data/app/tomcat-7-huanqiu/logs/catalina.out-rw-r--r--. 1 root root 95668 Jan 19 19:11 / Data/app/tomcat-7-huanqiu/logs/catalina.out.1.gz
-- an example of early used nginx log processing--
[root@letv-backup ~] # vim / Letv dates= mkdir mkdir for for ((item0 / dev/null I / dev/null 2 > $1) needs to be separated by spaces to separate the names of the log files in the directory where your log files are stored, such as logs_names= "/ logs_names" #
3) try to solve the problem that logrotate cannot automatically poll logs
The phenomenon shows that:
Use logrotate to poll the nginx log, and after configuration, it is found that the nginx log has not been cut for two consecutive days. Why?
Then start to check if there is something wrong with the log cutting configuration file, and then make sure that the configuration file is all right.
Suspected that the cron scheduled by logrotate did not execute, I checked the cron log and found a log like Dec 7 04:02:01 www crond [18959]: (root) CMD (run-parts / etc/cron.daily), proving that cron had executed the program in the / etc/cron.daily directory at 04:02.
Then look at the contents of / etc / cron.daily/logrotate (this is the script for logrotate automatic rotation):
[root@huanqiu_test ~] # cat / usr/bin/logger logrotate logrotate "ALERT exited abnormally with [$EXITVALUE]" fiexit 0
No exception was found, the configured log rotation operation is completed by this script, everything is running normally, and the script should be fine.
Execute the command directly:
[root@huanqiu_test] # / usr/sbin/logrotate / etc/logrotate.conf
These system logs are polled normally, but the nginx logs are still not polled
Then forcibly start the record file maintenance operation, even if the logrotate instruction does not think it is necessary, it should be that logroate thinks that the nginx log is too small to poll.
Therefore, forced polling is required, that is, the-t parameter is replaced with the-f parameter in the / etc/cron.daily/logrotate script
[root@huanqiu_test ~] # cat / usr/bin/logger logrotate logrotate "ALERT exited abnormally with [$EXITVALUE]" fiexit 0
Finally, restart the cron service:
[root@huanqiu_test ~] # / etc/init.d/crond restartStopping crond: [OK] Starting crond: [OK]
Summary
The above is the whole content of this article, I hope that the content of this article can bring some help to your study or work, if you have any questions, you can leave a message and exchange, thank you for your support.
Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.
Views: 0
*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.
Continue with the installation of the previous hadoop.First, install zookooper1. Decompress zookoope
"Every 5-10 years, there's a rare product, a really special, very unusual product that's the most un
© 2024 shulou.com SLNews company. All rights reserved.