Shulou(Shulou.com)06/01 Report--

This article will explain in detail the example analysis of logrotate in the Linux system. The editor thinks it is very practical, so I share it for you as a reference. I hope you can get something after reading this article.

Linux system logrotate

The logrotate program is a log file management tool. Used to split log files, delete old log files, and create new log files, play the role of "dump". Disk space can be saved. The following is to do a comb record of the logrotate log rotation operation.

1. Introduction of configuration file

The logrotate tool is installed by default on the Linux system, and its default configuration file is:

/ etc/logrotate.conf/etc/logrotate.d/

Logrotate.conf is the main configuration file. Logrotate.d is a directory in which all files are actively read into / etc/logrotate.conf.

In addition, if no details are set in the file in / etc/logrotate.d/, the settings in the file / etc/logrotate.conf will be used as the default value.

Logrotate runs based on CRON, its script is / etc/cron.daily/logrotate, and log rotation is done automatically by the system. When actually running, Logrotate invokes the configuration file / etc/logrotate.conf. You can place a custom configuration file in the / etc/logrotate.d directory to override the default value of Logrotate.

[root@huanqiu_web1 ~] # cat / ETC logrotate logrotate "ALERT exited abnormally with [$EXITVALUE]" fiexit 0

If you can't wait for cron to automatically perform log rotation, you need to add the-f parameter if you want to force log cutting manually. However, you'd better use the Debug option to verify (- d parameter) before formal execution, which is also important for debugging:

# / usr/sbin/logrotate-f / etc/logrotate.d/nginx# / usr/sbin/logrotate-d-f / etc/logrotate.d/nginx

Logrotate command format:

Logrotate [OPTION...]-d,-- debug: debug mode to test the configuration file for errors. -f,-- force: forcibly dump files. -m,-- mail=command: after compressing the log, send the log to the specified mailbox. -s,-- state=statefile: uses the specified status file. -v,-- verbose: shows the dump process.

Operate according to the log cutting settings and display details:

[root@huanqiu_web1] # / usr/sbin/logrotate-v / etc/ logrotate.confs [root @ huanqiu_web1 ~] # / usr/sbin/logrotate-v / etc/logrotate.d/php

It is executed according to the log cutting settings, and the details are displayed, but no specific operation is performed, debug mode

[root@huanqiu_web1 ~] # / usr/sbin/logrotate-d / etc/ logrotate.confs [root @ huanqiu_web1 ~] # / usr/sbin/logrotate-d / etc/logrotate.d/nginx

Check the specific implementation of each log file

[root@fangfull_web1 ~] # cat / var/lib/logrotate.status2, introduction to cutting

For example, use Syslog / var/log/message to make a simple explanation:

After performing the rotate for the first time, the original messages becomes messages.1 and an empty messages is created for the system to store logs

After the second execution, the messages.1 becomes messages.2 and the messages becomes messages.1, creating an empty messages to store the log!

If you set to keep only three logs (that is, rotate 3 times), then on the third execution, the messages.3 file will be deleted and replaced by a newer save log later! That is, the latest logs will be saved.

How many times the log is rotated is determined by the dateext parameter in the configuration file.

Take a look at logrotate.conf configuration:

The setting under # cat / etc/logrotate.conf# is "the default value of logrotate". If other files set other values, # will use the settings of other files as the main weekly / / default to perform rotate rotation once a week rotate 4 / / how many log files to keep (several times). Four are reserved by default. Is to specify the number of rotations before the log file is deleted. 0 means no backup create / / automatically creates a new log file, and the new log file has the same permissions as the original file; because the log has been renamed, it is important to create a new one to continue to store the previous log dateext / /. The cut log file ends in the current date format, such as xxx.log-20131216. If the log file is commented out, it is incremented by number, that is, whether the xxx.log-1 format compress / / is compressed and dumped by gzip, such as xxx.log-20131216.gz. If you don't need compression Comment out the line include / etc/logrotate.d# loads all the files in the / etc/logrotate.d/ directory / var/log/wtmp {/ / only for the parameter set by / var/log/wtmp monthly / / cut once a month Instead of the default weekly minsize 1m / / file size exceeding 1m, create 0664 root utmp / / specifies the permissions for newly created log files and the user and group rotate 1 / / keeps only one log.} # this wtmp records the time when the user logs in to the system and the system restart # because of the parameters of minsize So it doesn't have to be carried out once a month. Depends on the size of the file.

From the settings of this file, we can know that / etc/logrotate.d is actually the directory planned by / etc/logrotate.conf. Although all configurations can be written into / etc/logrotate.conf, this file is really too complex, especially when using a lot of services on the system, it seems unreasonable for each service to modify the settings of / etc/logrotate.conf.

So, if there is a separate directory, then each service that wants to cut the log can be a separate file and placed in / etc/logrotate.d/.

Other important parameters describe:

Compress uses gzip compression to dump logs nocompress does not do gzip compression processing copytruncate is used to backup and truncate the current log files that are still open. It is a way to copy and then empty. There is a time difference between copy and emptying, and some log data may be lost. Nocopytruncate backs up log files but does not truncate the properties that specify the creation of new files during create mode owner group rotation, such as when create 0777 nobody nobodynocreate does not create new log files delaycompress and compress are used together The nodelaycompress overwrite delaycompress option is not compressed until the next dump, and the dump is compressed at the same time. Missingok if the log is lost, the error message when scrolling the next log errors address storage is sent to the specified Email address ifempty even if the log file is empty. This is the default option for logrotate. When notifempty log file is empty, no rotation is performed mail address sends the dumped log file to the specified E-mail address nomail dump does not send log file olddir directory dump log file is placed in the specified directory The log files after noolddir dump and the current log files must be placed in the same directory as the current log files. Sharedscripts runs the postrotate script, which is used to execute the script once all logs are rotated. If this is not configured, then after each log rotation, the instructions that the script prerotate needs to execute before the logrotate dump is executed, such as modifying the properties of the file, must be executed independently by postrotate after the logrotate dump, such as kill-HUP a service! Must be a separate line daily specified dump cycle daily weekly specified dump cycle weekly monthly specified dump cycle monthly rotate count specified number of dumps before log file deletion, 0 means no backup 5 means to keep 5 backup dateext using the current date as the naming format dateformat.% s with dateext, which appears immediately after the next line. The file name after the cut of the definition file must be used with dateext. Only the four parameters of% Y% m% d% s, size (or minsize), log-size log files, are dumped when they reach the specified size. Log-size can specify bytes (default) and KB (sizek) or MB (sizem). Dump when the log file > = log-size. The following is a legal format: (unit case of other formats has not been tried) size = 5 or size 5 (> = 5 bytes is dumped) size = 100k or size 100ksize = 100m or size 100m

Small example: the following configuration for cutting nginx logs

[root@master-server ~] # vim / etc/logrotate.d/nginx/usr/local/nginx/logs/*.log {dailyrotate 7missingoknotifemptydateextsharedscriptspostrotate if [- f / usr/local/nginx/logs/nginx.pid]; then kill-USR1 `cat / usr/local/nginx/logs/ nginx.pid` fiendscript}

Share an example of a nginx log cutting script that has been used:

1) logrotate log segmentation configuration

[root@bastion-IDC ~ # vim / etc/logrotate.d/nginx/data/nginx_logs/*.access_log {nocompress daily copytruncate create ifempty olddir / data/nginx_logs/days rotate 0}

2) Log segmentation script

[root@bastion-IDC ~ # vim / usr _ vf _ nginx _ time=$ _ Nginx_logs/days# uniformly converts the file names of the dump log files in the directory for i in $(ls. / | grep "^\ (. *\)\. [[: digit:]] $") domv ${I}. / $(echo ${I} | sed-n's / ^\ (. *\)\.\ ([: digit:]]\) $/\ 1p')-$(echo $log) done# to compress and store the dumped log files And delete the original dump log file, save only the compressed log file. To save storage space for i in $(ls. / | grep "^\ (. *\)\ ([[: digit:] -]\ +\) $") dotar jcvf ${I} .bz2. / ${I} rm-rf. / ${I} done# only retains the compressed dump log file find / data/nginx_logs/days/*-name "* .bz2"-mtime 7-type f-exec rm-rf {}\

3) crontab timing execution

[root@bastion-IDC ~ # crontab-e#logrotate0 0 * / bin/bash-x / usr/local/sbin/logrotate-nginx.sh > / dev/null 2 >

Execute the script manually and test it:

[root@bastion-IDC ~ # / bin/bash-x / usr/local/sbin/logrotate-nginx.sh [root@bastion-IDC ~ # cd / data/nginx_logs/ days [root @ bastion-IDC days# lshuantest.access_log-2017-01-18.bz2

An example of php script cutting:

[root@huanqiu_web1 ~] # cat / etc/logrotate.d/php/Data/logs/php/*log {daily rotate 365missingok notifempty compress dateext sharedscripts postrotate if [- f / Data/app/php5.6.26/var/run/php-fpm.pid] Then kill-USR1 `cat / Data/app/php5.6.26/var/run/php- fpm.pid` fi endscript postrotate / bin/chmod 644 / Data/logs/php/*gz endscript} [root@huanqiu_web1 ~] # ll / Data/app/php5.6.26/var/run/php-fpm.pid-rw-r--r-- 1 root root 4 Dec 28 17:03 / Data/app/php5.6.26/var/ Run/php- fpm.pid [root @ huanqiu_web1 ~] # cd / Data/logs/ php [root @ huanqiu_web1 php] # lltotal 25676 Aug 1 Jun 1 2016 error.log-rw-r--r-- 1 nobody nobody 182 Aug 30 2015 error.log-20150830.gz-rw-r--r-- 1 nobody nobody 371 Sep 1 2015 error.log-20150901.gz-rw-r--r-- 1 nobody nobody 315 Sep 7 2015 error.log-20150907.gz.

An example of nginx Log cutting

[root@huanqiu_web1 ~] # cat / etc/logrotate.d/nginx/Data/logs/nginx/*/*log {daily rotate 365missingok notifempty compress dateext sharedscripts postrotate / etc/init.d/nginx reload endscript} [root@huanqiu_web1 ~] # ll / Data/logs/nginx/www.huanqiu.com/.-rw-r--r-- 1 root root 1652 Jan 1 00:00 error.log-20170101 .gz-rw-r--r-- 1 root root 1289 Jan 2 00:00 error.log-20170102.gz-rw-r--r-- 1 root root 1633 Jan 3 00:00 error.log-20170103.gz-rw-r--r-- 1 root root 3239 Jan 4 00:00 error.log-20170104.gz

An example of system log cutting

[root@huanqiu_web1 ~] # cat / etc/logrotate.d/syslog/var/log/cron/var/log/maillog/var/log/messages/var/log/secure/var/log/spooler {sharedscripts postrotate / bin/kill-HUP `cat / var/run/syslogd.pid 2 > / dev/ null`2 > / dev/null | | true endscript} [root@huanqiu_web1 ~] # ll / var/log/messages*-rw- 1 root root 34248975 Jan 19 18: 42 / var/log/messages-rw- 1 root root 51772994 Dec 25 03:11 / var/log/messages-20161225-rw- 1 root root 51800210 Jan 1 03:05 / var/log/messages-20170101-rw- 1 root root 51981366 Jan 8 03:36 / var/log/messages-20170108-rw- 1 root root 51843025 Jan 15 03:40 / var/log/messages-20170115 [root@huanqiu_web1 ~] # Ll / var/log/cron*-rw- 1 root root 2155681 Jan 19 18:43 / var/log/cron-rw- 1 root root 2932618 Dec 25 03:11 / var/log/cron-20161225-rw- 1 root root 2939305 Jan 1 03:06 / var/log/cron-20170101-rw- 1 root root 2951820 Jan 8 03:37 / var/log/cron-20170108-rw- 1 Root root 3203992 Jan 15 03:41 / var/log/cron-20170115 [root@huanqiu_web1 ~] # ll / var/log/secure*-rw- 1 root root 275343 Jan 19 18:36 / var/log/secure-rw- 1 root root 2111936 Dec 25 03:06 / var/log/secure-20161225-rw- 1 root root 2772744 Jan 1 02:57 / var/log/secure-20170101-rw- 1 root Root 1115543 Jan 8 03:26 / var/log/secure-20170108-rw- 1 root root 731599 Jan 15 03:40 / var/log/secure-20170115 [root@huanqiu_web1 ~] # ll / var/log/spooler*-rw- 1 root root 0 Jan 15 03:41 / var/log/spooler-rw- 1 root root 0 Dec 18 03:21 / var/log/spooler-20161225-rw- 1 root root 0 Dec 25 03:11 / var/log/spooler-20170101-rw- 1 root root 0 Jan 1 03:06 / var/log/spooler-20170108-rw- 1 root root 0 Jan 8 03:37 / var/log/spooler-20170115

An example of tomcat Log cutting

[root@huanqiu-backup ~] # cat/ etc/logrotate.d/tomcat/Data/app/tomcat-7-huanqiu/logs/catalina.out {rotate 14dailycopytruncatecompressnotifemptymissingok} [root@huanqiu-backup ~] # ll / Data/app/tomcat-7-huanqiu/logs/catalina.*-rw-r--r--. 1 root root 0 Jan 19 19:11 / Data/app/tomcat-7-huanqiu/logs/catalina.out-rw-r--r--. 1 root root 95668 Jan 19 19:11 / Data/app/tomcat-7-huanqiu/logs/catalina.out.1.gz

An example of nginx log processing used in the early days

[root@letv-backup ~] # vim / Letv error access pv_access dates= mkdir mkdir-p ${logs_path} $dates/num=$ {# logs_names [@]} for ((item0)) I / dev/null 2 > $13, try to solve the problem that logrotate cannot automatically poll logs

The phenomenon shows that:

Use logrotate to poll the nginx log, and after configuration, it is found that the nginx log has not been cut for two consecutive days. Why?

Then start to check if there is something wrong with the log cutting configuration file, and then make sure that the configuration file is all right.

Suspected that the cron scheduled by logrotate did not execute, I checked the cron log and found a log like Dec 7 04:02:01 www crond [18959]: (root) CMD (run-parts / etc/cron.daily), proving that cron had executed the program in the / etc/cron.daily directory at 04:02.

Then look at the contents of / etc / cron.daily/logrotate (this is the script for logrotate automatic rotation):

[root@huanqiu_test ~] # cat / ETC logrotate logrotate "ALERT exited abnormally with [$EXITVALUE]" fiexit 0

No exception was found, the configured log rotation operation is completed by this script, everything is running normally, and the script should be fine.

Execute the command directly:

[root@huanqiu_test] # / usr/sbin/logrotate / etc/logrotate.conf

These system logs are polled normally, but the nginx logs are still not polled.

Then forcibly start the record file maintenance operation, even if the logrotate instruction does not think it is necessary, it should be that logroate thinks that the nginx log is too small to poll.

Therefore, forced polling is required, that is, the-t parameter is replaced with the-f parameter in the / etc/cron.daily/logrotate script

[root@huanqiu_test ~] # cat / ETC logrotate logrotate "ALERT exited abnormally with [$EXITVALUE]" fiexit

Finally, restart the cron service:

[root@huanqiu_test ~] # / etc/init.d/crond restartStopping crond: [OK] Starting crond: [OK]

Logrotate default automatic cutting effective time

Logrotate runs based on CRON, its script is / etc/cron.daily/logrotate, and when it is actually run, Logrotate calls the configuration file / etc/logrotate.conf. [root@test ~] # cat / ETC EXITVALUE cron.dailyUniqqlogrotatexxxxxxxxxxxxxxx Then / usr/bin/logger-t logrotate "ALERT exited abnormally with [$EXITVALUE]" fiexit 0Logrotate is based on CRON, so this time is controlled by CRON For more information, you can query CRON's configuration file / etc/anacrontab (the old version is / etc/crontab) [root@test ~] # cat / etc/anacrontab# / etc/anacrontab: configuration file for anacron# See anacron (8) and anacrontab (5) for details.SHELL=/bin/shPATH=/sbin:/bin:/usr/sbin:/usr/binMAILTO=root# the maximal random delay added to the base delay of the jobsRANDOM_DELAY=45 / / this is a random delay time Indicates a maximum of 45 minutes # the jobs will be started during the following hours onlySTART_HOURS_RANGE=3-22 / / this is the start time # period in days delay in minutes job-identifier command1 5 cron.daily nice run-parts / etc/cron.daily7 25 cron.weekly nice run-parts / etc/cron.weekly@monthly 45 cron.monthly nice run-parts / etc/ The cron.monthly first is the Recurrence period and the second is the delay time, so the cron.daily will be executed during the period of 3glued 22 + (5pm 45). / etc/cron.daily is a folder configured by the default / etc/anacrontab file, and you will find that the default time for logrotate to automatically cut log files is after 3: 00 a.m. = now you need to adjust the cutting time to 12:00 every night, that is, the log of each day is between 0 and 24:00 the day before. The operation is as follows: [root@kevin ~] # mv / etc/anacrontab / etc/anacrontab.bak / / cancel the setting of automatic log rotation [root@G6-bs02 logrotate.d] # cat nstc_nohup.out/data/nstc/nohup.out {rotate 30dateextdailycopytruncatecompressnotifemptymissingok} [root@G6-bs02 logrotate.d] # cat syslog/var/log/cron/var/log/maillog/var/log/messages/var/log/secure/var/log/history {sharedscripts compress Rotate 30 daily dateext postrotate / bin/kill-HUP `cat / var/run/syslogd.pid 2 > / dev/ null` > / dev/null | | true endscript} custom scheduled rotation operation combined with crontab [root@kevin ~] # crontab-l#log logrotate59 23 * / usr/sbin/logrotate-f / etc/logrotate.d/syslog > / dev/null 2 > & 15923 * / usr/sbin/logrotate-f / etc/logrotate.d / nstc_nohup.out > / dev/null 2 > & 1 [root@G6-bs02 ~] # ll / data/nstc/nohup.out*-rw- 1 app app 33218 January 25 09:43 / data/nstc/nohup.out-rw- 1 app app 67678 January 25 23:59 / data/nstc/nohup.out-20180125.gz

In addition to using the built-in Logrotate tools to achieve log cutting, you can also write python scripts or shell scripts to achieve log cutting. Here are a few examples to illustrate:

First, Python script to achieve log cutting

Example 1: cut the jumpserver pythonimport datetime,os,sys [root@test-vm01 mnt] # cat log rotation. Pyrogen log Shutillog_path ='/ opt/jumpserver/logs/'log_file = 'jumpserver.log'yesterday = (datetime.datetime.now ()-datetime.timedelta (days = 1)) try: os.makedirs (log_path + yesterday.strftime ('% Y') + os.sep +\ yesterday.strftime ('% m') except OSError,e: print print e sys.exit () shutil.move (log_path + log_file) Log_path\ + yesterday.strftime ('% Y') + os.sep\ + yesterday.strftime ('% m') + os.sep\ + log_file +'_'+ yesterday.strftime ('% Y% m% d') + '.log') os.popen ("sudo / opt/jumpserver/service.sh restart") manually execute the script: [root@test-vm01 mnt] # chmod 755 log_rotate .py [root@test-vm01 mnt] # python log_rotate.py view the effect of log slicing: [root@test-vm01 mnt] # ls / opt/jumpserver/logs/2017 jumpserver.log [root@test-vm01 mnt] # ls / opt/jumpserver/logs/2017/09 [root@test-vm01 mnt] # ls / opt/jumpserver/logs/2017/09/jumpserver.log_20170916.log and then do daily scheduled slicing task: [root@test-vm01 mnt] # crontab -e301 * / usr/bin/python / mnt/log_rotate.py > / dev/null 2 > & 1-instance 2 : cut the nginx pythonimport datetime [root@test-vm01 mnt] # vim log rotation. Pyrogen log Os,sys,shutillog_path ='/ app/nginx/logs/'log_file = 'www_access.log'yesterday = (datetime.datetime.now ()-datetime.timedelta (days = 1)) try: os.makedirs (log_path + yesterday.strftime ('% Y') + os.sep +\ yesterday.strftime ('% m') except OSError,e: print print e sys.exit () shutil.move (log_path + log_file) Log_path\ + yesterday.strftime ('% Y') + os.sep\ + yesterday.strftime ('% m') + os.sep\ + log_file +'_'+ yesterday.strftime ('% Y% m% d') + '.log') os.popen ("sudo kill-USR1 `cat / app/nginx/logs/nginx.pid `")- -other business log cutting scripts are the same as above

Second, shell script to achieve log cutting

[root@qd-vpc-op-consumer01 ~] # cat / rm rm logs_path. For i in $* do (date-d "7 days ago" + "% Y%m%d"). Rotate $idone- cut regularly every day Task formulation of logs (such as cutting logs under a business of python / data/log/xcspam/) 0K logs are not sliced): [root@qd-vpc-op-consumer01 ~] # crontab-e#xcspam log slicing 300 * find / data/log/xcspam/-size + 0-name'* .log'| xargs / app/script/log_rotate.sh perform manual slicing: [root@qd-vpc-op-consumer01 ~] # find / data/log/xcspam/-size + 0-name'* .log'| xargs / app/script/log_rotate Log effect after .sh cutting: [root@qd-vpc-op-consumer01 ~] # ls / data/log/xcspam/xcspam_error.log xcspam_error.log-20170926 -- for example, cutting maridb logs [root@qd-vpc-op-consumer01 ~] # crontab-e#xcspam logs 300 * find / var/log/mariadb/-size + 0-name'* .log'| xargs / app/script/log_ rotate.sh [root @ qd-vpc-op-consumer01 ~] # find / var / log/mariadb/-size + 0-name'* .log'| xargs / app/script/log_ rotate.sh [root @ qd-vpc-op-consumer01 ~] # ll / var/log/mariadb/ total usage: 8kw / var/log/mariadb/. 1 mysql mysql 0 September 17 20:31 mariadb.log-rw-r-. 1 root root 4532 September 17 20:31 mariadb.log.20170916- -Log compression script: [root@localhost ~] # ls / var/log/fss/nginx/nginx.20190506.log nginx.20190507.log nginx.20190508.log [root@localhost ~] # cat / rootapaploglog clean.shrunxamusrbinbinBinLock # compress logs based on three parameters of system / service / log retention days # usage: sh clearlog.sh sysname appname keepdayssysName=$1appName=$2keepDay=$3logDir=/var/log/$ {sysName} / ${appName} log {appName} . * [0-9] [0-9] .logcd ${logDir} find. /-name "${logFile}"-mtime-${keepDay}-exec gzip {}\ [root@localhost ~] # sh / root/log_clean.sh fss nginx 3 [root@localhost ~] # ls / var/log/fss/nginx/nginx.20190506.log.gz nginx.20190507.log.gz nginx.20190508.log.gz can also be adjusted to log cleanup scripts for log retention policies.

Recommended Nginx log rotation method [deployed in the log directory of nginx]

#! / bin/bashyesterday= `date-d "- 1 days" +'% Y% m% d' `basedir = `pwd`logdir = "${basedir} / bak" bindir= "${basedir%/*} / sbin" mkdir-p ${logdir} for log in `basedir * .log 2 > / dev/ null`do mv ${logdir} / ${log} .bak # gzip ${logdir} / ${log}. ${yesterday} done$ {bindir} / nginx-s reloadcd ${logdir} find. -type f-name "* .bak"-mtime + 7 | xargs rm-f on "sample Analysis of logrotate in Linux system" this article ends here. I hope the above content can be helpful to you so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.