Shulou(Shulou.com)06/01 Report--

This article introduces the knowledge of "Linux log regular polling process mechanism". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Logrotate introduction

Log files are extremely important tools for Linux system security. Log files contain useful information about events that occur in the system and are often used in troubleshooting or system performance analysis. When log files continue to grow, they need to be cut regularly, otherwise, the speed and performance of writing logs will also decline, and it will not be convenient for us to file and query.

So when using logrotate, logrotate is a very useful tool that automatically truncates (or rotates) logs, compresses, and deletes old log files. For example, you can set up logrotate to rotate the / var/log/foo log file every 30 days and delete logs for more than 6 months. After configuration, the operation of logrotate is fully automated without any further human intervention.

Logrotate profile location

The logrotate tool is installed by default on the Linux system, and its default configuration file is:

/ etc/logrotate.conf / etc/logrotate.d/

Logrotate.conf is the main configuration file. Logrotate.d is a directory in which all files are actively read into / etc/logrotate.conf.

In addition, if no details are set in the file in / etc/logrotate.d/, the settings in the file / etc/logrotate.conf will be used as the default value.

When actually running, Logrotate invokes the configuration file / etc/logrotate.conf.

You can place a custom configuration file in the / etc/logrotate.d directory to override the default value of Logrotate.

Timing round robin mechanism

Logrotate runs based on CRON, its script is / etc/cron.daily/logrotate, and log rotation is done automatically by the system.

By default, the task logrotate is placed under cron's daily scheduled task cron.daily / etc/cron.daily/logrotate/etc/ directory and the directory of cron.weekly/, cron.hourly/, cron.monthly/ can be used for timing tasks.

[/ etc] $cat / etc/cron.daily/logrotate #! / bin/sh # Clean non existent logfile entries from status file cd / var/lib/logrotate test-e status | | touch status head-1 status > status.clean sed's / "/ / g 'status | while read logfile date do [- e" $logfile "] & & echo"\ "$logfile\" $date "done > status.clean mv status.clean status test-x / usr/sbin/logrotate | | exit 0 / usr/sbin/logrotate / etc/logrotate.conf

Here the actual operation of the polling command *

/ usr/sbin/logrotate / etc/logrotate.conf

Define the script cron.daily/logrotate that executes the task every day, and then look at the content of crontab, where the corresponding cron.xxly is set.

Execution time

[/ etc] $vim / etc/crontab SHELL=/bin/sh PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin # m h dom mon dow user command 17 * * root cd / & run-parts-- report / etc/cron.hourly 25 6 * root test-x / usr/sbin/anacron | | (cd / & & run-parts-- Report / etc/cron.daily) 47 6 * * 7 root test-x / usr/sbin/anacron | | (cd / & & run-parts-- report / etc/cron.weekly) 52 6 1 * * root test-x / usr/sbin/anacron | | (cd / & run-parts-- report / etc/cron.monthly)

You can see that as long as it's in

/ etc/cron.daily/ all the tasks below are performed at 6:25 every day

/ etc/cron.weekly/ all the tasks below are performed at 6:47 every Sunday

/ etc/cron.monthly/ all the following tasks are performed at 6:52 on the 1st of each month.

If you can't wait for cron to automatically perform log rotation, you need to add the-f parameter if you want to force log cutting manually.

However, it is also important to use the Debug option to verify (- d parameter) before formal execution.

# / usr/sbin/logrotate-f / etc/logrotate.d/nginx / / before the time or conditions for cutting, force cutting # / usr/sbin/logrotate-d-f / etc/logrotate.d/nginx / / output cutting debug information

So at this point, we know how logrotate automatically cuts logs.

Logrotate configuration case

Nginx common log slicing configuration

/ data/log/nginx/*.log / data/log/nginx/*/*.log {# cut the log files on the match weekly # cut weekly missingok # during the log cycle, any errors, such as "file cannot be found", will be ignored. Rotate 6 # reserves 6 backups compress # compressed delaycompress # delaycompress is used with compress, the dumped log files will not be compressed until the next dump notifempty # if it is an empty file, the create 0644 www-data ymserver # mode owner group dump file will not be dumped Create a new log file using the specified file mode sharedscripts # the following details the instructions that prerotate # needs to execute before the logrotate dump, such as modifying the properties of the file, etc. Must be a separate line if [- d / etc/logrotate.d/httpd-prerotate]; then\ run-parts / etc/logrotate.d/httpd-prerotate;\ fi\ endscript postrotate # instructions that need to be executed after a logrotate dump, such as kill-HUP a service! It must be executed independently in [- s / run/nginx.pid] & & kill-USR1 `cat / run/ nginx.pid` endscript su root ymserver # when the log is trained by switching the set user / user group (default is root). If the set user/group does not have the permission to allow the file to accommodate the owner specified by the create option, an error will be triggered. }

If you want to configure a daily 0: 00 to perform the cutting task, how to do it? By default, our logrotate execution time is written to the / etc/cron.daily/ directory, and the task execution time under this directory, as mentioned above, is defined in / etc/crontab at 6:25. I had a requirement like this before. Take a look at the following configuration.

/ data/log/owan_web/chn_download_stat/chn_app_rec.log {copytruncate # weekly is commented but inherits the global variable of / etc/logrorate.conf, which is also a very important parameter that weekly missingok rotate 10 compress delaycompress size=1000M # reaches size and starts to dump notifempty create 664 www-data ymserver su root dateext / /! The cut log file ends in the current date format, such as xxx.log-20131216. If the log file is commented out, it will be incremented by number, that is, whether the xxx.log-1 format compress / / is compressed and dumped by gzip, such as xxx.log-20131216.gz. If compression is not needed, comment out the log file}.

Then go to the crontab of root to configure a task to be executed at 0 o'clock.

Wwwadm@host:/etc/logrotate.d$ sudo crontab-l-u root 0 0 * / usr/sbin/logrotate / etc/logrotate.d/web_roteate-fv > / tmp/logro.log 2 > & 1

Because the cutting cycle of logrotate is weekly, each cut is carried out according to the time of the previous cut, if there is a week from the last time, it will be cut, but we set the daily cutting of crontab, neither the daily cutting of / etc/cron.daily/ nor the weekly cutting. In this way, you can customize the log cutting time you want.

Logrotate parameter description

Compress after gzip compression dump nocompress does not do gzip compression processing when create mode owner group rotation specifies the attributes of creating new files, for example, when create 0777 nobody nobody nocreate does not establish new log files delaycompress and compress are used together, the dumped log files will not be compressed until the next dump, nodelaycompress overwrite delaycompress option, and the dump will be compressed at the same time. Missingok if the log is lost, scroll to the next log without error. Ifempty rotates even if the log file is empty, which is the default option for logrotate. When the notifempty log file is empty, there is no rotation. Mail address sends the dumped log file to the specified E-mail address. The log file after olddir directory dump is placed in the specified directory. The log file and the current log file must be placed in the same directory as the current log file. The sharedscripts runs the postrotate script in the same directory as the current log file. The purpose is to execute the script once all logs are rotated. If this is not configured, then each log rotation will execute the instructions that the script prerotate needs to execute before the logrotate dump, such as modifying the properties of the file; it must be a separate line of instructions that postrotate needs to execute after the logrotate dump, such as kill-HUP a service! Must be independent daily specified dump cycle daily weekly specified dump cycle weekly monthly specified dump cycle monthly rotate count specified number of dumps before log file deletion, 0 refers to no backup, 5 refers to keeping 5 backup dateext using the current date as the naming format dateformat.% s with dateext, immediately following the next line to define the file name after the file is cut Must be used with dateext. Only the four parameters of% Y% m% d% s, size (or minsize) log-size log files, are dumped when they reach the specified size. Log-size can specify bytes (default) and KB (sizek) or MB (sizem). Dump when the log file > = log-size. The following is a legal format: (unit case of other formats has not been tried) size = 5 or size 5 (> = 5 bytes is dumped) size = 100k or size 100k size = 100m or size 100m

One noteworthy configuration is: copytruncate

If copytruncate does not have this option, it works by moving the original log log file to an old file similar to log.1, and then creating a new file. If set, how to do it: copy the original log file and change it to a file of size 0.

The difference is that if the process, such as nginx, uses a file to write the log without copytruncate, when cutting the log, put the old log log- > log.1, and then create a new log log. At this time, when the file descriptor opened by nginx is still log.1, there is no signal to tell nginx to change the log descriptor, so it will continue to log to log.1, which does not meet our requirements. Because after we want to cut the log, nginx will automatically write to the new log file instead of the old log.1 file

There are two solutions:

1. Cut the log configuration to the above nginx, and then write a script in postrotate

Postrotate # instructions that need to be executed after a logrotate dump, such as kill-HUP a service! Must be a separate line [- s / run/nginx.pid] & & kill-USR1 `cat / run/ nginx.pid` endscript

This signals nginx to close the old log file descriptor, reopen the new log file description, and write to the log

two。 Use the copytruncate parameter, as mentioned above, after configuring it, the mode of operation is to copy a copy of the log into log.1, and then clear the contents of the log so that the size is 0, then the log is still the original old log, for the process (nginx), the original file descriptor is still open, and you can continue to write logs to it without sending a signal to the nginx

When copytruncate operates in this way, there is a time difference between copying and emptying, and some log data may be lost.

Nocopytruncate backup log files are not truncated

This is the end of the "Linux log regular polling process mechanism". Thank you for your reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.