Shulou(Shulou.com)06/01 Report--

This article mainly explains "how to set up automatic cleaning of files n days ago in Linux system", the explanation content in the article is simple and clear, easy to learn and understand, please follow the idea of Xiaobian slowly in-depth, together to study and learn "how to set up automatic cleaning of files n days ago in Linux system"!

The server generates a global backup file once a week, which is about 100 gigabytes in size and needs to be cleaned up regularly.

Deleting big data during working hours when website visits are high and server I/O is high will have a bad impact on server status. So I want to use planned tasks to automatically execute them.

Under my backup directory/bakcup, each backup file is named with a date directory name:

The code is as follows:

# ls

2013-12-23 2014-01-06 2014-01-20 2014-02-03

2013-12-30 2014-01-13 2014-01-27 2014-02-10

Delete some backups while keeping some, you can use the find command, such as I want to keep the last four weeks of backup files, each backup interval of seven days:

The code is as follows:

# find /bakcup/ -maxdepth 1 -type d -mtime +28

/bakcup/2014-01-06

/bakcup/2014-01-13

/bakcup/2013-12-23

/bakcup/2013-12-30

- maxdepth1: Set the search directory depth to 1, and only search in/backup directory. If this parameter is not added, files in subordinate directories will be listed.

-type d: Set search type to directory

-mtime +28: Find directories 28 days ago

-exec parameter available after search connection delete command

The code is as follows:

rsync --delete-before -d /data/test/ {} \;

So, the whole command is:

The code is as follows:

# find /bakcup/ -maxdepth 1 -type d -mtime +28 -exec rsync --delete-before -d /data/test/ {} \;

Finally, you can put the command into a script and set crontab to execute automatically.

Reminder:

Before using the command, you should first try the find part of the command on the server. If you only find the directory to be cleaned, you can continue.

It is not excluded that some systems will./ Directory search out, must see clearly, prevent accidents.

In addition, you can replace-exec with-ok, which has the same effect and reminds the user to confirm before deleting it.

PS: rm command versus rsync command efficiency comparison

rm

The rm command makes a lot of calls to lstat64 and unlink, and you can assume that each file was deleted before an lstat operation was done from the file system.

There are other reasons why lstat64 is lower than the total number of files, which will be explained later in another article.

getdientries 64 This call is critical.

Process: The first stage of the formal deletion work requires the getdirentries64 call to read the directory in batches (about 4K each time) and build a list of rm files in memory; the second stage, lstat64 determines the status of all files; and the third stage, the actual deletion is performed through unlink. All three phases involve more system calls and file system operations.

rsync

Rsync makes very few system calls.

There are no lstat and unlink operations for individual files.

In the early stages of command execution, rsync opens a piece of shared memory and loads directory information via mmap.

Do directory synchronization only, no need to unlink individual files.

In addition, in other people's evaluation, rm context switch is more, will cause the System CPU consumption is more-for file system operations, simply increasing the number of concurrency does not always improve the speed of operation.

Thank you for reading, the above is "how to set up automatic cleaning Linux system n days ago files" content, after learning this article, I believe we have a deeper understanding of how to set up automatic cleaning Linux system n days ago files, the specific use of the situation also needs to be verified. Here is, Xiaobian will push more articles related to knowledge points for everyone, welcome to pay attention!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.