Shulou(Shulou.com)06/01 Report--

This article is about how rsync+crontab synchronizes backups on a regular basis under centos7. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

This time, the script sh under Linux executes regular automatic backup of data.

Client: 192.168.0.100 (rsync-Linux)

Server: 192.168.0.252 (storage)

Introduction to rsync:

Rsync is a data mirror backup tool under linux system. Use the fast incremental backup tool Remote Sync to synchronize remotely, support local replication, or synchronize with other SSH or rsync hosts.

1. Server configuration:

The server of rsync is mainly the storage of Qunghui, so it has interface configuration directly, so we will not explain it in detail here, as shown below:

two。 Client configuration:

Create a password file, / etc/rsyncd/rsyncd.pass, and modify it to 600 permissions

[root@localhost ~] # mkdir rsyncd [root@localhost ~] # vim / rsyncd/rsyncd.pass [root@localhost ~] # chmod 600 / rsyncd/rsyncd.pass

▲ description: the password in / rsyncd/rsyncd.pass needs to be consistent with the password of the specified user in the server / etc/rsyncd/rsyncd.pass or storage rsync

The client transfers files to the server. If it is port 873, you can remove-- port.

Rsync-arvz-progress / data rsync@192.168.0.252::log-password-file=/rsyncd/rsyncd.pass

If necessary, you can also pull files from the server. You need to remove the write only = yes in the / etc/rsyncd/rsyncd.conf of the server.

# pull the entire directory rsync-arvz-- progress-- password-file=/rsyncd/rsyncd.pass-- port=873 rsync@192.168.0.252::log/ var/log# pull a single file rsync-arvz-- progress-- password-file=/rsyncd/rsyncd.pass-- port=873 rsync@192.168.0.252::log/a.log / var/log

Rsync server configuration parameters:

Global parameters all parameters before [module] in the file are global parameters, of course, you can also define module parameters in the global parameters section, in which case the value of this parameter is the default value of all modules. Port specifies the port number used by the daemon, which defaults to 873. The motd file "motd file" parameter is used to specify a message file whose contents are displayed to the client when the client connects to the server. There is no motd file by default. Log file "log file" specifies the log file for rsync and does not send the log to syslog. For example, you can specify "/ var/log/rsyncd.log". Pid file specifies the pid file for rsync, usually "/ var/run/rsyncd.pid". Syslog facility specifies the message level at which rsync sends log messages to syslog. Common message levels are: uth, authpriv, cron, daemon, ftp, kern, lpr, mail, news, security, sys-log, user, uucp, local0, local1, local2, local3,local4, local5, local6 and local7. The default value is daemon. The module parameter is mainly to define which directory of the server will be synchronized. The format must be in the form of "[module]", which is the name you see on the rsync client, but is actually a bit like the share name provided by the Samba server. The data that the server actually synchronizes is specified through path. We can specify multiple modules according to our own needs, and the following parameters can be defined in the module: comment assigns a description to the module, which, together with the module name, is displayed to the customer when the customer connection gets the module list. There is no description definition by default. Path specifies the directory tree path of the module for backup, and this parameter must be specified. Use chroot if "use chroot" is specified as true, then rsync first chroot to the directory specified by the path parameter before transferring the file. The reason for this is to implement additional security, but the drawback is that you need roots permissions, and you cannot back up the directory files pointed to by symbolic links that point to the outside. By default, the color value is true. Uid this option specifies the uid that the daemon should have when the module transfers files. Use with the gid option to determine which file permissions can be accessed and what. The default value is "nobody". Gid this option specifies the gid that the daemon should have when the module transfers files. The default is "nobody". Max connections specifies the maximum number of concurrent connections for the module to protect the server, and connection requests that exceed the limit are told to try again later. The default value is 0, which means there is no limit. List this option sets whether the module should be listed when a customer requests a list of modules that can be used. If you set this option to false, you can create hidden modules. The default value is true. Read only this option sets whether customers are allowed to upload files. If it is true, then any upload request will fail, and if it is false and read and write permissions to the server directory are allowed, then upload is allowed. The default is true. Exclude is used to specify multiple files or directories (relative paths) separated by spaces and add them to the exclude list. This is equivalent to using-- exclude in the client command to specify the mode, and a module can specify only one exclude option. It is important to note, however, that this option has some security issues, and customers are likely to bypass the exclude list, and if you want to ensure that specific files cannot be accessed, it is best to use it in conjunction with the uid/gid option. Exclude from specifies a file name that contains the definition of the exclude schema from which the server reads the exclude list definition. Include is used to specify files or directories that meet the requirements that are not excluded. This is equivalent to using-- include in client commands to specify patterns, and combining include and exclude can define complex exclude/include rules. Include from specifies a file name that contains the definition of the include schema from which the server reads the include list definition. Auth users this option specifies a list of user names separated by spaces or commas, and only these users are allowed to connect to the module. The users here have nothing to do with the users of the system. If "auth users" is set, the client's connection request for the module will later be authenticated by the rsync request challenged to authenticate the challenge/response authentication protocol used here. The user's name and password are stored in clear text in the file specified by the "secrets file" option. By default, modules can be connected without a password (that is, anonymously). Secrets file this option specifies a file that defines a username: password pair. This file is useful only if "auth users" is defined. Each line of the file contains a username:passwd pair. Generally speaking, the password should not exceed 8 characters. There is no default secures file name, one needs to be specified in a limited format (for example: / etc/rsyncd.passwd). Note: the permissions of this file must be 600, otherwise the client will not be able to connect to the server. Strict modes this option specifies whether to monitor the permissions of the password file. If the value of this option is true, the password file can only be accessed by users who are running as the rsync server, and no other user can access the file. The default is true. Hosts allow this option specifies which IP customers are allowed to connect to the module. The definition of customer mode can be in the following form: a single IP address, for example: 192.167.0.1 entire network segment, for example: 192.168.0.0 IP 24, or 192.168.0.0 IP 255.255.255.0 or network segments need to be separated by spaces. "*" means all, and all hosts are allowed to connect by default. Hosts deny specifies the machines that are not allowed to connect to the rsync server, which can be defined using the definition of hosts allow. There is no hosts deny definition by default. Ignore errors specifies that rsyncd ignores IO errors on server when determining whether to run delete operations on transport. Generally speaking, rsync will skip the-- delete operation when an IO error occurs to prevent serious problems caused by temporary insufficient resources or other IO errors. Ignore nonreadable specifies that the rysnc server completely ignores files that the user does not have access to. This makes sense when there are some files in the directory that need to be backed up that should not be available to the backup person. Lock file specifies the lock file that supports the max connections parameter, and the default value is / var/run/rsyncd.lock. Transfer logging causes the rsync server to use files in ftp format to record downloads and uploads in its own separate log. Log format allows users to customize the fields of the log file when using transfer logging. Its format is a string that contains format definers. The format definitions that can be used are as follows:% h remote hostname% a remote IP address% l file length characters% p process id%o operation type for this rsync session: "send" or "recv"% f file name% P module name% m module name% t current time% u authenticated user name (null when anonymous)% b actual number of bytes transferred c when the file is sent The default log format for this field to record the file is: "% o% h [% a]% m (% u)% f% l", generally speaking, "% t [% p]" is added to the header of each line. A perl script called rsyncstats is also released in the source code to count log files in this format. This option allows timeout to override the IP timeout specified by the customer. This option ensures that the rsync server does not wait forever for a crashed client. The timeout unit is seconds, and 0 indicates that there is no timeout definition, which is also the default value. For anonymous rsync servers, an ideal number is 600. This option allows refuse options to define a list of command parameters that customers are not allowed to use on this module. The full name of the command must be used here, not the abbreviation. However, when a command is rejected, the server reports an error message and exits. If you want to prevent the use of compression, it should be: "dont compress = *". Dont compress is used to specify files that are not compressed and retransferred. The default value is * .gz * .tgz * .zip * .z * .rpm * .deb * .iso * .bz2 * .tbz

Common rsync commands:

-v,-- verbose detailed mode output-Q,-- quiet compact output mode-c,-- checksum turns on the check switch to force the file transfer to be checked-a,-archive archive mode, indicating that the file is transferred recursively and all file attributes are maintained. Equal to-rlptgoD-r,-- recursive processes subdirectories in recursive mode-R,-- relative uses relative path information-b,-- backup to create a backup, that is, rename the old file to ~ filename if the same file name already exists for the destination. You can use the-- suffix option to specify different backup file prefixes. -- backup-dir stores backup files (such as ~ filename) in a directory. -suffix=SUFFIX defines the backup file prefix-u,-update only to update, that is, to skip all files that already exist in DST and whose file time is later than that to be backed up. (do not overwrite updated files)-l,-- links retains soft links-- copy-links treats soft links like regular files-- copy-unsafe-links only copies links that point outside the SRC path directory tree-- safe-links ignores links outside the SRC path directory tree-H,-- hard-links retains hard links-- perms maintains file permissions-o -- owner keeps file ownership information-- g,-- group keeps file group information-- D,-- devices keeps device file information-- t,-- times keeps file time information-sparse performs special handling of sparse files to save DST space-- n,-- dry-run reality which files will be transferred-W,-- whole-file copy files. No incremental detection-x,-one-file-system do not cross file system boundaries-B,-block-size=SIZE check algorithm uses block size, default is 700byte-e,-rsh=COMMAND specifies data synchronization using rsh, ssh mode-rsync-path=PATH specifies the path information of rsync commands on the remote server-C,-cvs-exclude automatically ignores files in the same way as CVS Used to exclude files that you do not want to transfer-existing updates only those files that already exist in DST Without backing up those newly created files-- delete deletes those files that are not available in SRC in DST-- delete-excluded also deletes those files that are excluded by this option specified by this option-- delete after the delete-after transfer ends-- ignore-errors deletes even if there is an IO error-- max-delete=NUM deletes up to NUM files-- partial retains those files that have not been fully transferred for some reason. So speed up subsequent transfers-- force forcibly delete directories, even if not empty-- numeric-ids does not match numeric user and group ID to user and group names-- timeout=TIME IP timeout, in seconds-I,-- ignore-times does not skip files of the same time and length-- size-only when deciding whether to back up files Only look at the file size without considering the file time-- the timestamp window used by modify-window=NUM to determine whether the file is at the same time The default is 0-T-- temp-dir=DIR creates temporary files in DIR-- compare-dest=DIR also compares files in DIR to determine whether a backup is needed-P equals-- partial-- progress shows the backup process-z -- compress compresses backup files during transfer-- exclude=PATTERN specifies to exclude files that do not need to be transferred-- include=PATTERN specifies files that do not exclude but need to be transferred-- exclude-from=FILE excludes files in the specified mode in FILE-- include-from=FILE does not exclude files that match the specified pattern in FILE-- version print version information-- address binds to a specific address-- config=FILE specifies other configuration files. Do not use default rsyncd.conf files-port=PORT specifies other rsync service ports-blocking-io uses blocking IO-stats for remote shell to give the transfer status of certain files-progress actual transfer process during transfer-log-format=formAT specifies log file format-password-file=FILE gets password from FILE-bwlimit=KBPS limits FILE O bandwidth, KBytes per second-h,-- help displays help information

Introduction to crontab:

Crond is a daemon used under linux to periodically execute certain tasks or wait for certain events to be handled. Similar to the scheduled tasks under windows, when the installation of the operating system is completed, this service tool is installed by default, and the crond process is automatically started. The crond process periodically checks whether there is a task to be executed every minute, and automatically executes the task if there is a task to be executed. Task scheduling under Linux is divided into two categories: system task scheduling and user task scheduling. System task scheduling: the work that the system periodically performs, such as writing cached data to the hard disk, log cleaning, etc. There is a crontab file in the / etc directory, which is the configuration file for system task scheduling. The / etc/crontab file includes the following lines: [root@localhost ~] # cat / etc/crontabSHELL=/bin/bashPATH=/sbin:/bin:/usr/sbin:/usr/binMAILTO= "" HOME=/# run-parts51 * root run-parts / etc/cron.hourly24 7 * root run-parts / etc/cron.daily22 4 * * 0 root run-parts / etc/cron.weekly42 4 1 * * root run-parts / etc/cron.monthly

The first four lines are the environment variables used to configure the crond task to run. The first line SHELL variable specifies which shell the system will use, here is bash, the second line PATH variable specifies the path to which the system executes the command, and the third line MAILTO variable specifies that the task execution information of crond will be sent to the root user by email. If the value of the MAILTO variable is empty, the task execution information will not be sent to the user. The HOME variable on the fourth line specifies the home directory to be used when executing a command or script.

User task scheduling: tasks that users perform on a regular basis, such as user data backup, scheduled email reminders, etc. Users can use the crontab tool to customize their scheduled tasks. All user-defined crontab files are saved in the / var/spool/cron directory. Its file name is the same as the user name.

The meaning of the crontab file:

In the crontab file created by the user, each line represents a task, and each field on each line represents a setting. Its format is divided into six fields. The first five paragraphs are the time setting section, and the sixth paragraph is the command segment to be executed. The format is as follows:

Minute hour day month week command

Where:

Minute: represents minutes and can be any integer from 0 to 59.

Hour: represents an hour, which can be any integer from 0 to 23.

Day: represents the date and can be any integer from 1 to 31.

Month: represents the month and can be any integer from 1 to 12.

Week: represents the day of the week, which can be any integer from 0 to 7, where 0 or 7 represents Sunday.

Command: the command to be executed can be a system command or a script file written by yourself.

The following special characters can also be used in each of the above fields:

Asterisk (*): represents all possible values. For example, if the month field is an asterisk, it means that the command operation is performed every month after the constraints of other fields are met.

Comma (,): you can specify a list range with values separated by commas, for example, "1, 2, 5, 7, 8, 9"

Middle bar (-): you can use the middle bar between integers to represent a range of integers, for example, "2-6" means "2pens 3pr 4je 5pl 6"

Forward slash (/): the interval frequency of the time can be specified with a forward slash, for example, "0-23gam2" indicates that it is executed every two hours. At the same time, a forward slash can be used with an asterisk, such as * / 10, which, if used in the minute field, means that it is executed every ten minutes.

Common crontab commands:

1. List crontab files

To list crontab files, you can use:

[root@localhost ~] # crontab-l 0 dev/tty 15 30 Tokyo 45 18-06 * / bin/echo `date` > dev/tty

You will see something similar to the above. You can use this method to make a backup of the crontab file in the $HOME directory:

[root@localhost ~] # crontab-l > $HOME/mycron

In this way, once you accidentally delete the crontab file, you can restore it quickly using the method described in the previous section.

two。 Edit the crontab file

If you want to add, delete, or edit entries in the crontab file, and the editing environment variable is set to vi, you can edit the crontab file with vi, with the following command:

[root@localhost] # crontab-e

You can modify the crontab file and exit as you would any other file using vi. If some entries are modified or new entries are added, c r o n performs the necessary integrity checks on the file when it is saved. If one of these fields has a value that is out of the allowed range, it will prompt you.

When we edit the crontab file, we may add new entries. For example, add the following:

# DT:delete core files,at 3.30am on 1 bin/find 7 14 core' 26 days of each month (Note) 30 3 1 7 14 name 26 * / bin/find-name "core'-exec rm {}\

Now save and exit. It's best to put a comment on each entry in the crontab file so that you know its function, its running time, and more importantly, which user's job it is.

Now let's use the crontab-l command mentioned earlier to list all its information:

[root@localhost] # crontab-l # (crondave installed on Tue May 4 13:07:43 1999) # DT:ech the date to the console every 30 minites 015 minites 45 18-06 * / bin/echo `date` > / dev/tty1 # DT:delete core files,at 3.30am on 1 core' 21 days of each month 26 26 30 3 1 7 21 exec rm 26 * / bin/find-name "core'-exec rm {}\

3. Delete crontab Fil

To delete the crontab file, use:

[root@localhost ~] # crontab-r

Crontab usage example

Example 1: execute command every 1 minute

Command:

* command

Example 2: execute the rsync-sh script every Sunday at 00: 00 p.m.

Command:

[root@localhost ~] # crontab-e # DT:Execute rsync-sh script every Sunday at 00: 00 (Note) 00 * * 7 sh / root/rsyncd/rsync-gitbak.sh

The rsync client automatically synchronizes with the rsync server:

First, let's make a shell script

[root@localhost rsyncd] # vim rsyncd.shroud password-file=/rsyncd/rsyncd.pass-progress / data rsync@192.168.0.252::log

Command: crontab-e to edit the join schedule task

[root@localhost ~] # crontab-e # DT:Execute rsync-sh script every Sunday at 00: 00 (Note) 00 * * 7 sh / root/rsyncd/rsyncd.sh

Use crontab-l to view scheduled tasks joined

[root@localhost ~] # crontab-l00 * * 7 sh / root/rsyncd/rsyncd.sh

Note:

1. When the program runs at the time you specify, the system will send you a letter showing the contents of the program. If you do not want to receive such a letter, please add > / dev/null 2 > & 1 after each line is blank.

2.% is considered as newline in crontab, so you need to use\ to escape. For example, in the crontab execution line, if there is "date +% Y%m%d", it must be replaced by "date +\% Y\% m\% d".

Problems encountered and solutions:

Edit the sh script on Windows and upload it to Linux to report an error:

Bad interpreter:No such file or directory

Vim rsyncd.sh uses the command: set ff? Check whether it is in doc or unix format, and if it is in dos format, use the command: set ff=unix to convert to unix format

Thank you for reading! This is the end of this article on "how to synchronize and backup rsync+crontab under centos7". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, you can share it out for more people to see!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.