Shulou(Shulou.com)06/01 Report--

This article introduces the relevant knowledge of "how to install rsync and sersync in linux system". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

First, why use the Rsync+sersync architecture?

1. Sersync is developed based on Inotify, similar to Inotify-tools tools.

2. Sersync can record the name of a specific file or directory that has changed (including adding, deleting, modifying) in the listening directory, and then use rsync to synchronize only the changed file or directory.

Second, what is the difference between Rsync+Inotify-tools and Rsync+sersync?

1 、 Rsync+Inotify-tools

(1): Inotify-tools can only record changes in the monitored directories (including additions, deletions, and modifications), but not which files or directories have changed.

(2): rsync in synchronization, do not know which file or which directory has changed, every time is to synchronize the entire directory, when the amount of data is very large, the entire directory synchronization is very time-consuming (rsync to traverse the entire directory to find comparison files), therefore, the efficiency is very low.

2 、 Rsync+sersync

(1): sersync can record the name of a specific file or directory that has changed (including adding, deleting, modifying) in the listening directory.

(2): when rsync synchronizes, it only synchronizes the changed file or this directory (the data that changes each time is very small compared to the whole synchronized directory data, and rsync is very fast when traversing to find and compare files), so it is very efficient.

Summary: when the amount of synchronized directory data is small, it is recommended to use Rsync+Inotify-tools;. When the amount of data is large (hundreds of gigabytes or even 1T or more) and there are many files, it is recommended to use Rsync+sersync.

Description:

Operating system: CentOS 5.x

Source server: 192.168.21.129

Target server: 192.168.21.127192.168.21.128

Purpose: to synchronize the / home/www.yisu.com directory on the source server to the / home/www.yisu.com of the target server in real time

Specific operations:

The first part: operate on two target servers 192.168.21.127192.168.21.128 respectively

First, install the Rsync server on two target servers

1. Close SELINUX

Vi / etc/selinux/config # Editing Firewall profile

The code is as follows:

# SELINUX=enforcing # comment out

# SELINUXTYPE=targeted # comment out

SELINUX=disabled # increased

: wq! # Save, exit

Setenforce 0 # effective immediately

2. Open firewall tcp port 873 (Rsync default port)

Vi / etc/sysconfig/iptables # Editing Firewall profile

The code is as follows:

-A RH-Firewall-1-INPUT-m state-- state NEW-m tcp-p tcp-- dport 873-j ACCEPT

: wq! # Save exit

/ etc/init.d/iptables restart # finally restart the firewall for the configuration to take effect

3. Install Rsync server software

Yum install rsync xinetd # installation

Vi / etc/xinetd.d/rsync # edit the configuration file and set the boot rsync

The code is as follows:

Disable = no # changed to no

: wq! # Save exit

/ etc/init.d/xinetd start # startup (xinetd is used to manage Rsync services in CentOS)

4. Create a rsyncd.conf configuration file

Vi / etc/rsyncd.conf # create a configuration file and add the following code

The code is as follows:

Log file = / var/log/rsyncd.log # log file location, which is automatically generated after starting rsync, without the need to create it in advance

Pidfile = / var/run/rsyncd.pid # location of the pid file

Lock file = / var/run/rsync.lock # Lock files that support max connections parameters

Secrets file = / etc/rsync.pass # user authentication configuration file, which stores the user name and password, which will be created later

Motd file = / etc/rsyncd.Motd # rsync Welcome information page file location at startup (file content customization)

[home_www.yisu.com] # Custom name

Path = / home/www.yisu.com/ # rsync server data directory path

Comment = home_www.yisu.com # module name is the same as [home_www.yisu.com] custom name

Uid = root # set the rsync operation permission to root

Gid = root # set the rsync operation permission to root

Port=873 # default port

Use chroot = no # defaults to true, modifies it to no, and adds backup of soft connections to directory files

Read only = no # set rsync server files to read and write permissions

List = no # does not display the list of rsync server resources

Max connections = 200 # maximum connections

Timeout = 600 # set timeout

Auth users = home_www.yisu.com_user # user name to perform data synchronization. Multiple user names can be set, separated by commas in English.

Hosts allow = 192.168.21.129 # client IP addresses that allow data synchronization. Multiple addresses can be set, separated by commas in English.

Hosts deny = 192.168.21.254 # client IP addresses that prohibit data synchronization. Multiple addresses can be set, separated by commas in English.

: wq! # Save, exit

5. Create a user authentication file

Vi / etc/rsync.pass # configuration file, add the following

The code is as follows:

Home_www.yisu.com_user:123456 # format, user name: password, multiple can be set, one user name per line: password

: wq! # Save exit

6. Set file permissions

The code is as follows:

Chmod / etc/rsyncd.conf # sets the read and write permissions of the file owner

Chmod / etc/rsync.pass # sets the read and write permissions of the file owner

7. Start rsync

The code is as follows:

/ etc/init.d/xinetd start # launch

Service xinetd stop # stop

Service xinetd restart # restart

Part two: operating on the source server 192.168.21.129

Install the Rsync client

1. Close SELINUX

Vi / etc/selinux/config # Editing Firewall profile

The code is as follows:

# SELINUX=enforcing # comment out

# SELINUXTYPE=targeted # comment out

SELINUX=disabled # increased

: wq! Save exit

Setenforce 0 takes effect immediately

2. Open firewall tcp port 873 (Rsync default port, Rsync as a client does not have to open port 873)

Vi / etc/sysconfig/iptables # Editing Firewall profile

The code is as follows:

-A RH-Firewall-1-INPUT-m state-- state NEW-m tcp-p tcp-- dport 873-j ACCEPT

: wq! Save exit

/ etc/init.d/iptables restart # finally restart the firewall for the configuration to take effect

3. Install Rsync client-side software

The code is as follows:

Whereis rsync # check to see if rsync is installed on the system, and the following prompt appears indicating that it has been installed

Rsync: / usr/bin/rsync / usr/share/man/man1/rsync.1.gz

Yum install xinetd # can only install xinetd. Xinetd is used in CentOS to manage rsync services.

Yum install rsync xinetd # if there is no rsync by default, run this command to install rsync and xinetd

Vi / etc/xinetd.d/rsync # edit the configuration file and set the boot rsync

Disable = no # changed to no

/ etc/init.d/xinetd start # startup (xinetd is used to manage rsync services in CentOS)

4. Create authentication password file

Vi / etc/passwd.txt # edit the file to add the following

The code is as follows:

123456 # password

: wq! Save exit

Chmod / etc/passwd.txt # sets file permissions, and only sets the file owner with read and write permissions

5. Test data synchronization between source server 192.168.21.129 and two target servers 192.168.21.127192.168.21.128

The code is as follows:

Mkdir / home/www.yisu.com/ceshi # create a test folder on the source server, and then run the following two lines of command on the source server

Rsync-avH-port=873-progress-delete / home/www.yisu.com/ home_www.yisu.com_user@192.168.21.127::home_www.yisu.com-password-file=/etc/passwd.txt

Rsync-avH-port=873-progress-delete / home/www.yisu.com/ home_www.yisu.com_user@192.168.21.128::home_www.yisu.com-password-file=/etc/passwd.txt

After the operation is completed, check the two target servers 192.168.21.127192.168.21.128 respectively, and there is a ceshi folder in the / home/www.yisu.com directory, indicating that the data synchronization is successful.

System operation and maintenance www.yisu.com warm reminder: the original content of qihang01 all rights reserved, reprint please indicate the source and the original link

Install sersync tools to trigger rsync synchronization in real time

1. Check whether the server kernel supports inotify.

Ll / proc/sys/fs/inotify # lists the file directory and the following appears, indicating that the server kernel supports inotify

The code is as follows:

-rw-r--r-- 1 root root 0 Mar 7 02:17 max_queued_events

-rw-r--r-- 1 root root 0 Mar 7 02:17 max_user_instances

-rw-r--r-- 1 root root 0 Mar 7 02:17 max_user_watches

Note: the minimum kernel that supports inotify under Linux is 2.6.13. You can enter the command: uname-a to view the kernel.

The kernel of CentOS 5.x is 2.6.18, and inotify is supported by default.

2. Modify the default parameters of inotify (the default kernel parameter of inotify is too small)

View the system default parameter values:

Sysctl-a | grep max_queued_events

The result is: fs.inotify.max_queued_events = 16384

Sysctl-a | grep max_user_watches

The result is: fs.inotify.max_user_watches = 8192

Sysctl-a | grep max_user_instances

The result is: fs.inotify.max_user_instances = 128

Modify the parameters:

The code is as follows:

Sysctl-w fs.inotify.max_queued_events= "99999999"

Sysctl-w fs.inotify.max_user_watches= "99999999"

Sysctl-w fs.inotify.max_user_instances= "65535"

Parameter description:

Max_queued_events:

Maximum length of inotify queue. If the value is too small, an "* * Event Queue Overflow * *" error will occur, resulting in inaccurate monitoring files.

Max_user_watches:

To synchronize files, you can use: find / home/www.yisu.com-type d | wc-l statistics. You must ensure that the max_user_watches value is greater than the statistical result (where / home/www.yisu.com is the synchronization file directory)

Max_user_instances:

Maximum value for each user to create an inotify instance

3. Install sersync

Sersync download address: https://sersync.googlecode.com/files/sersync2.5.4_64bit_binary_stable_final.tar.gz

Upload sersync2.5.4_64bit_binary_stable_final.tar.gz to / usr/local/src directory

The code is as follows:

Cd / usr/local/src

Tar zxvf sersync2.5.4_64bit_binary_stable_final.tar.gz # decompression

Mv GNU-Linux-x86 / usr/local/sersync # move directory to / usr/local/sersync

4. Configure sersync

The code is as follows:

Cd / usr/local/sersync # enter the sersync installation directory

Cp confxml.xml confxml.xml-bak # back up the original file

Edit the vi confxml.xml to modify the following code

The code is as follows:

Http://pic.xoyo.com/cms"/>

: wq! # Save exit

Parameter description:

The code is as follows:

Localpath watch= "/ home/www.yisu.com": # Source server synchronization directory

192.168.21.127192.168.21.128blog # destination server IP address

Name= "home_www.yisu.com": # Target server rsync synchronization directory module name

Users= "home_www.yisu.com_user": # Target server rsync synchronization user name

Passwordfile= "/ etc/passwd.txt": # destination server rsync the path where the password of the synchronization user is stored on the source server

Remote ip= "192.168.21.127": # Target server ip, one per line

Remote ip= "192.168.21.128": # Target server ip, one per line

FailLog path= "/ tmp/rsync_fail_log.sh" # script run failure logging

Start= "true" # is set to true, and full synchronization is performed every 600min

5. Set the automatic execution of sersync monitoring on boot.

Edit vi / etc/rc.d/rc.local # and add a line at the end

The code is as follows:

/ usr/local/sersync/sersync2-d-r-o / usr/local/sersync/confxml.xml # set the boot auto-run script

: wq! # Save exit

6. Add a script to monitor whether sersync is running properly.

Vi / home/crontab/check_sersync.sh editor, add the following code

The code is as follows:

#! / bin/sh

Sersync= "/ usr/local/sersync/sersync2"

Confxml= "/ usr/local/sersync/confxml.xml"

Status=$ (ps aux | grep' sersync2' | grep-v 'grep' | wc-l)

If [$status-eq 0]

Then

$sersync-d-r-o $confxml &

Else

Exit 0

Fi

The code is as follows:

: wq! # Save exit

Chmod + x / home/crontab/check_sersync.sh # add script execution permissions

Edit vi / etc/crontab # and add the following line at the end

* / 5 * root / home/crontab/check_sersync.sh > / dev/null 2 > & 1 # execute the script every 5 minutes

Service crond reload # reload service

6. Test whether the sersync real-time trigger rsync synchronization script is running properly.

Create the file inotify_rsync_ceshi on the source server 192.168.21.129

The code is as follows:

Mkdir / home/www.yisu.com/inotify_rsync_ceshi

Restart the source server: 192.168.21.129

After the system starts, check to see if there is an inotify_rsync_ceshi folder under / home/www.yisu.com of the two target servers 192.168.21.127192.168.21.128

Then create the folder inotify_rsync_ceshi_new on the source server 192.168.21.129

The code is as follows:

Mkdir / home/www.yisu.com/inotify_rsync_ceshi_new

Continue to see if there is an inotify_rsync_ceshi_new folder under / home/www.yisu.com of the two target servers 192.168.21.127192.168.21.128

If all the above tests pass, the inotify real-time trigger rsync synchronization script is running normally.

At this point, real-time data synchronization is completed by Rsync+sersync under Linux.

Extended reading:

Rsync parameter

-v,-- verbose verbose mode output

-Q,-- quiet compact output mode

-c,-- checksum turns on the check switch to force the file transfer to be checked

-a,-- archive archiving mode, which means that files are transferred recursively and all file attributes are maintained, which is equal to-rlptgoD

-r,-- recursive processes subdirectories in recursive mode

-R,-- relative uses relative path information

-b,-- backup creates a backup, that is, rename the old file to ~ filename when the same file name already exists for the destination. You can use the-- suffix option to specify different backup file prefixes.

-- backup-dir stores backup files (such as ~ filename) in a directory.

-suffix=SUFFIX defines the backup file prefix

-u,-- update only updates, that is, skips all files that already exist in DST and the file time is later than the file to be backed up. (do not overwrite updated files)

-l,-- links retains soft links

-L,-- copy-links handles soft links like regular files

-- copy-unsafe-links only copies links that point outside the SRC path directory tree

-- safe-links ignores links that point outside the SRC path directory tree

-H,-- hard-links retains hard links

-p,-- perms maintains file permissions

-o,-- owner keeps file owner information

-g,-- group keeps file group information

-D,-- devices keeps device file information

-t,-- times keeps file time information

-S,-- sparse performs special processing on sparse files to save DST space

-n,-- dry-run reality which files will be transferred

-W,-- whole-file copies files without incremental detection

-x,-- one-file-system do not cross file system boundaries

-B,-- the block size used by the block-size=SIZE check algorithm. The default is 700byte.

-e,-- rsh=COMMAND specifies to use rsh and ssh methods for data synchronization

-- rsync-path=PATH specifies the path information where the rsync command is located on the remote server

-C,-- cvs-exclude automatically ignores files using the same method as CVS to exclude files that you don't want to transfer

-- existing updates only those files that already exist in DST, not those newly created files

-- delete deletes files in DST that are not available in SRC

-- delete-excluded also deletes files on the receiver that are specified and excluded by this option

-- delete delete-after after the transfer is finished.

Delete ignore-errors even if there are IO errors.

-- max-delete=NUM deletes a maximum of NUM files

-- partial retains files that were not fully transferred for some reason, thus speeding up subsequent retransfers

-- force forces directory deletion, even if it is not empty

-- numeric-ids does not match numeric user and group ID to username and group name

-- timeout=TIME IP timeout in seconds

-I,-- ignore-times does not skip files of the same time and length

-- size-only when deciding whether to back up a file, only look at the file size without considering the file time

-- the timestamp window used by modify-window=NUM to determine whether the file is at the same time. The default is 0.

-T-- temp-dir=DIR creates temporary files in DIR

-- compare-dest=DIR also compares files in DIR to decide whether a backup is needed

-P equals-- partial

-- progress shows the backup process

-z,-- compress compresses the backed-up files during transfer

-- exclude=PATTERN specifies to exclude files that do not need to be transferred

-- include=PATTERN specifies the file mode that needs to be transferred without exclusion

-- exclude-from=FILE excludes files of the specified pattern in FILE

-- include-from=FILE does not exclude files specified by FILE for pattern matching

-- version print version information

-- address is bound to a specific address

-- config=FILE specifies other configuration files and does not use the default rsyncd.conf file

-- port=PORT specifies other rsync service ports

-- blocking-io uses blocking IO for remote shell

-stats gives the transfer status of some files

-- the actual transmission process of progress during transmission

-- log-format=formAT specifies the log file format

-- password-file=FILE gets the password from FILE

-- bwlimit=KBPS restricts the bandwidth of IBO, KBytes per second

-h,-- help displays help information

This is the end of the content of "how to install rsync and sersync in linux system". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.