Shulou(Shulou.com)06/02 Report--

1. Data synchronization:

1. Data synchronization:

A directory on server An is synchronized to a directory on server B.

The method is: you can share this directory on server A (such as ftp,http, etc.), and then write shell script on server B and download it with crontab for scheduled tasks (data synchronization is not real-time)

Real-time synchronization

The method is: rsync remote directory real-time synchronization

Real-time synchronization of drbd remote storage (very old scheme)

Shared storage means that the same disk device is used by many people at the same time

Distributed storage (cloud storage)

2. Synchronous, asynchronous and remote synchronization

Sync synchronization

Async async

Rsync remote synchronization

The main point here is rsync. Before you start, you need to know the following questions:

What is rsync; how rsync works; advantages and disadvantages of rsync; rsync authentication mode; detailed description of installation rsync;rsync command parameters; configuration of rsync;rsync startup and shutdown; rsync synchronization instance

2.1 what is rsync

Rsync (remote synchronize) is a remote data synchronization tool under Liunx/Unix. It can quickly synchronize files and directories among multiple hosts through LAN/WAN, and make proper use of rsync algorithm (differential coding) to reduce data transmission.

Rsync will transfer all files for the first time, and then compare the differences between the two files. Only the updated parts of the files will be transferred, and the transfer speed is faster than that of ordinary tools.

Rsync can also copy and display directory attributes and copy files, and optionally compress and recursively copy.

2.2 how rsync works

2.2.1 after the command or script runs, the synchronizer starts to construct FileList,FileList, that is, the index of the file that needs to be synchronized by the synchronizer, name- > id (each file has an id value, such as MD5)

2.2.2 after the construction of the synchronous end is completed, the FileList is sent to the synchronized side, and the sent FileList is processed by the synchronized side rsync, and then the synchronization begins.

2.2.3 in the second synchronization, because there are files on both sides, the synchronizer constructs the FileList on the sending side, and the rsync of the synchronizer handles the FileList sent by the synchronizer. According to the comparison of MD5 values, it deletes the same file information on the sync side and retains the files that do not exist and updated to build a new FileList.

2.2.4 the synchronized side receives the new FileList of the synchronized side, and then starts to synchronize the files to the synchronized side.

2.3 rsync is a little bit

Advantages:

High security

Backup quickly, use synchronization algorithm, only compare changes

Support for incremental backup

Save the source directory, the entire directory tree and file system

Maintain the permissions, time, soft and hard connections of the source file, etc.

Can be installed without special envoy permission

Optimize the process and high file transfer efficiency.

Transfer files in a variety of ways.

Anonymous operation is supported to facilitate website mirroring.

Compared with scp, the transmission speed of rsync is much higher than scp. (local area network test, rsync is 20 times that of scp)

Solve the data backup requirements with low real-time requirements

Deficiency:

Synchronous data, need to scan all the files for comparison, before the differential transmission. If the number of files reaches millions or even tens of millions, scanning files to compare files will be very time-consuming, reducing the efficiency of rsync.

Rsync can not monitor and synchronize data in real time. Although synchronization can be triggered by a daemon, there is a time difference between the two actions, resulting in inconsistent data and unable to fully recover data in the event of a failure.

Solution: rsync+inotify to make up for the deficiency.

2.4 Authentication method

2.4.1 two ways

Rsync-daemon

Ssh

Note: when using rsync, both the synchronized side and the synchronized side must install the rsync program.

2.4.2 rsync-daemon Certification

In rsync-daemon authentication mode, port 873 of poison tcp is listened by default.

Under rsync-daemon authentication, rsync can write passwords to a file.

Note: the rsync on the synchronized side must be started, configuration file. The synchronous side may not start the service, but there must be this application.

2.4.3 ssh Certification

Transfer through the ssh tunnel, similar to the scp tool, synchronization is not limited to the synchronous folder defined in rsync, and as long as both parties install rsync, do not start rsync on both sides.

For example: rsync-avz / root/test root@192.168.19.248:/root/

If the rsync server SSH is a non-standard port, you can specify the port through the-e parameter of rsync. The mode of use is as follows:

Rsync-avz / root/test-e'ssh-p1234' root@192.168.19.248:/root/

2.5 commonly used tools and examples

2.5.1 wget tool

Wget http://10.1.1.9/linux_basic01.txt-- download directly to the current directory wget-- no-check-certificate https://www.baidu.com/img/bd_logo1.png # with https connection When downloading, you need to add-- no-check-certificate parameter wget http://10.1.1.9/linux_basic01.txt-O / test/123 # download the file and rename it to / test/123wget-m-k-np http://10.1.1.9/shell/# only download the shell directory, but be sure to add / wget-m http://www.xxx.com# after shell to download the entire website.

2.5.2 convert local yum source:

How to download an extended yum source for epel and convert it to a local yum source

Download using wget; download using reposync

The steps for downloading reposync are as follows:

First write down the path of the yum source you want to download in the local yum configuration, such as

# vim / etc/yum.repos.d/rhel-source.repo [epel] server=epelbaseurl= http://xxx.xxx.xxx/-- network path gpgcheck=0enabled=1# yum repolist-- list the names of all your current configured repo # reposync-- repoid=epel-- download all the software packages in the epel path to the local current directory, download to the local directory, cd, and use createrepo. Command to create the inventory directory of the software warehouse repodata change the writing of yum to # vim / etc/yum.repos.d/rhel-source.repo [epel] server=epelbaseurl= file:///xxx/-- write your local directory path (that is, the parent directory address of repodata) gpgcheck=0enabled=1

3 install rsync

Two ways: source code installation and rpm installation

3.1 Source code installation

3.1.1 download

First go to the official website to download the source package.

Https://rsync.samba.org/wget-- no-check-certificate https://download.samba.org/pub/rsync/rsync-3.1.2.tar.gz

If you have a https connection, you need to add-- no-check-certificate parameter when downloading.

3.1.2 installation

[root@localhost rsync-3.1.2] # tar xf rsync-3.1.2.tar.gz [root@localhost rsync-3.1.2] # cd rsync-3.1.2 [root@localhost rsync-3.1.2] #. / configure [root@localhost rsync-3.1.2] # make & & make install Note: if you need to install the gcc library in advance, it will be installed under / usr/local/ by default, via. / configure-- help

After the installation is complete, check the rsync help. You need to change to the / usr/local/bin directory: / usr/local/binrsync-h [root@localhost rsync-3.1.2] # cd / usr/local/bin [root@localhost bin] # lltotal 1344-rwxr-xr-x 1 root root 1373922 May 16 18:46 rsync [root@localhost bin] # pwd/usr/local/bin source code installed rsync, you need to boot. Put the startup command in the rc.local file. As follows: echo "/ usr/local/bin/rsync-daemon-config=/etc/rsyncd.conf" > > / etc/rc.local can also be written as: echo "/ usr/local/bin/rsync-daemon" > > / etc/rc.localcat / etc/rc.local

3.2 installation in RPM mode

Direct yum installation: yum-y install rsync looks at the rsync installation location as follows: [root@localhost bin] # rpm-ql rsyncrsync is installed to / usr/bin and generates a / etc/xinetd.d/rsync file. Check out the help documentation generated during installation: [root@localhost bin] # more / usr/share/doc/rsync-3.0.6/READMEwhich rsyncrsync-- help

Can rsync be made into a service with some control functions or logging functions?

3.3.1 two knowledge points:

Xinetd

Man rsyncd.conf

Two coexist together and control rsync together.

3.3.2 introduction to xinetd

Xinetd is similar to a managed service, hosting other small services, the advantage is that you can use the powerful parameters of xinetd to control these services

Such as:

/ etc/xinetd.conf-- main configuration file, generally no configuration, main configuration sub-configuration file / etc/xinetd.d-- sub-profile directory xinetd service is equivalent to the master switch of all towed services # yum install-y xinetdxinted requires configuration file: vi / etc/xinetd.d/rsync-(man xinetd.conf reference help) service rsync {disable = no-yes means off, no means on (start rsync service) Two conditions: 1 Disable = no 2 Xinetd service to start) flags = IPv6socket_type = streamwait = no-- concurrent connection user = root-- user running daemon server = / usr/bin/rsync-- startup program path server_args =-- daemon-config=/etc/rsyncd.conf-- startup parameters (taken together, these three parameters mean to start the rsync service with root identity / usr/bin/rsync-- daemon) log_on_failure + = USERID-- controls the log format Indicates that when the login service fails, add a record USERID} to the original log format and install the xinted package after configuration. Otherwise, you cannot start xinetdroot@localhost bin] # yum install-y xinetd [root@localhost bin] # / etc/init.d/xinetd startStarting xinetd: [OK] [root@localhost bin] # / etc/init.d/xinetd startStarting xinetd: [OK] [root@localhost bin] # chkconfig xinetd on [ Root@localhost bin] # netstat-tlunp | grep 873tcp 0 0: 873: * LISTEN 8831/xinetd

4 detailed explanation of rsync command parameters

The best way to view the parameters is to check the help information:

Rsync-help

4.1 Common parameters

-v,-verbose verbose mode output.

-a,-archive archiving mode, which means that files are transferred recursively, leaving all file attributes unchanged.

-l keep soft links

-R preserves the relative path

-H retains hard links

-pmam quotient, copyright, group, acl, etc., respectively, but if you add-a, these will be included.

-z,-compress compresses the backed-up files during transfer.

-D equals-- devices-- specials indicates a file that supports the type bmeno cjorsrep.

-delete: delete files that exist in DST but not in SRC.

4.2 General applications

Rsync-av / home/ / backup/-synchronization on the local machine, synchronizing the contents of the / home directory to the / backup directory (including hidden files) rsync-av / home/ 192.168.19.248:/backup/-putting the local contents of the / home directory Synchronize to the / backup directory of the remote 218, need the remote password of 192.168.19.248 rsync-av 192.168.19.248:/backup/ / backup/-synchronize the contents of the / backup directory of the remote 218 to the local / backup directory Need the remote password of 192.168.19.248 rsync-a / test/'- e ssh-p 2222' 192.168.19.248:/test1/-if the ssh port of the remote machine 192.168.19.248 is not the default 22, such as 2222, then you need to write this.

Questions to be noted:

Keep the original path when synchronizing, that is, synchronizing / home to / backup will become / backup/home/* re-content

Whether or not you add / after the original directory also affects your synchronization.

There is a difference between the following three:

The content under rsync-av / home/ / backup/-- / home is directly synchronized to / backup rsync-av / home/ backup/rsync-aRv / home/ / backup/-- where / home is synchronized into / backup/home/* # rsync-av / share/20160516 / backup/-- is synchronized into / backup/20160516/* # rsync-aRv / share/20160516 / backup/-- is synchronized into / backup/share/20160516 / * if a file in the source directory is deleted Is the synchronized target directory deleted or not deleted? Can it be controlled by parameters?-- delete# rsync-aHAX-- delete / test/ / test1/-- A more complete extension of file extended attributes# lsattr / etc/shadow-- list file attributes-e-/ etc/shadow# chattr + I / etc/shadow-add an I attribute to the file (I attribute indicates that the file is completely immutable and cannot be deleted) It will prevent all users from changing the password # lsattr / etc/shadow----i-e- / etc/shadow# chattr-I / etc/shadow-- and remove the I attribute.

5 configure rsync

5.1rsync is configured by the synchronous side

Two files: rsyncd.conf rsyncd.password

Literally, one is the main configuration file, and the other is to store the password of the authenticated user. When the installation of rsync is complete, these two configuration files are not generated and need to be created manually. Cat / etc/rsyncd.conf-this file does not exist and needs to be created manually (man rsyncd.conf reference help). The path must be / etc/rsyncd.conf

Motd file=/etc/rsyncd.welcome-Welcome document Path Custom uid = rootgid = rootuser chroot = nomax connections = 200timeout = 600pid file = / var/run/rsyncd.pidlock file = / var/run/rsyncd.locklog file = / var/run/rsyncd.log [backup]-name Custom path = / backup/-the server path of the actual share The name must be the actual path you want to share ignore errorsread only = nolist = nohosts allow = 192.168.19.0/255.255.255.0auth users = testsecrets file = / etc/rsyncd.password [www] path = / www/ignore errorsread only = nolist = nohosts allow = 192.168.19.0/255.255.255.0auth users = apachesecrets file = / etc/rsyncd.passwordvim / etc/rsyncd.welcome-- manually create your own defined motd file And write the custom welcome information rsyncd.password configuration echo "test:test" > / etc/rsyncd.passwordecho "apache:apache" > > / etc/rsyncd.passwordcat / etc/rsyncd.password

Note: the user and password in this password file can be customized, and the user does not have to be a system user in the server.

After the password file is created, its permissions should be set to 600, otherwise it will prompt for a verification error when synchronizing later. As follows:

Chmod 600 / etc/rsyncd.password

After the above two files are created, let's create their corresponding directories and authorize them, as follows:

Mkdir / {backup,www} chown root:root-R / backup/chown root:root-R / www/

5.2 synchronous side configuration

You don't need to start the rsync service, just create a password file.

Echo "apache" > > / etc/rsyncd.password

Chmod 600 / etc/rsyncd.password

Note: only one user's password can be stored in this password file, and this file, like the password file on the rsync server, needs to set its permissions to 600.

5.3 Verification

# rsync-a 10.1.1.9 rsyncd.welcome notes-directly this way you can see what "rsync! @ _ @" is shared by the server 10.1.1.9-- this is the welcome content notes written in your rsyncd.welcome-- this is the name of the share # share-a share / share

In this way, no password is required for synchronization. Note that notes is the shared name, followed by two:: followed by / notes for the local directory

There are both advantages and disadvantages of not having a password. The advantage is that you don't need a password to make it easy to write scripts for remote synchronization. If you want a password to write a script, you need to use expect auto-answer. The downside is that it is not safe, but you can use iptables and so on to enhance security.

If you want to have a password and do not need to pass a password, you can use the secrets file that comes with rsyncd as the authentication user.

5.4 problems with synchronization

If you synchronously report to permission denied, it may be that the server selinux is not closed.

How to turn off selinux:

# setenforce 0-this immediately changes selinux from mandatory mode to warning mode

# vim / etc/selinux/config

SELINUX=disabled-- changed this to disabled, and then you need to restart the computer to take effect.

Synchronization time is slow

Solution: bind the host name of the other party

5.5 add some functions to the above

Request:

Log to / var/log/rsyncd.log

The shared module is required to be hidden (that is, the client cannot view the module name)

And only one client connection can synchronize the module at the same time.

Only 192.168.19.x (ip you customize) is allowed to synchronize this module

Can only synchronize from 9: 00 a.m. to 6: 00 p.m.

Achieve:

Method 1: add the parameter log_type file / var/log/rsyncd.log to / etc/xinetd.d/rsync

Method 2: add the parameter log_type syslog local5 to / etc/xinetd.d/rsync

Method 3: add the parameter log file=/var/log/rsyncd.log to / etc/rsyncd.conf

Method 4: add the parameter syslog facility=local5; to / etc/rsyncd.conf-- but this sentence has been tested in rhel6.5 and must be placed in global configuration, but cannot generate logs in modules configuration.

Methods 2 and 4 also need to add local5.* / var/log/rsyncd.log to / etc/rsyslog.conf

# vim / etc/rsyncd.confmotd file=/etc/ rsyncd.notes [notes] path=/share/20160516list=falsemax connections=1log file=/var/log/rsyncd.loghosts allow=192.168.19.x # vim / etc/xinetd.d/rsyncaccess_times = 9:00-18 path=/share/20160516list=falsemax connections=1log file=/var/log/rsyncd.loghosts allow=192.168.19.x / etc/init.d/xinetd restart

6. Startup and shutdown of rsync

6.1 start the service

Rsync installed in the source package, using rsync-daemon to start rsync.

Echo PATH=$PATH:/usr/local/bin/ > > / etc/profilesource / etc/profilersync-daemon # launch command ps aux | grep rsyncnetstat-tunlp | grep 873

If the configuration is not in the default location, you need to manually load rsync-daemon-config=/etc/rsyncd.conf

If you load, you can view the help documentation: rsync-daemon-help

If rpm is added to xinetd, just start xinetd directly

6.2 shut down the service

Source code pkill rsync

Rpm / etc/init.d/xinetd stop

Implement real-time synchronization (that is, as soon as the source changes, it triggers synchronization)

7 inotify

What is 7.1 inotify?

Inotify is a powerful, fine-grained, asynchronous file system event monitoring mechanism.

Through inotify, you can monitor various minor events in the file system, such as adding, deleting, modifying, moving, and so on. Using this kernel interface, third-party software can monitor the changes of files under the file system, and inotify-tools is such a third-party software. Inotify happens to make up for the real-time problem of rsync synchronous data.

7.2 Analysis of inotify related parameters

Inotify defines three interface parameters that can be used to limit the amount of kernel memory consumed by inotify.

These parameters are memory parameters and can be resized in real time according to your needs.

/ proc/sys/fs/inotify/max_queued_evnets

Represents the maximum number of request events, and events beyond this value are discarded. The value defaults to 16384.

Note: max_queued_events is the maximum length of queues managed by inotify, and the more frequently the file system changes, the greater this value should be.

If you see Event Queue Overflow in the log, it means that max_queued_ events is too small and needs to be used again after adjusting the parameters.

/ proc/sys/fs/inotify/max_user_instances

Represents the maximum number of instances that can be created per user. The value defaults to 128.

/ proc/sys/fs/inotify/max_user_watches

Represents the maximum number of directories that can be monitored. The value defaults to 8192.

To modify the above default values, we can use the following similar means. As follows:

Echo 30000000 > / proc/sys/fs/inotify/max_user_watches

8 inotify monitoring file event types

The file system event types that can be monitored by inotify are as follows:

IN_ACCESS: the file is accessed.

IN_MODIFY: the file is write.

IN_ATTRIB: file properties have been modified, such as chmod, chown, etc.

IN_CLOSE_WRITE: writable files are close.

IN_CLOSE_NOWRITE: non-writable files are close.

IN_OPEN: the file is open.

IN_MOVED_FROM: files are moved out of the monitored directory, such as mv.

IN_MOVED_TO: files are moved to monitored directories, such as mv and cp.

IN_CREATE: files / folders are created.

IN_DELETE: files / folders are deleted, such as rm.

IN_DELETE_SELF: self-deletion, that is, an executable file deletes itself during execution.

IN_MOVE_SELF: self-moving, that is, an executable file moves itself during execution.

IN_UNMOUNT: the host file system is umount.

IN_CLOSE: the file is closed, which is equivalent to (IN_CLOSE_WRITE | IN_CLOSE_NOWRITE).

IN_MOVE: the file is moved, which is equivalent to (IN_MOVED_FROM | IN_MOVED_TO).

Note: the files mentioned above also include directories.

What is 9 inotify-tools?

Inotify is an API that needs to be called by developing an application. Inotify-tools is such an inotify software, it is a set of components, including a C library and several command-line tools.

Inotify is designed to replace dnotify. It overcomes the shortcomings of dnotify and provides a more user-friendly, more concise and powerful file change notification mechanism.

1) inotify does not need to open a file descriptor for the monitored target. If the monitored target is on a removable media, the watch corresponding to the monitored target will be automatically deleted after umount the file system on that media, and a umount event will be generated.

2) inotify can monitor both files and directories.

3) inotify uses system calls instead of SIGIO signals to notify file system events.

4) inotify uses the file descriptor as the interface, so you can use the usual file Icano operations select and poll to monitor file system changes.

10 install inotify-tools

Two ways: source code and rpm

Note: inotify-tools works mainly through inotifywait and inotifywatch, two commands. The inotifywait command is the most frequently used command in a production environment.

10.1 Source package installation

Download address: https://github.com/rvoicilas/inotify-tools/wiki

Note: it is not recommended to use wget to download inotify-tools, because the download was not successful due to website reasons, wget https://cloud.github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz

We can download locally through the browser first, and then upload it to the server through the rz command.

To use the rz command, we need to install the lrzsz software, as follows:

Yum-y install lrzsztar-xf inotify-tools-3.14.tar.gz./configuremake&&make installls / usr/local/bin/# / usr/local/bin/inotifywait-- help/usr/local/bin/inotifywait: error while loading shared libraries: libinotifytools.so.0: cannot open shared object file: No such file or director if an error is reported, the library cannot be found This library is actually the solution # echo / usr/local/lib/ > / etc/ld.so.conf.d/abc.conf# ldconfig under / usr/local/lib

10.2 rpm package installation

Configure yum Feed

We can download the latest yum source at the following link, as follows:

Http://dl.fedoraproject.org/pub/epel/6/x86_64/ downloads the epel-release-6-8.noarch.rpm package and installs it as follows: wget http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm rpm-ivh epel-release-6-8.noarch.rpmyum-y install inotify-toolsrpm-ql inotify-tools after installation The following two commands [root@localhost test] # ls / usr/bin/inotify*/usr/bin/inotifywait / usr/bin/inotifywatch are generated

Detailed explanation of 11.inotifywait usage

Inotifywait is a monitoring wait event, which is mainly used to monitor changes in files or directories, and can recursively monitor the entire directory tree.

11.1. Detailed explanation of inotifywait command

Use inotifywait-h to view help information

-m means to keep event listening at all times.

-r represents a recursive query directory.

-Q indicates that a monitoring event is printed.

-e this parameter allows you to specify events to monitor. Events that can be monitored are as follows:

Access: access, read files.

Modify: modified, the content of the file is modified.

Attrib: attribute, file metadata is modified.

Move: move, move files. And rename, rename the file.

Create: creating, generating new fil

Open: open, open the file.

Close: close, close the file.

Delete: delete, file is deleted.

-timefmt is the output format for the specified time and is used for the% T format in the-format option.

-format specifies the details output format for file changes. The format parameters are as follows:

W represents the directory where the event occurred

% f represents the file in which the event occurred

% e indicates the event that occurred

% T uses the time format defined by-timefmt

Xe events are separated by "X"

11.2 inotifywait command instance

We want to monitor the changes in all files and directories under the / home directory, with the following command:

Inotifywait-mrq-timefmt'% y modify,delete,create,attrib,move,open,close,access% m modify,delete,create,attrib,move,open,close,access% d max% HGV% M'-format'% T% w% f% e'- e charger / home/

This command monitors the operation of all files and directories in the / home directory.

Open two windows of a machine, create files and check the status to see if there will be any changes.

12. Detailed explanation of the use of inotifywatch

Inotifywatch is mainly used to collect monitored file system statistics, including information such as how many times each inotify event occurs.

Detailed explanation of 12.1.inotifywatch command

Inotifywatch-h view help information

The inotifywatch parameters are described as follows:

-h: output help information.

-v: outputs details.

@: exclude files that do not need to be monitored, either relative or absolute.

-- fromfile: reads files that need to be monitored or excluded from files, one line for each file, and the excluded files start with @.

-z: outputs the rows and columns of the table, even if the element is empty.

-- exclude: regular matching of files to be excluded, case-sensitive.

-- excludei: regularly matches files that need to be excluded, ignoring case.

-r: monitor all subdirectories under a directory.

-t: sets the timeout.

-e: only listens for specified events. This event is the same as the event type that inotifywait listens on.

-a: sorts in ascending order of specified events.

-d: sorts in descending order of specified events.

To count the events of the file system in the / home directory within 60 seconds, use the following command:

Inotifywatch-v-e modify,delete,create,attrib,move,open,close,access-e modify-t 60-r / home

Open two windows of a machine, create files and check the status to see if there will be any changes.

13 inotify-tools integrates with rsync

The integration of inotify-tools and rsync is mainly realized through the integration of inotifywait commands and rsync commands, and the integration mainly reflects the synchronous side of rsync, while the synchronous side of rsync does not need too much configuration.

Note: the main purpose of this integration is to push the files that need to be backed up by the rsync synchronization side to the rsync synchronization side in real time.

This is illustrated by the following example

14 Application examples

14.1 realize real-time synchronization of one-way local directory

Test the local / test directory as soon as it has (add, delete, change, attribute change), it will be synchronized to / test1 directory

Use the following script to do it

# vim / tmpevents 1. Date events > / var/log/rsync.log 2 > & 1done# nohup sh/ tmp/1.sh & while read eventsdo rsync-a-- delete / test/ / test1/ echo "`events +'% F% T'` event occurs

Use nohup to suspend execution in the background, the terminal shuts down, and the process will not be shut down.

If you want to boot automatically, you can add this command to / etc/rc.local, or write a shell service script to support automatic startup of chkconfig.

Then perform various operations on / test to verify that it can be synchronized to / test1 in real time.

14.2 realize real-time synchronization of two-way local directories

There are also related software (such as unison) that can be specially used to do two-way real-time synchronization.

But we can do it in reverse with inotify and write another script just as easily, which is not introduced here.

14.3 remote real-time synchronization

Method 1: inotify+rsync+nfs

The idea of this method is:

Mount the remote directory locally through the nfs service, and then use it as a local directory, thus avoiding the problem of typing a password when rsync synchronizes the remote.

The test in rhel6.5 is as follows: use innotify to monitor the directory mounted by remote nfs and then synchronize it to the local directory, which will not be successful; (if you operate on the remote machine, you will not synchronize to the local directory. However, when hanging to the local directory operation, it will also be synchronized to the local. Reason: inotify can only monitor directories on this machine, not directories on another machine)

But conversely, using innotify to monitor the local directory and then synchronize to the remote nfs mounted directory can be successful.

Realize real-time synchronization of the / test directory on 10.1.1.9 to the / test1 directory on 192.168.19.248

10.1.1.9 / test directory 192.168.19.248 / test1 directory

On 192.168.19.248

# vim / etc/exports/test1 * (rw,no_root_squash) # / etc/init.d/rpcbind restart# / etc/init.d/nfs restart on 10.1.1.9 # mkdir / test1/# mount-t nfs-o vers=3 192.168.19.248:/test1 / test1-mount with nfs3 version So the / test1 directory is the remote / test1 directory on 192.168.19.248 and executes the same script for synchronizing the two local directories mentioned above. It's all right to test # vim / tmpswap 1. The event $events "> / var/log/rsync.log 2 > & 1done#nohup sh/ tmp/1.sh & occurs when event occurs in while read eventsdo rsync-a-delete / test/ / test1/ echo" `date +'% F% T'` "> / var/log/rsync.log 2 > & 1done#nohup sh/ tmp/1.sh &

Test (process omitted) but note that the script process run in the previous example is found through ps and kill before testing.

-- extension, is it feasible to replace nfs with samba service and mount the remote samba shared directory locally through the mount-t cifs instruction?

Method 2: use ssh equivalence (ssh key login) to make remote machine connections without a password, so it is easy to use remote monitoring or synchronization.

Generate a pair of keys on 10.1.1.9

# ssh-keygen# scp / root/.ssh/id_rsa.pub 192.168.19.248:/root/.ssh/authorized_keys# ssh-add / root/.ssh/id_rsa and then implement remote real-time synchronization, just add remote IP to the remote directory on the basis of the original script (because of the ssh equivalence No password is required) #! / bin/bash/usr/local/bin/inotifywait-mrq-e modify,delete,create,attrib,move / test | while read eventsdo rsync-a-- delete / test/ 192.168.19.248:/test1/ echo "`date +'% F% T'` event events" > / var/log/rsync.log 2 > & 1done

Method 3: instead of doing the ssh key of method 2 above, use the expect script to solve the password problem

#! / bin/bash/usr/local/bin/inotifywait-mrq-e modify,delete,create,move / test | while read eventsdosed-I'/ ^ 192.168.19.248Accord d' / root/.ssh/known_hostsexpect > / var/log/rsync.logdone

Test: first kill all the script processes executed above, and then re-execute the script to verify

Method 4, use rsync's secrets file method and then write a script

Example:

Local synchronous side-> remote synchronized side

10.1.1.9 192.168.19.248

I want to synchronize the contents of the / test/ directory on the local synchronization terminal 10.1.1.9 to the / test1/ directory of the remote synchronized terminal 192.168.19.248.

Prepare:

Close iptables and selinux

Realization process

Operate on the remote synchronized side 192.168.19.248

# mkdir / test1/-- create the synchronized directory for the test # vim / etc/rsyncd.confuid=root gid=root-- here specify to synchronize to 192.168.19.248 using the root identity, otherwise use the nobody user by default (so you need to change the / test1/ directory to the permission of 757) To have permission to write) [testsource] path=/test1/ignore errorsread only=no-Service level gives write permission auth users=test-the authenticated user is named test (this user has nothing to do with the system user) secrets file=/etc/rsyncd.secrets-validates the user name and password of the user test, which needs to be available on both the synchronized side and the synchronized side (so that we can verify it in this way Without the need for remote ssh password authentication) hosts allow=10.1.1.9-further enhanced security, only 10.1.1.9 synchronization # vim / etc/rsyncd.secrets-create this password file, the user name should be the same as that specified in the above auth users, and the password is arbitrary (I tested it with 321 here) test:321# chmod 600 / etc/rsyncd.secrets-- for security Change the permission to 600 Only allow root to read and write (password files of many other software will also require you to change to 600permission) # yum install xinetd-y # vim / etc/xinetd.d/rsync disable = no-- the back value of disable has been changed from yes to no# / etc/init.d/xinetd restart-- restart the service # netstat-ntlup | grep: 873-- verify whether port 873 starts tcp 00: 873. :: * LISTEN 50438/xinetd # chkconfig xinetd on-- set boot to start automatically

Operate on the local synchronization terminal 10.1.1.9

Install the inotify software first. Refer to the above procedure and omit it here.

# mkdir / test/-- create the test synchronization directory # vim / etc/rsyncd.secrets-- also create this password file, which only contains the password. User, in the command in the later script, we specify 32 chmod / etc/rsyncd.secrets-- also changed to 600 permissions # vim / tmp/rsync.sh-- the script and its contents. (to emphasize here, it is convenient for testing, so it is put under / tmp. The production environment can be placed in a directory like / bin to make a command) #! / bin/bash/usr/local/bin/inotifywait-mrq-e modify,delete,create,attrib,move / test | while read eventsdo rsync-a-- delete-- password-file=/etc/rsyncd.secrets / test/ test@192.168.19.248::testsourcedone # chmod 755 / tmp/rsync.sh-- give execution permission It is convenient to execute # nohup sh / tmp/rsync.sh &-- suspend the execution of # vim / etc/rc.local-- add it to rc.local so that it starts # nohup sh / tmp/rsync.sh automatically at the beginning

Test process omitted

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.