Shulou(Shulou.com)06/01 Report--

First, understand rsync

1. Rsync is a unix-like incremental backup tool that can achieve fast image synchronization.

2. The characteristics of rsync:

1) synchronize the entire directory (recursive synchronization) tree

2) can maintain the attributes of the file (permissions, time, soft and hard links)

3) Fast: the first synchronization is completely replicated, and the second synchronization only synchronizes the modified part, and the transmission is compressed.

4) Security: you can call ssh protocol to encrypt the transmission.

5) support anonymous site synchronization to achieve public resource sharing.

3. How rsync works:

1) the architecture of rsync: Cramp S; the role is divided into synchronization source (backup source) and initiator.

Initiator: initiates the rsync request.

Synchronization side: the corresponding rsync request.

2) the direction of synchronization:

Uplink synchronization: the data is at the initiator, the local login user must have read access to the data, and the remote user must have write permission to the directory

Downlink synchronization: the data is in the synchronization source, the local login user must have write access to the directory, and the remote user must have read permission to the data.

II. Basic use of rsync

1. Basic grammar:

Local synchronization: rsync [options] data destination directory

Remote uplink synchronization: rsync [option] data user @ IP:/ destination directory

Remote downlink synchronization: rsync [option] user @ IP:/ data / destination directory

2. Common options:

-avzH # # an archives and preserves attributes, v displays details, z compresses, H retains hard links

-- delete # # Delete files that exist in the destination directory but not in the source data

3. Points for attention

1) add "/" to the source data to synchronize the contents of the directory; do not add "/" to synchronize the directory itself and its contents.

2)-the delete option is often used to delete large files, which is very fast.

4. Remote synchronization: SSH is recommended, which is safe and convenient.

1) create a user (done on both the initiator and the synchronization source)

2) Setfacl sets permissions to ensure the permissions of rsync and remote programs

3) configure the key team of ssh to log in to implement rsync without password

4) write scripts and schedule tasks to complete timing synchronization

III. Automatic synchronization of rsync+inotify

1. Understand the mechanism of file system change monitoring and notification provided by the inotify:linux kernel, and manage it through inotify-tool tools.

2. Adjust kernel parameters:

Max_queue_events: monitoring queue size

Max_user_instances: maximum number of monitoring instances

Max_user_watches: maximum number of monitoring files per instance

Fs.inotify.max_queued_events = 16384

Fs.inotify.max_user_instances = 1024

Fs.inotify.max_user_watches = 1048576

3. The syntax of inotify-wait command:

Inotifwait-mrq-e modify,create,move,delete,attrib directory options:-m continuous monitoring, r recursion, Q simplified output,-e specify monitoring events

4. Write a script and set up the background boot to run

[lu01@localhost bin] $cat rsync_ inotify.sh

#! / bin/bash

# # by www.linuxfan.cn 2016-11-25

DIR=/var/www/html/

RSYNC= "rsync-azH-- delete $DIR ru01@192.168.100.151:$DIR"

INOTIFY= "inotifywait-mrq-e modify,move,create,attrib,delete $DIR"

$INOTIFY | while read DIRECTORY EVENT FILE;do

$RSYNC

Done

/ home/lu01/bin/rsync_inotify.sh & # # running in the background

Note: rsync+inotify is only used synchronously in uplink.

Case: download

Backup source operation:

Mkdir / opt/data

Touch / opt/data/ {1..9} .txt

Useradd r_get

Echo 123123 | passwd-- stdin r_get

Rpm-qa rsync # # confirm that the rsync software is installed

Netstat-uptln | grep 22

Confirm that the r_get user has read access to / opt/data

Initiator operation:

Rsync-avzH r_get@192.168.100.151:/opt/data/ / tmp # # download

Rsync-avzH r_get@192.168.100.151:/opt/data / tmp # # comparison

Rsync-avzH r_get@192.168.100.151:/opt/data/ / tmp-- delete # # ensures data consistency, and downlink synchronization is based on synchronization sources

Case: upload

Backup source operation:

Mkdir / opt/data_bak

Useradd r_put

Echo 123123 | passwd-- stdin r_put

Setfacl-m u:r_put:rwx / opt/data_bak # # ensure that the user has write access to the target directory

Getfacl / opt/data_bak

Make sure rsync and sshd are normal

Initiator operation:

Mkdir / data

Touch / data/ {1..9} .avi

Rsync-avzH / data r_put@192.168.100.151:/opt/data_bak

Rsync-avzH / data/ r_put@192.168.100.151:/opt/data_bak

Rsync-avzH / data/ r_put@192.168.100.151:/opt/data_bak-- delete

Verify the results on the backup source.

Case: rsync synchronization after authentication of ssh key pair

Ssh-keygen-t rsa

Ssh-copy-id r_get@192.168.100.151 # # upload the public key to the download user

Ssh-copy-id r_put@192.168.100.151

Ssh r_get@192.168.100.151 # # Login verification, same as r_put

Perform rssync synchronization verification:

Vi / root/bin/rsync.sh

#! / bin/bash

Rsync-avzH / data/ r_put@192.168.100.151:/opt/data_bak-- delete

Rsync-avzH r_get@192.168.100.151:/opt/data/ / tmp-- delete

Grep rsync / var/spool/cron/root

If [$?-ne 0]; then

Echo "300 * * 6 / root/bin/rsync.sh & > / var/log/rsync.log" > / var/spool/cron/root

: wq

Chmod + x / root/bin/rsync.sh

2.inotify+rsync real-time synchronization: uplink synchronization

1) install inotify-tools

Tar zxvf inotify-tools-*.tar.gz-C / usr/src/

Cd / usr/src/inotify-tools-*/

. / configure & & make & & amake install

Ls / usr/local/bin/inotify*

2) use of inotify:

Vi / etc/sysctl.conf

Fs.inotify.max_queued_envents = 16384 # # number of monitoring event queues

Fs.inotify.max_user_instances = 1024 # # number of monitoring instances

Fs.inotify.max_user_watches = 1048576 # # number of files monitored

: wq

Sysctl-p

Inotifywait-mrq-e modify,create,attrib,move,delete / root/data # #-m continuous monitoring, r recursive directory, Q simplified output,-e specified monitoring events: modify modification, create creation, attrib permission modification, move movement, delete deletion; operate files in other terminals to view changes

Vi rsync_inotify.sh # # Real-time synchronization script

#! / bin/bash

RSYNC= "rsync-avzH / root/data/ r_put@192.168.100.151:/opt/data_bak/-- delete"

INT_CMD= "inotifywait-mrq-e modify,create,move,delete,attrib / root/data/"

$INT_CMD | while read DIRECOTRY EVENT FILE;do

$RSYNC

Done

: wq

Chmod + x rsync_inotify.sh

Rsync_inotify.sh & # # Startup script

Test verification.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.