Shulou(Shulou.com)06/01 Report--

The structure of this chapter

Introduction to 1.rsync synchronization

two。 Configure the rsync backup source

Basic usage of 3.rsync command

Example of 4.rsync backup operation

Combined use of 5.rsync+inotify

About rsync. A fast incremental backup tool

1.Remote Sync, remote synchronization

two。 Support local replication or synchronize with other SSH or rsync hosts

3. Official website: http://rsync.samba.org

Configure the rsync source server rsync synchronization source:

The remote server of the backup operation, also known as the backup source

Basic ideas for configuring rsync sources:

1. Set up rsyncd.conf configuration files and independent account files

. Enable rsync-- daemon mode

Application example:

1. User backuper, which allows downstream synchronization

two。 The directory of the operation is / var/www/html/

Profile rsyncd.conf

1. It needs to be established manually. The syntax is similar to Samba configuration.

two。 Authentication is configured with auth users and secrets file. If not added, it is anonymous.

Rsync account file

1. Adopt the record format of "user name: password", with one user record per line

two。 Independent account data, not dependent on system account

Enable the rsync service

1. Use-- daemon to provide service alone [execute kill $(catIvar/run/rsyncd.pid) to shut down rsync service]

Usage of the rsync command using the rsync backup tool:

Rsync [option] original location target location

1. Common options:

-a: archiving mode, recursive merge retains object attributes, equivalent to-rlptgoD

-v: displays the verbose information of the synchronization process

-z: compress (compress) when transferring files

-H: keep hard connection files

-A: keep ACL attribute information

-- delete: delete files that exist in the target location but not in the original location

-- checksum: determines whether to skip files based on the checksum of the object

There are two ways to identify a source:

Demo: environment preparation: two hosts

CentOS 7-4 as synchronization source: 192.168.18.148

CentOS 7-5 as a client: 192.168.18.145

Source operation of CentOS 7-4: [root@localhost ~] # hostnamectl set-hostname rsyncd [root@localhost ~] # su [root@rsyncd ~] # rpm-Q rsyncrsync-3.0.9-18.el7.x86_64 [root@rsyncd ~] # vim / etc/rsyncd.confuid = nobodygid = nobodyuse chroot = yespid file = / var/run/rsyncd.piddont compress = * .gz * .tgz * .zip * .z * .Z * .rpm * .deb * .bz2 # Note address = 192.168.18.148 # Native IP address port 873 # start port log file = / var/rsyncd.log # specify log file hosts allow = 192.168.18.0ax 24 # allow network segment access # add the above in the lower line of pid [wwwroot] path = / var/www/htmlcomment = www.kgc.comread only = yesauth users = backupersecrets file = / etc/rsyncd_users.db# on the next line dont Insert the above: share module # after modification, press Esc to exit the insert mode Input: wq save exit # add password file [root@rsyncd ~] # vim / etc/rsyncd_users.dbbackuper:abc123 # need to be consistent with the auth_users name in the sharing module # press Esc to exit the insert mode after adding Enter: wq save and exit [root@rsyncd ~] # chmod 600 / etc/rsyncd_ users.db[ root @ rsyncd ~] # rsync-- daemon [root@rsyncd ~] # netstat-ntap | grep rsynctcp 00 192.168.18.148chmod 873 0.0.0.0 grep rsynctcp * LISTEN 6150/rsync# port 873 open [root@rsyncd ~] # systemctl stop firewalld.service [root@rsyncd ~] # setenforce 0 [root@rsyncd ~] # yum install httpd-y [root@rsyncd html] # cd. [root@rsyncd www] # chmod 777html/ [root@rsyncd www] # ls-l Total dosage 0drwxr-xr-x. 2 root root 6 August 8 19:42 cgi-bindrwxrwxrwx. 2 root root 24 December 16 08:41 htmlCentOS 7-5 client operation: [root@localhost ~] # hostnamectl set-hostname client [root@localhost ~] # su [root@client ~] # systemctl stop firewalld.service [root@client ~] # setenforce 0 [root@client ~] # rpm-Q rsyncrsync-3.0.9-18.el7.x86_64 [root@client ~] # yum install httpd-y [root@client ~] # cd / var/www/html/# at this time Empty [root@client html] # cd. [root@client www] # chmod 777html/ [root@client www] # ls-l Total usage 0drwxr-xr-x. 2 root root 6 August 8 19:42 cgi-bindrwxrwxrwx. 2 root root June 8 19:42 html# synchronization method 1: [root@client www] # rsync-avz backuper@192.168.18.148::wwwroot / var/www/html/Password: # enter the password abc123 at this time Enter receiving incremental file list./index.htmlsent 83 bytes received 172 bytes 12.44 bytes/sectotal size is 17 speedup is 0.07 [root@client www] # cd html/ [root@client html] # ls # at this time the index.html file is synchronized index.html [root@client html] # cat index.htmlthis is test web# synchronization method 2: [root@client www] # rsync-avz rsync://backuper@192.168.18.148::wwwroot / var/www/html / Password: # enter password abc123 at this time Hit enter receiving incremental file list./index.htmlsent 83 bytes received 172 bytes 12.44 bytes/sectotal size is 17 speedup is 0.07 [root@client www] # cd html/ [root@client html] # ls # at this time the index.html file is synchronized index.html [root@client html] # cat index.htmlthis is test web at this time if we use scheduled tasks You need to consider the problem of no interaction: [root@client html] # vim / etc/server.passabc123# write password information and press Esc to exit insert mode Enter: wq save exit [root@client html] # chmod 600 / etc/server.pass [root@client html] # rsync-avz-- delete-- password-file=/etc/server.pass backuper@192.168.18.148::wwwroot / var/www/html/ # you can directly enter receiving incremental file list./index.htmlsent 83 bytes received 172 bytes 510.00 bytes/sectotal size is 17 speedup is 0.07 [root@client html] # lsindex.html with this command [root@client html] # cat index.htmlthis is test web# can then add scheduled tasks to crontab-e. The deficiency of rsync real-time synchronization and regular synchronization

1. The time for performing backup is fixed, with obvious delay and poor real-time performance.

two。 When the synchronization source remains unchanged for a long time, intensive periodic tasks are unnecessary.

Advantages of real-time synchronization

1. Start the backup as soon as the synchronization source changes

two。 As long as there is no change in the synchronization source, no backup is performed.

On the inotify Mechanism of inotifyLinux Kernel

1. Available from version 2.6.13

two。 Can monitor changes in the file system and respond to notifications

3. Auxiliary software: inotify-tools

Rsync+inotify adjusts inotify kernel parameters synchronously in real time:

Max_queue_events: monitoring queue size

Maxuser instances: maximum number of monitoring instances

Maximum number of monitoring files per instance of max_ user_watches::

Install inotify-tools accessibility:

Inotifywait: for continuous monitoring and real-time output of results

Inotifywatch: used for short-term monitoring. The result will be returned after the task is completed.

-m: continuous monitoring

-r: recursively monitor all child objects

-Q: simplify the output information

-e: specify which event types to monitor

Demo: operation in the client initiator: [root@client html] # vim / etc/sysctl.conf# needs to enable monitoring at the initiator # insert the following under the last line fs.inotify.max_queued_events = 16384fs.inotify.max_user_instances = 1024fs.inotify.max_user_watches = 104857 after adding, press Esc to exit the insert mode Input: wq save exit [root@client html] # sysctl-p # refresh data fs.inotify.max_queued_events = 16384fs.inotify.max_user_instances = 1024fs.inotify.max_user_watches = 104857 load inotofy management tool [root@client html] # mkdir / aaa [root@client html] # mount.cifs / / 192.168.10.190/rpm / aaaPassword for root@//192.168.10.190/rpm: [root@client html] # Cd / aaa [root@client aaa] # lsDiscuz_X3.4_SC_UTF8.zip nginx-1.12.2.tar.gzerror.png phpextundelete-0.2.4.tar.bz2 redis-5.0.7.tar.gzhaproxy-1.5.19.tar.gz ruby-2.4.1.tar.gzhttpd-2.4.29.tar.bz2 Ruby.pnghzw.jpeg squidinotify-tools-3.14.tar.gz TC [root@client aaa] # tar zxvf inotify-tools-3.14.tar.gz-C / opt/ [root@client opt] # cd / opt/inotify-tools-3.14/ [root@client inotify-tools-3.14] # lsaclocal.m4 config.h.in COPYING libinotifytools man srcAUTHORS config.sub depcomp ltmain. Sh missingChangeLog configure INSTALL Makefile.am NEWSconfig.guess configure.ac install-sh Makefile.in README [root@client inotify-tools-3.14] # yum install gcc gc-c++ make-y [root@client inotify-tools-3.14] #. / configure [root@client inotify-tools-3.14] # make & & make install [root@client inotify-tools-3.14] # inotifywait-mrq-e modify Create,move,delete / var/www/html/# enters monitoring state Monitoring local html files for updates, creation, movement, and deletion cannot be performed at this time We need to open another remote connection to operate [root@client ~] # cd / var/www/html/ [root@client html] # lsindex.html [root@client html] # touch abc # create a new abc file [root@client html] # rm-rf abc # deleting abc will now show synchronization to this action in the monitoring interface: / var/www/html/ CREATE abc # synchronize to create action Do / var/www/html/ DELETE abc # synchronize to delete action we can use: monitor trigger action Then call rsync for synchronization in monitoring customers first use Ctrl+c to stop monitoring, and then create a script The operation is as follows: [root@client inotify-tools-3.14] # cd / opt/ [root@client opt] # lsinotify-tools-3.14 rh [root@client opt] # vim inotify.shangqian Binder BinderBinOTIFY operations CMD = "inotifywait-mrq-e modify,create,attrib,move Delete / var/www/html/ "RSYNC_CMD=" rsync-azH-- delete-- password-file=/etc/server.pass / var/www/html/ backuper@192.168.18.148::wwwroot/ "$INOTIFY_CMD | while read DIRECTORY EVEVT FILEdo if [$(pgrep rsync | wc-l)-le 0] Press Esc to exit the insert mode after adding then $RSYNC_CMD fidone#, and enter: wq save exit [root@client opt] # chmod + x inotify.sh [root@client opt] # ls-l / var/www/ total dosage 0drwxr-xr-x. 2 root root 6 August 8 19:42 cgi-bindrwxrwxrwx. 2 root root 24 December 16 10:00 html at this time also need to note the CentOS 7-4 rsync server configuration file: [root@rsyncd www] # vim / etc/rsyncd.confread only = no # turn off read-only function # press Esc to exit insert mode after modification Enter: wq save and exit execution script [root@rsyncd ~] # netstat-ntap | grep rsynctcp 0 192.168.18.148netstat 873 0.0.0.0 * LISTEN 2768/rsync [root@rsyncd ~] # kill-9 2768 # kill the process [root@rsyncd ~] # rsync-daemon # start rsync [root @ rsyncd ~] # failed to create pid file / var/run/rsyncd.pid File exists# prompts for the existence of a pid file [root@rsyncd ~] # cd / var/run/ [root@rsyncd run] # lsabrt dhclient-ens33.pid lock radvd syslogd.pidalsactl.pid dmeventd-client log rpcbind systemdatd.pid dmeventd-server lsm rpcbind.sock tmpfiles.dauditd.pid faillock lvm rsyncd. Pid tunedavahi-daemon firewalld lvmetad.pid samba udevcertmonger gdm mdadm sepermit udisks2chrony gssproxy.pid media setrans userchronyd.pid gssproxy.sock mount setroubleshoot utmpconsole httpd named sm-notify.pid vmwarecrond.pid initramfs netreport Spice-vdagentd xtables.lockcron.reboot ksmtune.pid NetworkManager sshd.pidcups libvirt plymouth sudodbus libvirtd.pid ppp sysconfig [root@rsyncd run] # cat rsyncd.pid2768 [root@rsyncd run] # rm-rf rsyncd.pid # Delete this pid file [root@rsyncd run] # rsync-- daemon # launch [root@rsyncd run] # netstat-ntap again | Grep rsync # at this time, a new pid number tcp 00192.168.18.148tcp 873 0.0.0.0root@rsyncd run * LISTEN 5416/rsync [root@rsyncd run] # cat rsyncd.pid5416# is generated at this time. Rsync is running normally at this time. Monitoring is enabled on the client end of CentOS 7-5: [root@client opt] #. / inotify.sh# at this time, monitoring is enabled and then another 7-5 remote connection is opened. Write: [root@client html] # echo "this is test" > test.txt at this time the files are synchronized to the CentOS 7-4 rsync server We can query: [root@rsyncd run] # cd / var/www/html/ [root@rsyncd html] # lsindex.html test.txt [root@rsyncd html] # cat test.txtthis is test or above for real-time synchronization!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.