Shulou(Shulou.com)06/01 Report--

This article mainly introduces "how to use notify+rsync to achieve batch update of linux files". In daily operation, I believe many people have doubts about how to use notify+rsync to achieve batch update of linux files. Xiaobian consulted all kinds of materials and sorted out simple and easy-to-use methods of operation. I hope it will be helpful for you to answer the doubts of "how to use notify+rsync to achieve batch updates of linux files". Next, please follow the editor to study!

Rsync remote synchronization tool, incremental backup synchronization of remote server data can be achieved through rsync, but rsync itself has bottlenecks. When synchronizing data, rsync uses the core algorithm to compare the target files of the remote server, and only carries out differential synchronization. If the number of server files reaches the order of millions or even tens of millions, then file comparison will be very time-consuming. And the changes are often only a small part of them, this is a very inefficient way, the emergence of Inotify, can alleviate the shortcomings of rsync, learn from each other.

System environment

CentOS_5.7-x86_64 update source server: 192.168.9.227 destination server: 192.168.9.226 192.168.9.228...

Destination server configuration

192.168.9.226 192.168.9.228 (rsync server):

Check whether rsync installs rpm-qa | grep rsync

If it is not shipped, install it with the following command

Yum-y install rsync definition rsync profile / etc/rsyncd.conf

192.168.9.226:

Cat > > / etc/rsyncd.conf log file = / var/log/ rsyncd.log [Web1] path = / data/www1/ignore errorsread only = nolist = nohosts allow = 192.168.9.0/255.255.255.0auth users = www1secrets file = / etc/www1.pwdEOF

192.168.9.228:

Cat > > / etc/rsyncd.conf log file = / var/log/ rsyncd.log [web2] path = / data/www2/ignore errorsread only = nolist = nohosts allow = 192.168.9.0/255.255.255.0auth users = www2secrets file = / etc/www2.pwdEOFrsyncd.conf configuration file details uid = nobody / / users running RSYNC daemons gid = nobody / / groups running RSYNC daemons use chroot = 0 / / do not use chrootmax connections = 0 / / maximum number of connections 0 is not restricted port = 873 / / default port 873 below these files are automatically generated after installing the RSYNC service pid file = / var/run/rsyncd.pid / / pid file storage location lock file = / var/run/rsync.lock / / lock file location. Specify the lock file that supports the max connections parameter. The default value is / var/run/rsyncd.lock.log file = / var/log/rsyncd.log / / the location of the log file Timeout = 300. This option overrides the IP timeout specified by the customer. This option ensures that the rsync server does not wait forever for a crashed client. The timeout unit is seconds, and 0 indicates that there is no timeout definition, which is also the default value. For anonymous rsync servers, an ideal number is 600.Log format =% t% a% m% f% b. Through this option, users can customize the fields of the log file themselves when using transfer logging. Its format is a string that contains format definers. The format definitions that can be used are as follows:% h remote hostname% a remote IP address% l file length characters% p process id%o operation type for this rsync session: "send" or "recv"% f file name% P module name% m module name% t current time% u authenticated user name (null when anonymous)% b actual number of bytes transferred c when the file is sent The default log format for the file recorded in this field is:% o% h [% a]% m (% u)% f% l ". Generally speaking,"% t [% p] "is added to the header of each line. A perl script called rsyncstats is also published in the source code to count log files in this format. # transfer logging = yes causes rsync servers to use files in ftp format to record download and upload operations in their own separate logs. Syslog facility = local3 specifies the message level when rsync sends log messages to syslog. Common message levels are: uth, authpriv, cron, daemon, ftp, kern, lpr, mail, news, security, sys-log, user, uucp, local0 Local1, local2, local3,local4, local5, local6 and local7. The default value is daemon. Module parameter [web1] / / here is the authenticated module name. On the client side, you need to specify the directory where path = / data/www1/ needs to be mirrored, which is indispensable! Comment = backup web / / comment information for this module ignore errors / can ignore some extraneous IO errors read only = yes / / this option sets whether customers are allowed to upload files. If it is true, then any upload request will fail, if it is false and the server directory read and write permissions are allowed, then upload is allowed. The default value is true.list = no / / does not allow the column file auth users = bak / / authenticated user name, if not this line indicates anonymous, this user has nothing to do with the system this option specifies a list of user names separated by spaces or commas, and only these users are allowed to connect to the module. The users here have nothing to do with the system users. If "auth users" is set, the client's connection request for the module will later be authenticated by the rsync request challenged to authenticate the challenge/response authentication protocol used here. The user's name and password are stored in clear text in the file specified by the secrets file option. By default, modules can be connected without a password (that is, anonymously). Secrets file = / etc/www1.pwd / / password and user name comparison table, the password file generates its own option to specify a file that defines the user name: password pair. This file is useful only if "auth users" is defined. Each line of the file contains a username:passwd pair. Generally speaking, the password should not exceed 8 characters. There is no default secures file name, you need to specify one in a limited format (for example: / etc/www1.pwd). Note: the permissions of this file must be 600, otherwise the client will not be able to connect to the server. Hosts allow = 192.168.9.0 / 255.255.255.0 / allow hosts or network segments this option specifies which IP customers are allowed to connect to the module. The definition of customer mode can be in the following form: a single IP address, for example: 192.168.9.227 entire network segment, for example: 192.168.9.0 IP 24, or 192.168.9.0 IP 255.255.255.0 or network segments need to be separated by spaces, "*" means all, and all hosts are allowed to connect by default. Hosts deny = 0.0.0.0and0 / disable hosts

Establish certification document / etc/www1.pwd

This file must be consistent with the file name specified in the configuration file. The format here is: username:password, security issues, and it is not recommended to use root user 192.168.9.226 in actual use:

Echo "www1:741852" > > / etc/www1.pwd

192.168.9.228:

Echo "www2:951753" > > / etc/www2.pwd

And we need to set the permissions for this file to 600

Chmod 600 / etc/www1.pwdchmod 600 / etc/www2.pwdchmod 600 / etc/rsyncd.conf

Create a motd file (optional)

Rsyncd.motd records welcome messages for rsync services, in which you can enter any text information, such as:

Echo "Welcome to use the rsync services!" > > / var/rsyncd.motd

Start rsync

/ usr/bin/rsync-- daemonecho "/ usr/bin/rsync-- daemon" > / etc/rc.local

Update source server configuration

192.168.9.227 (rsync client)

File system events that inotify can monitor include

IN_ACCESS, that is, files are accessed IN_MODIFY, files are writeIN_ATTRIB, file attributes are modified, such as chmod, chown, touch, IN_CLOSE_WRITE, writable files are closeIN_CLOSE_NOWRITE, unwritable files are closeIN_OPEN, files are openIN_MOVED_FROM, files are removed, such as mvIN_MOVED_TO, files are moved, such as mv, cpIN_CREATE, create new files IN_DELETE, files are deleted, such as rmIN_DELETE_SELF, self-delete That is, an executable file deletes its own IN_MOVE_SELF during execution and moves itself, that is, an executable file moves its own IN_UNMOUNT during execution, the host file system is umountIN_CLOSE, the file is closed, which is equivalent to (IN_CLOSE_WRITE | IN_CLOSE_NOWRITE) IN_MOVE, and the file is moved, which is equivalent to (IN_MOVED_FROM | IN_MOVED_TO).

Note: the files mentioned above also include directories.

Install inotify-tools

Before installing inotify-tools, please make sure that your linux kernel reaches 2.6.13 and the CONFIG_INOTIFY option is turned on during compilation. You can also detect it by using the following command

Ls / proc/sys/fs/inotify

If there are three items of max_queued_events,max_user_instances,max_user_watches, it means support.

Wget http://cloud.github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gztar xvf inotify-tools-3.14.tar.gzcd inotify-tools-3.14./configuremake Make install writes the rsync monitoring script vi / rootspur rsync.shallows bind binapacheBashhost1, 192.168.9.226host2, 192.168.9.228srcNow modify,delete,create des1, web1des2, web2user1, www.w1user2, www.w2Uniplex localbinplet, inotifywait-mrq-- timefmt'% d/%m/%y% Hcharge% M'-- format'% T% w% f'\-e modify,delete,create Attrib\ ${src}\ | while read filedorsync-vzrtopg-- delete-- progress ${src} ${user1} @ ${host1}:: ${des1}-- password-file=/etc/www1.pwd & & rsync-vzrtopg-- delete-progress ${src} ${user2} @ ${host2}:: ${des2}-- password-file=/etc/www2.pwd & & echo "${files} was rsynced" > / tmp/rsync.log 2 > & 1echo "- -"done-m" That is,-- monitor, which means to keep event listening state at all times. -r, that is,-- recursive, represents the recursive query directory. -Q, that is,-- quiet, means to print out monitoring events. -e, namely-- event. This parameter allows you to specify the events to be monitored. Common events include modify, delete, create, attrib, etc.-- timefmt: output format of specified time-- format: specify the details of the change file.

Establish authentication file (only add password for rsync client authentication file)

Echo "741852" > > / etc/www1.pwdecho "951753" > > / etc/www2.pwdchmod / etc/www1.pwdchmod 600 / etc/www2.pwd/bin/sh-n / root/rsync.sh / / Syntax checking chmod + x / root/rsync.shnohup sh / root/rsync.sh & echo "nohup sh / root/rsync.sh &" > > / etc/rc.local synchronization Test

Create a new file on the update source server and run the following command to see if the files can be synchronized properly and to see if there is an error message

Rsync-vzrtopg-- delete-- progress / data/www1/ www1@192.168.9.226::web1-- password-file=/etc/www1.pwd

Submit the updated files to the update source server, so that the updated files will be synchronized to all destination servers in batches through inotify+rsync, which is quite convenient and quick.

At this point, the study on "how to use notify+rsync to achieve batch updates of linux files" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.