Shulou(Shulou.com)06/03 Report--

In the actual production environment, we always meet the need to back up some important data, and with the increase of the scale of the application system, the requirements for the security, reliability and timeliness of the data are relatively high.

So I am using rsync+inotify to achieve real-time synchronous backup of data. Write down the steps below in case I forget it in the future.

Experimental background:

Operating system IP machine name role

CentOS 7.2 172.16.22.1 nginx01 data Source (server side)

CentOS 7.2 172.16.22.7 nginx02 backup location (client)

1. Installation of rsync

Install rsync on both the server and the client

[root@nginx01] # yum install-y rsync [root@nginx02] # yum install-y rsync

Start the server-side and client-side rsync daemons respectively

[root@nginx01] # / usr/bin/rsync-- daemon [root@nginx02] # / usr/bin/rsync-- daemon

Second, install inotify

Because inotify is a powerful, fine-grained, asynchronous file system event monitoring mechanism, inotify support has been added since version 2.6.13 of the Linux kernel.

Inotify can monitor all kinds of changes in the file system, and rsync synchronization will be triggered when there are any changes in the file, which just solves the problem of real-time data synchronization.

Inotify only needs to be installed on the server side.

[root@nginx01] # uname-r3.10.0-327.el7.x86_64 [root@nginx01] # ll / proc/sys/fs/inotifytotal 0RWMurray. 1 root root 0 Jun 12 13:43 max_queued_events-rw-r--r--. 1 root root 0 Jun 12 13:43 max_user_instances-rw-r--r--. 1 root root 0 Jun 12 13:43 max_user_watches

If the above three items appear, indicating that the system supports inotify by default, then we can install inotify-tools.

We can download the corresponding version of inotify-tools from http://inotify-tools.sourceforge.net, and I found that the latest one is the inotify-tool-3.1.14 released in 2010.

After the download is complete, install

Decompress first

[root@nginx01] # tar-xf inotify-tools-3.14.tar.gz

Enter the unzipped installation package directory

[root@nginx01] # cd inotify-tools-3.14

Check the compilation and work out the installation path

[root@nginx01 inotify-tools-3.14] #. / configure-prefix=/data0/inotify

Compile and install make

[root@nginx01 inotify-tools-3.14] # make & & make install

After installation, check to see if inotifywait and inotifywatch instructions have been generated

[root@nginx01] # ll / data0/inotify/bin/inotifywa*-rwxr-xr-x. 1 root root 60892 Jun 12 13:45 / data0/inotify/bin/inotifywait-rwxr-xr-x. 1 root root 55183 Jun 12 13:45 / data0/inotify/bin/inotifywatch

Note: inotifywait-used to wait for a file or file set for a specific time, can monitor any file and directory settings, and can recursively monitor the entire directory tree; inotifywatch-- is used to collect monitored file system statistics, including the number of times each inotify event occurred and other related information. You can learn about their usage by using / data0/inotify/bin/inotifywait-help, / data0/inotify/bin/inotifywatch-help

Third, the relevant parameters of inotify

Inotify defines some interface parameters that can be used to limit the size of kernel memory consumed by inotify, so we need to adjust its size according to the needs of the actual application.

[root@nginx01] # ll / proc/sys/fs/inotifytotal 0 RW Murray. 1 root root 0 Jun 12 13:43 max_queued_events-rw-r--r--. 1 root root 0 Jun 12 13:43 max_user_instances-rw-r--r--. 1 root root 0 Jun 12 13:43 max_user_watches

/ proc/sys/fs/inotify/max_queued_events-indicates the maximum number of events that can be queued in the inotify instance when calling inotify_init. Once this value is exceeded, the event is discarded, but the IN_Q_OVERFLOW event is triggered.

/ proc/sys/fs/inotify/max_user_instances-represents the maximum number of inotify instances that can be created per real user ID

/ proc/sys/fs/inotify/max_user_watches-indicates the upper limit of the watches associated with each inotify instance, that is, the maximum number of directories that each inotify instance can monitor. If you need to monitor a large number of directories, you can increase it appropriately.

I just casually increased my above setting value.

[root@nginx01] # echo 32768 > / proc/sys/fs/inotify/max_queued_ events [root @ nginx01] # echo 1024 > / proc/sys/fs/inotify/max_user_ instances [root @ nginx01] # echo 90000000 > / proc/sys/fs/inotify/max_user_watches

4. Configure dual-computer ssh trust

There are two ways to do this using rsync synchronization:

One: use rsync users and passwords

Second: users on the machine can be implemented without entering a password.

I chose the latter for convenience.

Create a RSA key on the data source and backup machine

The following operations are performed on both machines, and the following provides an example of one of them

1. Use root users to log in to the machine

two。 Create a .ssh directory under the root user's home directory and set the correct permissions

[root@nginx01] # mkdir ~ / .ssh [root@nginx01] # chmod 700 ~ / .ssh

3. Use the ssh-keygen command to generate the RSA key for version 2 of the SSH protocol

[root@nginx01] # ssh-keygen-t rsa

The next action is to use the default values when prompted to save the location of the private key (key) and public key (public key). If a private key password (passphrase) is required, enter a private key password (if you use a private key password, you need to enter a private key password when executing remote commands using ssh)

For the sake of convenience, just enter directly!

Add a key to the authorized key file

1. Use root users to log in to the machine

two。 Do the following on the data source machine

[root@nginx01] # cd ~ / .ssh [root@nginx01. / ssh] # ssh 172.16.22.7 cat / root/.ssh/id_rsa.pub > > authorized_ Keys [root @ nginx01. / ssh] # ssh 172.16.22.1 cat / root/.ssh/id_rsa.pub > > authorized_ Keys [root @ nginx01. / ssh] # scp authorized_keys 172.16.22.7:/root/.ssh/ [root@nginx01. / ssh] # chmod 600 / root/.ssh/authorized_keys

3. Execute on the backup machine

[root@nginx02] # chmod 600 / root/.ssh/authorized_keys

4. Test on two machines

[root@nginx01] # ssh 172.16.22.1 dateTue Jun 13 20:39:56 CST 2017 [root@nginx01] # ssh 172.16.22.7 dateTue Jun 13 20:39:57 CST 2017 [root@nginx02] # ssh 172.16.22.1 dateTue Jun 13 20:39:57 CST 2017 [root@nginx02] # ssh 172.16.22.7 dateTue Jun 13 20:39:58 CST 2017

In the first execution, you may have to enter password information, and when you execute again, you will find that you do not need to enter password information to display the system date, which means that ssh mutual trust configuration is successful!

Fifth, realize real-time synchronization

Because inotifywait is a monitoring wait event, we can use the shell script with it.

Common inotifywait parameters:

-m-- monitor means always keep the listening state of the event

-r-- recursive represents a recursive query directory

-Q-- quit means to print out monitoring events

-e-- event can specify the events to be monitored through this parameter, such as modify, delete, create, attrib, etc.

Here is one of my real-time synchronization scripts for reference only:

[root@nginx01] # vim / data0xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Give the synchronization script executable permission

[root@nginx01] # chmod + x / data0/webscripts/inotify.sh

Script operation of real-time backup in the background

[root@nginx01] # sh / data0/webscripts/inotify.sh &

While backing up, you can go to the backup directory on your backup machine to see if it is growing until it is as big as the data source directory.

If your backup directory is relatively large and may not be so easy to identify in terms of numbers, it is recommended to check the number of directory files.

For example, after viewing the data source directory, go to the backup directory to see if it is consistent.

[root@nginx01] # find / data0/nginx/res/home-type f | wc-l30777 [root@nginx02] # find / data0/nginx/res/home-type f | wc-l30777

The configuration is completed here, and the test is successful.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.