Shulou(Shulou.com)06/02 Report--

1.1 description of the network-wide backup project:

Make a summary backup and save the important data on all servers in the whole network architecture.

Automation: automatic backup of data information

Normalization: avoid backup data errors

1.2 implementation process of network-wide backup project

1. Plan and define the backup server

A deploy rsync backup service (in rsync daemon mode)

B write script files to manage backup data information

C write scheduled task execution script to manage data (difficulty)

two。 Verify that the architecture server is validated

A verify rsync backup service

B write script files to back up data information uniformly (difficulty)

C write scheduled task execution scripts to back up data

1.3 full-network backup project rsync daemon mode deployment 1.3.1 rysnc server deployment operation

Verify that the software exists

[root@backup~] # rpm-qa rsync

Rsync-3.0.6-12.el6.x86_64

Check the configuration file

# # rsyncd.conf start## configuration file description information

Uid = rsync # user's remote command uses rsync to access the shared directory

Gid = rsync # user group

Use chroot = no # Security related

Max connections = 200 # maximum number of connections

Timeout = 300 # timeout

Pid file = / var/run/rsyncd.pid # holds the ID number (PID) of the service runtime process

Lock file = / var/run/rsync.lock # lock file for the process

Log file = / var/log/rsyncd.log # run the program log file, display error messages, etc.

[backup] # Module name

Path = / backup # the location (path) of the module

Ignore errors # ignore faulty programs

Read only = whether false # is read-only

List = false # can be listed

Hosts allow = 172.16.1.0 hosts allow 24 # range of customers allowed to access the rsync server (whitelist)

Hosts deny = 0.0.0.0 hosts deny 32 # range of customers whose access to the rsync server is prohibited (blacklist)

Auth users = rsync_backup # users who do not exist, only for opening doors (authentication)

Secrets file = / etc/rsync.passwor # key file for non-existent users for authentication

Comment = "backup dir by oldboy" #

Create a virtual user management rsync program

[root@backup~] # # create a user

[root@backup~] # useradd-s / sbin/nologin-M rsync

View after the creation is completed

[root@backup~] # id rsync

Uid=500 (rsync) gid=500 (rsync) groups=500 (rsync)

4. Check whether the rsync service authenticated user password file exists

[root@backup~] # cat / etc/rsync.password

Rsync_backup:123456

Remember to check whether the password file permission is 600.

[root@backup~] # chmod 600 / etc/rsync.password

[root@backup~] # ll / etc/rsync.password

-rw- 1root root 20 May 4 12:04/etc/rsync.password

5. Create a shared storage directory

[root@backup~] # # create a directory

[root@backup~] # mkdir / backup-p

[root@backup~] # ll-d / backup/

Drwxr-xr-x 2root root 4096 May 4 12:00 / backup/

[root@backup~] # # modify backup directory permissions

[root@backup~] # chown-R rsync.rsync / backup/

[root@backup~] # ll-d / backup/

Drwxr-xr-x 2rsync rsync 4096 May 4 12:00 / backup/

6. Start the rsync server

[root@backup~] # ps-ef | grep rsync

Root 5138 1 0 01:31? 00:00:00 rsync-daemon

Root 6503 6391 0 09:04 pts/1 00:00:00 grep-color=auto rsync

[root@backup~] # netstat-luntp | grep 873

Tcp 0 0 0.0.0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0

Tcp 0 0: 873: * LISTEN 5138/rsync

If not, open it.

Rsync-daemon

7. Put the rsync startup command into the boot file

[root@backup~] # echo "rsync-- daemon" > > / etc/rc.local

[root@backup~] # tail-1 / etc/rc.local

Rsync--daemon

1.3.2 rsync client verification operation

The first milestone to be done

1. Create user authentication password file to realize data backup and transmission without key

[root@nfs01~] # cat / etc/rsync.password

123456

View directory permissions changed to 600

[root@nfs01~] # ll / etc/rsync.password

-rw-1 root root 7 May 5 10:02/etc/rsync.password

two。 Test rsync service data backup transfer

1.4 Network-wide backup project scripting 1.4.1 rsync client scripting requirements

1. Create a local backup backup directory mkdir-p / backup

two。 Compress the backup data uniformly and save the soft link file tar-zchf

3. Push backup directory data to rsync backup server-- password-file=

4. Delete locally saved data information from 7 days ago

Find / backup/$Host_IP-type f-mtime+7-name "* .tar.gz" | xargs rm-f

5. Verify the backup data and add fingerprint information md5sum to generate data file fingerprint information

Vim/server/scripts/backup_nginx.sh

#! / bin/bash

# oldboy at2017 5 backup data

Host_IP=$ (hostname-I | awk'{print $2}')

Date_info=$ (date+%F_week0%w)

# createbackup data dir

Mkdir-p/backup

Mkdir/var/html/www-p

Mkdir/app/logs/-p

# tar datainfo

Cd / & &\

Mkdir / backup/$Host_IP-p

Tar zchf/backup/$Host_IP/sysbak_date_$ {Date_info} .tar.gz. / var/spool/cron/root. / etc/rc.local./server/scripts. / etc/sysconfig/iptables

Tar zchf/backup/$Host_IP/www_date_$ {Date_info} .tar.gz. / var/html/www

Tar zchf/backup/$Host_IP/logs_date_$ {Date_info} .tar.gz. / app/logs

# rsync pushfiger info

Find/backup/$Host_IP/-type f-name "* ${Date_info} .tar.gz" | xargs md5sum > / backup/$Host_IP/finger_$ {Date_info} .txt

# rsync pushdata to rsync_server

Rsync-az/backup/ rsync_backup@172.16.1.41::backup-- password-file=/etc/rsync.password

# cleardata info for 7 day ago

Find/backup/$Host_IP-type f-mtime + 7-name "* .tar.gz" | xargs rm-f

Find/backup/$Host_IP-type f-mtime + 7-name "finger*" | xargs rm-f

1.4.2 rsync server scripting requirements

a. Verify data integrity fingerprint files transferred by md5sum-c client

b. The integrity verification results, the configuration of the mailbox sent to the supervisor by email and the way of sending email (two ways to send email)

01: send mail, using file to express the content of mail.

Mail-s "check data" 18735676310@163.com / tmp/figer_cheak.txt

# send cheakinfo mail

Mail-s "cheak date" 18735676310@163.com / dev/null 2 > & 1

Scheduled tasks for the nfs client

# crond-id-002:push backup torsync

00 00 * / bin/bash/server/scripts/backup_nfs.sh > / dev/null 2 > & 1

② .rsync server writes scheduled tasks

Scheduled tasks on the rsync server

# crotab-id002:backup NFS nginx

00 06 * / bin/bash/server/scripts/backup_rsync.sh > / dev/null 2 > & 1

The scheduled tasks of the above two clients are packaged and sent at exactly 12:00 every night, while the scheduled task of the rsync server is sent at 6 o'clock every morning.

1.6 Analysis of the difficulties of the whole network backup project

①. Save Monday's data to make sure what Monday's data is.

/ backup/$ (hostname-I | awk'{print $2}') / sysbak_date_$ (date+%F_week0%w) .tar.gz./var/spool/cron/root

So take this into account when packing compressed files, $(date+%F_week0%w), which adds the day of the week to the file name, so it can play a key role in saving data.

Find / backup/-type f-mtime + 180!-name "* week01.tar.gz" | xargs rm-f

②. What is the concept of verifying data integrity

Because the data is transmitted from the client, in order to ensure that the data is not modified or lost, the data is encrypted when the data is sent, and then when the server receives the data and the encrypted file, it will be compared. If the string code is the same, it means that it has not been lost or modified.

If you do not leave me. I will by your side untillthe life end

Level 4: if you don't leave me, I will die with you.

Cet-6: if you don't leave and don't give up, I will depend on life and death.

Level 8: what is love in the world? Directly teach people the promise of life and death.

Level 10: heaven and earth are in harmony, but dare to never be with the king.

Buddha level: if you are there or not, love is there, neither increasing nor decreasing.

1.7 deployment of network-wide backup project 1.7.1 first milestone: implementation of data transfer backup

Rsync daemon mode configuration steps

Check the server

1. Check whether the software exists

[root@backup~] # rpm-qa rsync

Rsync-3.0.6-12.el6.x86_64

two。 Check the configuration file

[root@backup~] # cat / etc/rsyncd.conf

# # rsyncd.conftart##

Uid = rsync

Gid = rsync

Usechroot = no

Maxconnections = 200

Timeout= 300

Pidfile = / var/run/rsyncd.pid

Lockfile = / var/run/rsync.lock

Logfile = / var/log/rsyncd.log

[backup]

Path= / backup

Ignoreerrors

Readonly = false

List= false

Hostsallow = 172.16.1.0 Compact 24

Hostsdeny = 0.0.0.0amp 32

Authusers = rsync_backup,oldboy

Secretsfile = / etc/rsync.password

[data]

Path= / data

Ignoreerrors

Readonly = false

List= false

Hostsallow = 172.16.1.0 Compact 24

Hostsdeny = 0.0.0.0amp 32

Authusers = rsync_backup,oldboy

Secretsfile = / etc/rsync.password

3. Create a virtual user management rsync program

[root@backup~] # # create a user

[root@backup~] # useradd-s / sbin/nologin-M rsync

4. Check whether the rsync service authenticated user password file exists

[root@backup~] # cat / etc/rsync.password

Rsync_backup:123456

Remember to check whether the password file permission is 600.

5. Create a shared storage directory

Mkdir / backup/-p

Modify the group and owner of the catalogue

[root@backup~] # ll-d / backup/

Drwxr-xr-x5 rsync rsync 4096 May 5 15:35 / backup/

6. Start the rsync server

[root@backup~] # ps-ef | grep rsync

Root 5138 1 0 01:31? 00:00:00 rsync-daemon

Root 6503 6391 0 09:04 pts/1 00:00:00 grep-color=auto rsync

[root@backup~] # netstat-luntp | grep 873

Tcp 0 0 0.0.0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0

Tcp 0 0: 873: * LISTEN 5138/rsync

If not, open it.

Rsync-daemon

7. Put the rsync startup command into the boot file

[root@backup~] # echo "rsync-- daemon" > > / etc/rc.local

[root@backup~] # tail-1 / etc/rc.local

Rsync--daemon

Client:

1. Create user authentication password file to realize data backup and transmission without key

[root@nfs01~] # cat / etc/rsync.password

123456

View directory permissions changed to 600

[root@nfs01~] # ll / etc/rsync.password

-rw-1 root root 7 May 5 10:02 / etc/rsync.password

two。 Test rsync service data backup transfer

Second milestone: writing data backup scripts

Nginx nfs

-- backup script written by rsync client

Script written by client-side nginx

Vim/server/scripts/backup_nginx.sh wants to standardize the name of the script and the directory where the script is stored

#! / bin/bash command interpreter

# oldboy at2017 5 backup data necessary comments

Alias for Host_IP=$ (hostname-I | awk'{print $2}') setting

Alias for Date_info=$ (date+%F_week0%w) settings

# the directory to be created by createbackup data dir

Mkdir-p/backup

Mkdir/var/html/www-p

Mkdir/app/logs/-p

# Notes for the purpose of tar datainfo packaging

Cd / & &\ must be packaged under the root directory.

Mkdir/backup/$Host_IP-p

Tar zchf/backup/$Host_IP/sysbak_date_$ {Date_info} .tar.gz. / var/spool/cron/root. / etc/rc.local./server/scripts. / etc/sysconfig/iptables remember to add the h parameter, you can package the soft link

Tar zchf / backup/$Host_IP/www_date_$ {Date_info} .tar.gz. / var/html/www

Tar zchf/backup/$Host_IP/logs_date_$ {Date_info} .tar.gz. / app/logs

# rsync pushfiger info

Find/backup/$Host_IP/-type f-name "* ${Date_info} .tar.gz" | xargs md5sum > / backup/$Host_IP/finger_$ {Date_info} .txt encrypts the files found

# rsync pushdata to rsync_server

Rsync-az/backup/ rsync_backup@172.16.1.41::backup-- password-file=/etc/rsync.password

Key-free packaging for data transmission

# cleardata info for 7 day ago delete files from seven days ago

Find/backup/$Host_IP-type f-mtime + 7-name "* .tar.gz" | xargs rm-f

Find/backup/$Host_IP-type f-mtime + 7-name "finger*" | xargs rm-f

-- backup script written by nfs client

Vim/server/scripts/backup_nfs.sh wants to standardize the name of the script and the directory where the script is stored

#! / bin/bash command interpreter

# oldboy at2017 5 backup data necessary comments

Host_IP=$ (hostname-I | awk'{print $2}')

Date_info=$ (date+%F_week0%w)

# createbackup data dir

Mkdir-p/backup

# tar datainfo

Cd / & &\ must be packaged under the root directory.

Mkdir/backup/$Host_IP-p

Tar zchf

/ backup/$Host_IP/sysbak_date_$ {Date_info} .tar.gz. / var/spool/cron/root. / etc/rc.local./server/scripts. / etc/sysconfig/iptables remember to add the h parameter, you can package the soft link

# rsync pushfiger info

Find/backup/$Host_IP/-type f-name "* ${Date_info} .tar.gz" | xargs md5sum > / backup/$Host_IP/finger_$ {Date_info} .txt

Encrypt the files found

# rsync pushdata to rsync_server

Rsync-az/backup/ rsync_backup@172.16.1.41::backup-- password-file=/etc/rsync.password

Key-free packaging for data transmission

# cleardata info for 7 day ago

Find/backup/$Host_IP-type f-mtime + 7-name "* .tar.gz" | xargs rm-f

Find/backup/$Host_IP-type f-mtime + 7-name "finger*" | xargs rm-f

Backup server side-rsync server side writes backup script

-- rsync server writes backup script

Vim/server/scripts/backup_rsync.sh wants to standardize the name of the script and the directory where the script is stored

#! / bin/bash command interpreter

Date_info=$ (date+%F_week0%w)

# check datainfo looks up the passed password and compares it, and appends the result to the figer_cheak.txt file

Find / backup/-type f-name "finger_$ {Date_info} .txt" | xargs md5sum-c > / tmp/figer_cheak.txt

# send cheakinfo mail sends the contents of the file to the mailbox

Mail-s "cheak date" 18735676310@163.com / dev/null 2 > & 1

Scheduled tasks for the nfs client

# crond-id-002:push backup torsync

00 00 * / bin/bash/server/scripts/backup_nfs.sh > / dev/null 2 > & 1

two。 Rsync server writes scheduled tasks

Scheduled tasks on the rsync server

# crotab-id002:backup NFS nginx

00 06 * / bin/bash/server/scripts/backup_rsync.sh > / dev/null 2 > & 1

The scheduled tasks of the above two clients are packaged and sent at exactly 12:00 every night, while the scheduled task of the rsync server is sent at 6 o'clock every morning. Because if the server also makes a backup at 12:00 in the evening, I'm afraid that if the client hasn't sent it, there will be no files, so it's best to postpone it a little bit.

1.8 knowledge points used for network backup 1.8.1 md5sum

Encrypt the transferred files

[root@nfs01~] # md5sum / etc/hosts

C559a5da81a3ecc708ae5d84985a0776 / etc/hosts

Fingerprint encryption: avalanche effect

1.8.2 send email modified configuration file vim / etc/mail.rc

Go to the configuration file for sending mail and edit the parameter information for sending mail.

Vim/etc/mail.rc

Setfrom=18735676310@163.com smtp=smtp.163.com

Setsmtp-auth-user=18735676310@163.com smtp-auth-password=Zr950220 smtp-auth=login

Setfrom= email address @ 163.com smtp=smtp.163.com

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.