Shulou(Shulou.com)05/31 Report--

This article introduces the relevant knowledge of "how to install and configure Nginx+Keepalived master / standby high availability". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Environment description:

Operating system: CentOS6.7 x86x64

Nginx version: nginx-1.9.7

Keepalived version: keepalived-1.2.24

Main nginx + Keepalived: 10.219.24.26

Standby nginx + Keepalived: 10.219.24.23

Virtual IP:10.219.24.100

Back-end tomcat_1:10.219.24.21:8080

Back-end tomcat_2:10.219.24.21:8081

Architecture and principle: front-end dual Nginx + Keepalived, Nginx reverse proxy to back-end tomcat cluster to achieve load balancing, Keepalived to achieve high availability of cluster

After the failure of the primary nginx, the virtual IP automatically drifts to the standby nginx.

1. Nginx + Keepalived installation

1. 1 nginx-1.9.7 compilation and installation

Download address: http://nginx.org/download/nginx-1.9.7.tar.gz

Add: before installing nginx, you need to install some dependent packages: gcc, pcre, zlib

A, nginx gzip module requires zlib library

B. Pcre library is required for nginx rewrite module

C. Nginx ssl module requires openssl library

1. Install the necessary dependency packages

[root@mysql03 ~] # yum install-y pcre pcre-devel

Centos 6.7Configuring yum Local Source Link reference: http://blog.csdn.NET/zhang123456456/article/details/56690945

2. Nginx installation

[root@mysql03 ~] # ll nginx-1.9.7.tar.gz

-rw-r--r--. 1 root root 885562 Jun 14 21:46 nginx-1.9.7.tar.gz

[root@mysql03 ~] # tar zxvf nginx-1.9.7.tar.gz

[root@mysql03 ~] # cd nginx-1.9.7

-- configure nginx installation options

[root@mysql03 nginx-1.9.7] #. / configure-prefix=/usr/local/nginx

Description: after the configuration, you can see a configuration summary. All 5 items in the summary must have corresponding library support.

Configuration summary

+ using system PCRE library

+ OpenSSL library is not used

# # if you want to install openssl module, you need to specify. / configure-- prefix=/usr/local/nginx-- with-openssl=/root/openssl-1.0.2d # #

+ md5: using system crypto library

+ sha1: using system crypto library

+ using system zlib library

-- install nginx

[root@mysql03 nginx-1.9.7] # make & & make install

3. Check whether the installation is normal.

[root@mysql03 nginx-1.9.7] # cd / usr/local/nginx

[root@mysql03 nginx] # ll

Total 16

Drwxr-xr-x. 2 root root 4096 Jun 14 22:14 conf

Drwxr-xr-x. 2 root root 4096 Jun 14 22:14 html

Drwxr-xr-x. 2 root root 4096 Jun 14 22:14 logs

Drwxr-xr-x. 2 root root 4096 Jun 14 22:14 sbin

-- start

[root@mysql03 nginx] #. / sbin/nginx # if it does not start normally, it may be due to port occupation

[root@mysql03 nginx] # ps-ef | grep nginx

Root 5212 1 0 22:17? 00:00:00 nginx: master process. / sbin/nginx

Nobody 5213 5212 0 22:17? 00:00:00 nginx: worker process

Root 5228 2359 0 22:20 pts/0 00:00:00 grep nginx

-visit

Browser input: http://10.219.24.26/ # ip for your own ip

See the contents of the following page, everything is normal.

Welcome to nginx!

If you see this page, the nginx web server is successfully installed and working. Further configuration is required.

For online documentation and support please refer to nginx.org.

Commercial support is available at nginx.com.

Thank you for using nginx.

-- close

[root@mysql03 nginx] #. / sbin/nginx-s stop

[root@mysql03 nginx] # ps-ef | grep nginx

Root 5241 2359 0 22:25 pts/0 00:00:00 grep nginx

One, two, install keepalived-1.2.24

I. 2.1 install LVS

Premise: the local Yum source configuration process has been configured in advance. Please see > http://blog.csdn.NET/zhang123456456/article/details/56690945.

1. Check the system kernel

Before installation, first check whether the system kernel supports LVS's IPVS module; since Linux 2.6, the system kernel has completely built-in LVS modules.

[root@mysql03 ~] # modprobe-l | grep ipvs

Kernel/net/netfilter/ipvs/ip_vs.ko

Kernel/net/netfilter/ipvs/ip_vs_rr.ko

Kernel/net/netfilter/ipvs/ip_vs_wrr.ko

Kernel/net/netfilter/ipvs/ip_vs_lc.ko

....

Kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko

Seeing the output information above indicates that the system kernel supports the IPVS module by default.

2. Install the dependency package

Since it is a source package installation, the following dependency packages need to be typed when compiling.

# yum-y install kernel-headers glibc-headers glibc-devel libgomp gcc libstdc++-devel gcc-c++

3. Yum install ipvsadm-1.26-4

[root@mysql03 ~] # yum-y install kernel-devel ipvsadm

=

Package Arch Version Repository Size

=

Installing:

Ipvsadm x8634 1.26-4.el6 c6-media 42k

Transaction Summary

=

Install 1 Package (s)

Total download size: 42 k

Installed size: 78 k

Downloading Packages:

Running rpm_check_debug

Running Transaction Test

Transaction Test Succeeded

Running Transaction

Installing: ipvsadm-1.26-4.el6.x86_64 1 Compact 1

Verifying: ipvsadm-1.26-4.el6.x86_64 1 Compact 1

Installed:

Ipvsadm.x86_64 0vl 1.26-4.el6

Complete!

4. Be a ln

[root@mysql03 ~] # ln-sv / usr/src/kernels/2.6.32-573.el6.x86_64/ / usr/src/linux can see the help information, which means that IPVS is installed successfully.

I. 2.2 install keepalived-1.2.24

1. Download address on the official website:

[root@mysql03 ~] # wget http://www.keepalived.org/software/keepalived-1.2.24.tar.gz

2. Extract the installation package:

[root@mysql03 ~] # ll keepalived-1.2.24.tar.gz

-rw-r--r--. 1 root root 601873 Jun 16 23:49 keepalived-1.2.24.tar.gz

[root@mysql03 ~] # tar-zxvf keepalived-1.2.24.tar.gz

3. Enter the installation directory:

[root@mysql03 ~] # cd keepalived-1.2.24

4. Install keepalived:

[root@mysql03 keepalived-1.2.24] # / configure-- prefix=/usr/local/keepalived-- with-kernel-dir=/usr/src/kernels/2.6.32-573.el6.x86_64/

Note:-- with-kernel-dri: specifies to use the header file in the kernel source code, that is, the include directory. This parameter is important and is only used when using LVS.

Main output:

Keepalived configuration

-

Keepalived version: 1.2.24

Compiler: gcc

Preprocessor flags:

Compiler flags:-Wall-Wunused-Wstrict-prototypes

Linker flags:

Extra Lib:-ldl-lssl-lcrypto

Use IPVS Framework: Yes

IPVS use libnl: No

IPVS syncd attributes: No

IPVS 64 bit stats: No

Fwmark socket support: Yes

Use VRRP Framework: Yes

Use VRRP VMAC: Yes

Use VRRP authentication: Yes

With ip rules/routes: Yes

SNMP keepalived support: No

SNMP checker support: No

SNMP RFCv2 support: No

SNMP RFCv3 support: No

SHA1 support: No

Use Debug flags: No

Stacktrace support: No

Memory alloc check: No

Libnl version: None

Use IPv4 devconf: No

Use libiptc: No

Use libipset: No

Build genhash: Yes

Build documentation: No

[root@mysql03 keepalived-1.2.24] # make & & make install

5. Copy scripts and files:

5.1 copy the execution file

[root@mysql03 keepalived-1.2.24] # cp / usr/local/keepalived/sbin/keepalived / usr/sbin/

5.2 copy the init.d file to etc and add the boot entry

[root@node6 keepalived-1.2.16] # cp / usr/local/keepalived/etc/rc.d/init.d/keepalived / etc/init.d/

5.3 copy the keepalived file to the etc and add the Nic configuration

[root@node6 keepalived-1.2.16] # cp / usr/local/keepalived/etc/sysconfig/keepalived / etc/sysconfig/

5.4 create a keepalived folder

[root@node6 keepalived-1.2.16] # mkdir / etc/keepalived

Copy the keepalived configuration file to etc

[root@node6 keepalived-1.2.16] # cp / usr/local/keepalived/etc/keepalived/keepalived.conf / etc/keepalived/

5.6 copy startup script

[root@node6 keepalived-1.2.16] # cp / usr/local/keepalived/sbin/keepalived / etc/keepalived/

6. Add boot entry

[root@mysql03 keepalived-1.2.24] # chkconfig-- add keepalived # must be added to ensure the existence of / etc/init.d/keepalived

7. Start

[root@mysql03 keepalived-1.2.24] # service keepalived start

Starting keepalived: [OK]

8. Close

[root@mysql03 keepalived-1.2.24] # service keepalived stop

Stopping keepalived: [OK] > OK, success!

Description: after installing the main nginx + Keepalived, install nginx + Keepalived in the same way.

II. Nginx + Keepalived configuration

1. Master nginx/ usr/local/nginx/conf/nginx.conf configuration

[root@mysql03 ~] # vi / usr/local/nginx/conf/nginx.conf

[root@mysql03 nginx] # cat / usr/local/nginx/conf/nginx.conf

User root root

Worker_processes 1

Events {

Worker_connections 1024

}

Http {

Include mime.types

Default_type application/octet-stream

Upstream web_app {

Server 10.219.24.21:8080 weight=1 max_fails=2 fail_timeout=30s

Server 10.219.24.21:8081 weight=1 max_fails=2 fail_timeout=30s

}

Server {

Listen 80

Server_name aa.com

Charset utf8

Location / {

Proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504

Proxy_pass http://web_app;

Proxy_redirect default

}

}

}

Description: after configuring the master nginx, similarly, configure the standby nginx

two。 Keepalived configuration of the primary nginx

[root@mysql03 nginx] # cat / etc/keepalived/keepalived.conf

! Configuration File for keepalived

Global_defs {

Router_id nginx-ha1

}

Vrrp_script check_nginx {

# script to check the status of nginx

Script "/ data/script/check_nginx.sh"

Interval 2

}

Vrrp_instance VI_1 {

State BACKUP

Interface eth0

Virtual_router_id 55

Priority 100

Advert_int 1

Nopreempt

Authentication {

Auth_type PASS

Auth_pass 1111

}

Virtual_ipaddress {

10.219.24.100

}

Track_script {

Check_nginx

}

}

3. Keepalived configuration of standby nginx

[root@redis01 ~] # vi / etc/keepalived/keepalived.conf

[root@redis01 ~] # cat / etc/keepalived/keepalived.conf

! Configuration File for keepalived

Global_defs {

Router_id nginx-ha2

}

Vrrp_script check_nginx {

Script "/ data/script/check_nginx.sh"

Interval 2

}

Vrrp_instance VI_1 {

State BACKUP

Interface eth0

Virtual_router_id 55

Priority 80

Advert_int 1

Authentication {

Auth_type PASS

Auth_pass 1111

}

Virtual_ipaddress {

10.219.24.100

}

Track_script {

Check_nginx

}

}

Add: if you have a firewall, you also need to set it. If there is no firewall setting, you can ignore it.

# iptables release Multicast address Traffic:

Iptables-I INPUT-d 224.0.0.18-j ACCEPT

Service iptables save

4. Deploy the nginx status check script check_nginx.sh

Check whether nc is installed, no NC package is installed, and install yum-y install nc, otherwise the nc command in the script check_nginx.sh will not be available.

[root@mysql03] # rpm-Q nc

Nc-1.84-24.el6.x86_64

4.2 create a script directory

[root@mysql03] # mkdir-p / data/script/

[root@mysql03 ~] # touch / data/script/check_nginx.sh

[root@mysql03 ~] # cat / data/script/check_nginx.sh

#! / bin/bash

# check nginx server status

# nginx port, if there are two ports and only need to be set to PORTS= "8081", then the following should also be changed to "$ret1"! = 11

PORTS= "80"

Function check_ports {

For port in $PORTS;do

Nc-z 127.0.0.1$ port | grep-Q succeeded

["${PIPESTATUS [1]}"-eq 0] & & mark=$ {mark} 1

Done

# if the mark value is empty, the port is impassable.

# if mark is equal to 1, a port is open.

Echo $mark

}

Ret1=$ (check_ports)

# if the nginx port is not available, an attempt will be made to restart nginx once

If ["$ret1"! = 1]; then

/ sbin/service nginx stop

/ sbin/service nginx start

Sleep 1

Ret2=$ (check_ports)

# if the port is still blocked, which means that the nginx service is abnormal, then stop keepalived and cause the VIP to switch.

["$ret2"! = 1] & & / etc/init.d/keepalived stop

Fi

5. Set executable permissions to the script:

[root@mysql03 ~] # chmod + x / data/script/check_nginx.sh

Description: 4. 5. All operations are operated on both the master and the standby.

One more point to add:

If keepalived does not start automatically after nginx returns to normal, you need to write a script to do this: after determining that nginx is normal, pull up keepalived.

The script is put into cron and executed every minute.

6. Enable the log of keepalived (the default log location of keepalived is / var/log/messages, it is best if we put it separately)

6.1Editor / etc/sysconfig/keepalived change KEEPALIVED_OPTIONS= "- D" to KEEPALIVED_OPTIONS= "- D-d-S 0"

[root@mysql03 ~] # vi / etc/sysconfig/keepalived

KEEPALIVED_OPTIONS= "- D-d-S 0"

6.2Editor / etc/rsyslog.conf

[root@mysql03 ~] # vi / etc/rsyslog.conf (# configuration file with the following line at the end)

Local0.* / var/log/keepalived.log

6.3 restart rsyslog:

[root@mysql03 ~] # service rsyslog restart

Note: after configuring as above, keepalived will log to / var/log/keepalived.log.

Description: 6. The operation is operated on both the master and the standby.

7. test

7.1 start all related services

Check the vip on the master nginx and use ip a l to view the floating vip. You can't see it directly with ifconfig.

[root@mysql03 ~] # ip a l

1: lo: mtu 65536 qdisc noqueue state UNKNOWN

Link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

Inet 127.0.0.1/8 scope host lo

Inet6:: 1/128 scope host

Valid_lft forever preferred_lft forever

2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000

Link/ether 00:0c:29:79:f4:02 brd ff:ff:ff:ff:ff:ff

Inet 10.219.24.26/8 brd 10.255.255.255 scope global eth0

Inet 10.219.24.100/32 scope global eth0 # VIP #

Inet6 fe80::20c:29ff:fe79:f402/64 scope link

Valid_lft forever preferred_lft forever

7.3 tcpdump grabs packets and only master ip 10.219.24.26 multicast packets are found, which is also in line with the working mechanism of VRRP

[root@mysql03] # tcpdump-n 'host 224.0.0.18'

Tcpdump: verbose output suppressed, use-v or-vv for full protocol decode

Listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes

23 Advertisement 07 IP 46.796186 IP 10.219.24.26 > 224.0.0.18: VRRPv2, Advertisement, vrid 55, prio 100, authtype simple, intvl 1s, length 20

23 Advertisement 07 IP 47.796910 IP 10.219.24.26 > 224.0.0.18: VRRPv2, Advertisement, vrid 55, prio 100, authtype simple, intvl 1s, length 20

23 Advertisement 07PAR 48.798713 IP 10.219.24.26 > 224.0.0.18: VRRPv2, Advertisement, vrid 55, prio 100, authtype simple, intvl 1s, length 20

7.4 keepalived is closed in master nginx, vip is found floating to slave nginx, and front-end access is uninterrupted.

-- master nginx

[root@mysql03 ~] # service keepalived stop

Stopping keepalived: [OK]

[root@mysql03 ~] # ip a l

1: lo: mtu 65536 qdisc noqueue state UNKNOWN

Link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

Inet 127.0.0.1/8 scope host lo

Inet6:: 1/128 scope host

Valid_lft forever preferred_lft forever

2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000

Link/ether 00:0c:29:79:f4:02 brd ff:ff:ff:ff:ff:ff

Inet 10.219.24.26/8 brd 10.255.255.255 scope global eth0

Inet6 fe80::20c:29ff:fe79:f402/64 scope link

Valid_lft forever preferred_lft forever

-- slave nginx

[root@redis01 ~] # ip a l

1: lo: mtu 65536 qdisc noqueue state UNKNOWN

Link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

Inet 127.0.0.1/8 scope host lo

Inet6:: 1/128 scope host

Valid_lft forever preferred_lft forever

2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000

Link/ether 00:0c:29:52:56:60 brd ff:ff:ff:ff:ff:ff

Inet 10.219.24.23/8 brd 10.255.255.255 scope global eth0

Inet 10.219.24.100/32 scope global eth0

Inet6 fe80::20c:29ff:fe52:5660/64 scope link

Valid_lft forever preferred_lft forever

7.5 when the previously closed keepalived is turned on, it is found that the vip has not drifted back, which coincides with the non-preemptive type of master / slave.

[root@mysql03 ~] # service keepalived start

Starting keepalived: [OK]

[root@mysql03 ~] # ip a l

1: lo: mtu 65536 qdisc noqueue state UNKNOWN

Link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

Inet 127.0.0.1/8 scope host lo

Inet6:: 1/128 scope host

Valid_lft forever preferred_lft forever

2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000

Link/ether 00:0c:29:79:f4:02 brd ff:ff:ff:ff:ff:ff

Inet 10.219.24.26/8 brd 10.255.255.255 scope global eth0

Inet6 fe80::20c:29ff:fe79:f402/64 scope link

Valid_lft forever preferred_lft forever

7.6 close nginx on slave nginx and find that the keepalived process on slave nginx has also been kill by script

The vip floated back to the master nginx successfully, and the front-end access was not affected.

-- slave nginx

[root@redis01] # / usr/local/nginx/sbin/nginx-s stop

[root@redis01 ~] # ps-ef | grep keepalived

Root 4907 2191 0 23:29 pts/0 00:00:00 grep keepalived

-- master nginx

[root@mysql03 ~] # ip a l

1: lo: mtu 65536 qdisc noqueue state UNKNOWN

Link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

Inet 127.0.0.1/8 scope host lo

Inet6:: 1/128 scope host

Valid_lft forever preferred_lft forever

2: eth0: mtu 1500 qdisc pfifo_fast state UP qlen 1000

Link/ether 00:0c:29:79:f4:02 brd ff:ff:ff:ff:ff:ff

Inet 10.219.24.26/8 brd 10.255.255.255 scope global eth0

Inet 10.219.24.100/32 scope global eth0

Inet6 fe80::20c:29ff:fe79:f402/64 scope link

Valid_lft forever preferred_lft forever

This is the end of the content of "how to install and configure Nginx+Keepalived master / standby high availability". Thank you for your reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.