Shulou(Shulou.com)06/02 Report--

I. Architecture

In the Keepalived + Nginx highly available load balancer architecture, keepalived is responsible for implementing the High-availability (HA) function to control the front-end machine VIP (virtual network address). When a device fails, the hot backup server can automatically switch the VIP instantly. The actual operation experience is only 2 seconds, and the DNS service can be responsible for the load balancing of the front-end VIP.

Nginx is responsible for controlling the load balancing of the backend web server and forwards the client's request to the backend Real Server for processing according to a certain algorithm, while Real Server returns the response directly to the client.

Two. simple principle

Both NGINX_MASTER and NGINX_BACKUP servers bind the ens32 network card to a virtual IP (VIP) address 192.168.2.242 through keepalived software. The VIP is currently bound to the ens32 of whoever carries the service. When the NGINX_MASTER fails, the NGINX_BACKUP will pass the heartbeat time advert_int 1 check set in the / etc/keepalived/keepalived.conf file. If the NGINX_MASTER normal state cannot be obtained. NGINX_BACKUP will instantly bind VIP to take over the work of nginx_master. When NGINX_MASTER is restored, keepalived will rebind the virtual VIP address 192.168.2.242 to the ens32 network card of NGINX_MASTER by determining the priority of the priority parameter.

Advantages of using this scheme

1. A flexible architecture is implemented, and when the pressure increases, you can temporarily add web servers to this architecture.

2.upstream has load balancing ability, can automatically judge the back-end machines, and automatically kick out the machines that can not provide services normally.

3. Regular distribution and redirection are more flexible than lvs. Keepalvied can ensure the effectiveness of a single nginx load balancer and avoid a single point of failure.

4. Using nginx for load balancing, there is no need to make any changes to the backend machine.

5.nginx is deployed in a docker container, which not only saves a lot of time in development, testing and deployment, but also quickly recovers business through mirroring in case of failure.

III. System environment

Two load machines are installed: centos7.5+docker+nginx+keepalived, named: NGINX_MASTER,NGINX_BACKUP.

The back-end web server, which can be any architecture that provides web services, is named WEB_1,WEB_2.

The backend database machine can be constructed arbitrarily, as long as it can provide database services.

Server operating system IP address installation software NGINX_MASTERCentos 7.5 64-bit 192.168.2.228docker+nginx+keepalivedNGINX_BACKUPCentos 7.5 64-bit 192.168.2.229docker+nginx+keepalivedWEB_1Centos 7.5 64-bit 192.168.2.226docker+apache+phpWEB_2Centos 7.5 64-bit 192.168.2.227docker+apache+php database cluster Centos 7.5 64-bit mysql cluster IV, web server deployment

Web server I use LAMP architecture here, specific installation and deployment, please refer to my other blog post "Centos 7 uses docker deployment LAMP to build wordpress blog system", https://blog.51cto.com/andyxu/2177116.

5. Install and configure nginx

Operate on NGINX_MASTER and NGINX_BACKUP servers respectively

1. Deploy the docker environment

(1) install docker

Note: docker Community version is installed

Yum install-y yum-utils device-mapper-persistent-data lvm2yum-config-manager-- add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repoyum makecache fastyum-y install docker-ce

(2) modify the configuration file, add the private warehouse address and Ali Cloud image address, and specify the docker data storage directory

Mkdir-p / data/dockermkdir-p / etc/dockervim / etc/docker/daemon.json {"registry-mirrors": ["https://registry.docker-cn.com"]," graph ":" / data/docker "," insecure-registries ": [" 192.168.2.225 https://registry.docker-cn.com"], 5000 "]}

(3) start docker and add boot to boot

Systemctl start dockersystemctl enable docker

2. Configure nginx container

(1) download nginx image

Docker pull nginx

(2) copy the nginx master configuration file to the local

Mkdir-p / data/docker/nginx/confdocker run-- name tmp-nginx-container-d nginx:latestdocker cp tmp-nginx-container:/etc/nginx/nginx.conf / data/docker/nginx/conf/docker rm-f tmp-nginx-container

(4) create a script that runs the nginx image

Vim docker_nginx.sh

#! / bin/bashdocker run-- name nginx-- restart=always-p 80:80\-v / data/docker/nginx/conf/nginx.conf:/etc/nginx/nginx.conf:ro\-v / data/docker/nginx/conf/conf.d:/etc/nginx/conf.d\-v / data/docker/nginx/html:/usr/share/nginx/html\-v / data/docker/nginx/logs:/var/log/nginx\-d nginx:latest

Note:-- restart=always is the restart policy. When the docker service is restarted, the container will start automatically.

(5) start the nginx container

Sh docker_nginx.sh

(6) modify nginx master configuration file

Vim / data/docker/nginx/conf/nginx.conf

User nginx;worker_processes 4; # number of worker processes, which is the number of cores of CPU or twice the number of error_log / var/log/nginx/error.log warn;pid / var/run/nginx.pid;events {use epoll; # Linux most commonly used event trigger mechanism worker_connections 65535;} http {include / etc/nginx/mime.types # set the mime type, which is defined by the mime.type file default_type application/octet-stream; log_format main'$remote_addr-$remote_user [$time_local] "$request"'$status $body_bytes_sent "$http_referer"'"$http_user_agent"$http_x_forwarded_for"; access_log / var/log/nginx/access.log main Sendfile on; # tcp_nopush on; keepalive_timeout 120; # gzip on; limit_conn_zone $binary_remote_addr zone=perip:10m; # add limit_zone and limit the same IP concurrency include / etc/nginx/conf.d/*.conf; # to include the nginx virtual host profile directory}

(7) create a upstream configuration file

Vim / data/docker/nginx/conf/conf.d/myhost.conf

Upstream xuad {ip_hash; # session persistence server 192.168.2.226 max_fails=1 fail_timeout=60s; server 192.168.2.227 max_fails=1 fail_timeout=60s;}

(8) create a virtual host profile

Vim / data/docker/nginx/conf/conf.d/xuad.conf

Server {listen 80; server_name localhost; # charset GB2312; location / {proxy_redirect off; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for Proxy_pass http://xuad;} # View the number of concurrent connections in nginx configuration location / NginxStatus {stub_status on; access_log off; auth_basic "NginxStatus";} access_log off; error_page 404 / 404.html Error_page 500502 503504 / 404.html; location = / 404.html {root html;} limit_conn perip 200; # the number of concurrency of the same ip is 200.503} will be returned if it exceeds.

(9) restart the nginx container

Docker restart nginx

6. Install and configure keepalived

Operate on NGINX_MASTER and NGINX_BACKUP servers respectively

1. Download and install keepalived

Note: keepalived is installed on the physical machine.

Yum install wget make gcc gcc-c++ openssl-develwget http://www.keepalived.org/software/keepalived-2.0.7.tar.gztar zxvf keepalived-2.0.7.tar.gzcd keepalived-2.0.7./configure-- prefix=/data/keepalived

If you report the following warning:

* WARNING-this build will not support IPVS with IPv6. Please install libnl/libnl-3 dev libraries to support IPv6 with IPVS.

Don't worry, we just need to use the VRRP function, not the IPVS function, so please make sure that the following three items are yes.

Use VRRP Framework: Yes

Use VRRP VMAC: Yes

Use VRRP authentication: Yes

Makemake install

2. Start keepalived as a service

Mkdir / etc/keepalivedcp / data/keepalived/etc/keepalived/keepalived.conf / etc/keepalived/systemctl enable keepalived

3. Modify keepalived configuration file

Vim / etc/keepalived/keepalived.conf

! Configuration File for keepalivedglobal_defs {notification_email {xuad@xuad.com} notification_email_from root@xuad.com smtp_server mail.xuad.com smtp_connect_timeout 30 router_id LVS_DEVEL vrrp_skip_check_adv_addr vrrp_strict vrrp_garp_interval 0 vrrp_gna_interval 0} vrrp_script chk_nginx {script "/ etc/keepalived/nginx_pid.sh" # script to check nginx status Interval 2 weight 3} vrrp_instance VI_1 {state MASTER # backup server change MASTER to BACKUP interface ens32 virtual_router_id 51 priority 100 # backup service to less than 100 Can be configured to 90 advert_int 1 authentication {auth_type PASS auth_pass 1111} virtual_ipaddress {192.168.2.242 # there are multiple vip to continue to add} track_script {chk_nginx}} below

4. Add a script to check the status of nginx

Vim / etc/keepalived/nginx_pid.sh

#! / bin/bash#version 0.0.1 nginx A = `ps-C nginx-- no-header | wc-l`if [$A-eq 0]; then systemctl restart docker sleep 3 if [`ps-C nginx-- no-header | wc-l`-eq 0]; then systemctl stop keepalivedfi fi

Script description: when the nginx process does not exist, the docker service will be automatically restarted, and the nginx container will be automatically started when the docker service starts; check the nginx process again, if it does not exist, stop the keepalived service, and then the NGINX_BACKUP host will automatically take over the work of the NGINX_MASTER.

Chmod + x / etc/keepalived/nginx_pid.sh

5. Configure firewalld firewall to allow vrrp protocol

VRRP (Virtual Router Redundancy Protocol, Virtual Router redundancy Protocol)

Firewall-cmd-permanent-add-rich-rule= "rule family=" ipv4 "source address=" 192.168.2.229 "protocol value=" vrrp "accept" firewall-cmd-reload

If it is a backup server, change the source address to the IP of the master server

6. Start keepalived

Systemctl start keepalived

VII. Testing

1. When NGINX_MASTER and NGINX_BACKUP server nginx are working normally

On NGINX_MASTER:

On NGINX_BACKUP:

The ens32 network card of the master server is normally bound to VIP, but backup is not bound, and the website can be accessed normally through the browser.

2. Close the nginx container of NGINX_MASTER

When the nginx container stops, it starts again immediately, and the nginx startup script is fine.

3. Disable the keepalived service of NGINX_MASTER

On NGINX_MASTER:

On NGINX_BACKUP:

NGINX_BACKUP 's ens32 network card has been instantly bound to VIP, and it is normal to access the website through a browser.

4. Start the keepalived service of NGINX_MASTER

On NGINX_MASTER:

On NGINX_BACKUP:

NGINX_MASTER 's ens32 network card is re-bound to VIP, and it is normal to access the website through a browser.

5. Shut down the WEB_1 server and access the website through the browser is normal.

Attachment 1: configure time synchronization

1. Install ntp on NGINX_MASTER and NGINX_BACKUP

Yum-y install ntp

2. Modify the ntp configuration file on NGINX_MASTER

Add the following two lines

Server 127.127.1.0 iburst local clock # add an IP address field that uses local time restrict 192.168.2.0 mask 255.255.255.0 nomodify # to allow updates

3. Start the ntp service on NGINX_MASTER, and join boot to start.

Systemctl start ntpdsystemctl enable ntpd

4. Add firewall policy to NGINX_MASTER

Only 192.168.2.229 is allowed to access the ntp service

Firewall-cmd-permanent-add-rich-rule= "rule family=" ipv4 "source address=" 192.168.2.229 "port protocol=" udp "port=" 123 "accept" firewall-cmd-reload

5. Synchronize the time of NGINX_MASTER on NGINX_BACKUP

Ntpdate 192.168.2.228

6. Set up scheduled tasks on NGINX_BACKUP and synchronize time at 05:01 every morning

Crontab-e1 5 * / usr/sbin/ntpdate 192.168.2.228 > > / var/log/upClock.log

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.