Shulou(Shulou.com)06/03 Report--

The following brings you an experiment on how keepalived+nginx+docker implements load balancing and highly available services. I hope it can give you some help in practical application. There are many things involved in load balancing, not many theories, and there are many books on the Internet. Today, we will use the accumulated experience in the industry to do an answer.

Four hosts are needed in this lab:

Centos7.6 operating system is installed on all four virtual machines.

Hostname ip function nginx_master192.168.8.128 load balancing, health check nginx_backup192.168.8. 129nginx_master 's younger brother web1192.168.8.126 provides web services and web2192.168.8.127 provides web services.

To put it simply, the role of the younger brother is to take the knife for the eldest brother when he is in danger. When nginx_master fails, nginx_backup works instead of nginx_master.

There is also a virtual ip:192.168.8.242

Keepalived can build multiple stateless single points into a highly available service through virtual IP (VIP) drift.

1. Configuration of cloud load balancer CVM nginx

The following operations are required on both nginxmaster and nginxbackup

1. Docker installation

Install yum-utils, which provides a yum-config-manager unit and installs device-mapper-persistent-data and lvm2 to store the two packages necessary for device mapping (devicemapper).

Yum install-y yum-utils device-mapper-persistent-data lvm2

Then configure a stable (stable) warehouse

The warehouse configuration is saved to the / etc/yum.repos.d/docker-ce.repo file.

Generate metadata

Install docker-ce

2. Configure deamon.json and nginx.conf files

Edit the daemon.json file, fill in the address of Aliyun image repository and specify the docker data storage directory

Vim daemon.json .

{"registry-mirrors": ["address of your Ali image warehouse"], "graph": "/ data/docker"}

Reload daemon and restart the docker service.

[root@localhost docker] # systemctl daemon-reload

[root@localhost docker] # systemctl restart docker

Add docker to boot boot

Pull a nginx image

Create the nginx configuration file directory, start a nginx-c container and copy the nginx.conf in the container locally

Delete Container nginx-c

These two steps are to modify the parameters based on the configuration file of the nginx container to achieve the desired functionality.

3. Create and configure the nginx container

(1) write a script to create a nginx container

#! / bin/bash

Docker run-name nginx-restart=always-p 80:80\

-v / data/docker/nginx/conf/nginx.conf:/etc/nginx/nginx.conf:ro\

-v / data/docker/nginx/conf/conf.d:/etc/nginx/conf.d\

-v / data/docker/nginx/html:/usr/share/nginx/html\

-v / data/docker/nginx/logs:/var/log/nginx\

-d nginx:latest

Note:-- restart=always is the restart policy. When the docker service is restarted, the container will start automatically.

(2) give the script execution permission and run

Chmod + x docker_nginx.sh

Sh docker_nginx.sh

(3) modify nginx master configuration file

Vim / data/docker/nginx/conf/nginx.conf

Before modification

After modification

(4) create upstream configuration file

Vim / data/docker/nginx/conf/conf.d/myhost.conf

Upstream xuad {

Ip_hash; # session persistence

Server 192.168.2.226 max_fails=1 fail_timeout=60s

Server 192.168.2.227 max_fails=1 fail_timeout=60s

}

(5) create a virtual host configuration file

Vim / data/docker/nginx/conf/conf.d/xuad.conf

Server {

Listen 80

Server_name localhost

# charset GB2312

Location /

{

Proxy_redirect off

Proxy_set_header Host $host

Proxy_set_header X-Real-IP $remote_addr

Proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for

Proxy_pass http://lxc;

}

# View the configuration of the number of concurrent connections in nginx

Location / NginxStatus

{

Stub_status on

Access_log off

Auth_basic "NginxStatus";} access_log off

Error_page 404 / 404.html

Error_page 500 502 503 504 / 404.html

Location = / 404.html {

Root html

}

Limit_conn perip 200; # the number of concurrency for the same ip is 200.503 is returned if it exceeds it.

}

(6) restart the nginx container

Docker restart nginx

Second, the configuration of load balancing server keepalived

The following operations are required on both nginxmaster and nginxbackup

1 download and install keepalived

Operate on NGINX_MASTER and NGINX_BACKUP servers respectively

Note: keepalived is installed on the physical machine.

Yum install wget make gcc gcc-c++ openssl-devel

Wget http://www.keepalived.org/software/keepalived-2.0.7.tar.gz

Tar zxvf keepalived-2.0.7.tar.gz

Cd keepalived-2.0.7

. / configure-- prefix=/data/keepalived

If you report the following warning:

WARNING-this build will not support IPVS with IPv6. Please install libnl/libnl-3 dev libraries to support IPv6 with IPVS.

Don't worry, we just need to use the VRRP function, not the IPVS function, so please make sure that the following three items are yes.

Compile

Make

Make install

2. Start keepalived as a service

Create the / etc/keepalived directory and copy the keepalived configuration file to the / etc/keepalived directory and add it to boot.

Mkdir / etc/keepalived

Cp / data/keepalived/etc/keepalived/keepalived.conf / etc/keepalived/ systemctl enable keepalived

3. Modify keepalived configuration file

Vim / etc/keepalived/keepalived.conf

! Configuration File for keepalived

Global_defs {

Notification_email {

Lxc@lxc.com

}

Notification_email_from root@lxc.com

Smtp_server mail.lxc.com

Smtp_connect_timeout 30

Router_id LVS_DEVEL

Vrrp_skip_check_adv_addr

Vrrp_strict

Vrrp_garp_interval 0

Vrrp_gna_interval 0

}

Vrrp_script chk_nginx {

Script "/ etc/keepalived/nginx_pid.sh" # script to check the status of nginx

Interval 2

Weight 3

}

Vrrp_instance VI_1 {

Change MASTER to BACKUP on state MASTER # backup server

Interface ens32 virtual_router_id 51 priority 100 # backup service changes 100 to less than 100 and can be configured to 90

Advert_int 1

Authentication {

Auth_type PASS

Auth_pass 1111

}

Virtual_ipaddress {

192.168.8.242 # there are several vip that can be added below

}

Track_script {

Chk_nginx

}

}

4. Add a script to check the status of nginx

Vim / etc/keepalived/nginx_pid.sh

#! / bin/bash

# version 0.0.1

A=ps-C nginx-- no-header | wc-l

If [$A-eq 0]; then

Systemctl restart docker

Sleep 3

If [ps-C nginx-- no-header | wc-l-eq 0]; then

Systemctl stop keepalived

Fi

Fi

Script description: when the nginx process does not exist, the docker service will be automatically restarted, and the nginx container will be automatically started when the docker service starts; check the nginx process again, if it does not exist, stop the keepalived service, and then the NGINX_BACKUP host will automatically take over the work of the NGINX_MASTER.

Chmod + x / etc/keepalived/nginx_pid.sh

5. Configure firewalld firewall to allow vrrp protocol

VRRP (Virtual Router Redundancy Protocol, Virtual Router redundancy Protocol)

Firewall-cmd-permanent-add-rich-rule= "rule family=" ipv4 "source address=" 192.168.129 "protocol value=" vrrp "accept"

Firewall-cmd-reload

If it is a backup server, change the source address to the IP of the master server

6. Start keepalived

Systemctl start keepalived

Third, configure time synchronization

Install ntp on NGINX_MASTER and NGINX_BACKUP

Yum-y install ntp

Modify the ntp configuration file on NGINX_MASTER

Add the following two lines.

Vim / etc/ntp.conf

Server 127.127.1.0 iburst local clock # add an IP address field that uses local time restrict 192.168.8.0 mask 255.255.255.0 nomodify # to allow updates

Start the ntp service on NGINX_MASTER and join boot boot

Systemctl start ntpd systemctl enable ntpd

Add a firewall policy on NGINX_MASTER

Only 192.168.8.129 is allowed to access the ntp service.

Synchronize the time of NGINX_MASTER on NGINX_BACKUP

Ntpdate 192.168.8.128

Set up scheduled tasks on NGINX_BACKUP

Synchronize time at 05:01 every morning.

Crontab-e 1 5 * / usr/sbin/ntpdate 192.168.8.128 > > / var/log/upClock.log

Fourth, load balancing server testing

1. Test the nginx server

(1) View VIP when NGINX_MASTER and NGINX_BACKUP server nginx are working normally

On NGINX_MASTER

On NGINX_BACKUP

The ens32 network card of the master server is normally bound to VIP, but backup is not bound, and the website can be accessed normally through the browser.

(2) close the nginx container of NGINX_MASTER

When the nginx container stops, it starts again immediately, and the nginx startup script is fine.

(3) disable the keepalived service of NGINX_MASTER

On NGINX_MASTER.

On NGINX_BACKUP

NGINX_BACKUP 's ens32 network card has been instantly bound to VIP, and it is normal to access the website through a browser.

(4) start the keepalived service of NGINX_MASTER

On NGINX_MASTER

On NGINX_BACKUP

NGINX_MASTER 's ens32 network card is re-bound to VIP, and it is normal to access the website through a browser.

2. Test the WEB server

Shut down the WEB_1 server

It is normal to access the website through a browser.

After reading the above about how the experimental keepalived+nginx+docker implements load balancing and highly available services, if there is anything else you need to know, you can find out what you are interested in in the industry information or find our professional and technical engineers for answers. Technical engineers have more than ten years of experience in the industry.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.