Shulou(Shulou.com)06/03 Report--

Preface

Reverse proxy (Reverse Proxy) means that the proxy server accepts the connection request on the internet, then forwards the request to the server on the internal network, and returns the result obtained from the server to the client requesting the connection on the internet. At this time, the proxy server is externally represented as a server. By placing reverse proxy node servers everywhere in the network to form a layer of intelligent virtual network based on the existing Internet, the CDN system can redirect users' requests to the nearest service node in real time according to the comprehensive information such as network traffic, connection of each node, load status, distance to the user and response time. In this article, we will use Docker+Etcd+Confd+Nginx architecture to implement reverse proxy.

one。 Architecture design

In this article, we mainly introduce the Docker+Etcd+Confd+Nginx solution, which is more efficient and fast, with lower maintenance cost and fault tolerance, and stronger distributed support, as shown in the following figure:

The approximate flow of the above diagram is as follows:

1. Deploy the consul service as a binary package on the docker01 host and run it in the background, and its identity is leader.

2. Docker02 and docker03 run the consul service as a container and add it to the docker01 consensus cluster

3. Run registrator container in the background on the host docker02 and docker03 to automatically discover the services provided by docker container.

4. Deploy Nginx on docker01, provide reverse proxy service, run two web containers based on Nginx image on docker02 and docker03 hosts, and provide different web page files to test the effect.

5. Install the consul-template command on docker01, write the collected information (the information collected by registrator to the container) into the template template, and finally write it into the configuration file of Nginx.

6. At this point, the client can access the Nginx reverse proxy server (docker01) to obtain the web page files provided by the Nginx container running on the docker02 and docker03 servers.

Note: registrator is an automatic discovery of services provided by docker container and registers services in the back-end services registry (data center). It is mainly used to collect information about the container running service and send it to consul. In addition to consul, the data center also has etcd, zookeeper and so on.

two。 Architectural advantage

Although Docker+Consul+Nginx appears to be the use of three components, it turns out to be an organic whole. They are interrelated and interact with each other to fully meet our needs for highly available and efficient service architecture solutions. They are one of the most ideal combinations in the Docker ecosystem and have the following advantages:

1. The discovery and registration component consul uses the Raft algorithm to ensure consistency, which is more direct than the complex Paxos algorithm. By comparison, zookeeper uses Paxos, while etcd uses Raft

two。 Multi-data center, multi-data center cluster can avoid a single point of failure of a single data center. Zookeeper and etcd do not provide support for multiple data center functions.

3. Real-time discovery and unaware service refresh, with resource flexibility and flexibility

4. Health check. Load can be dynamically balanced on available service instances. Etcd does not provide this function.

5. Enough Docker containers (provided that the architecture resources are sufficient to ensure performance support)

6.http and dns protocol interface, the integration of zookeeper is more complex, etcd only supports http protocol

7. The scale is convenient for rapid adjustment. The official web management interface is provided, but etcd does not have this function.

8.nsul template is used with consul to support multiple access layers, such as Nginx and Haproxy.

three。 Experimental environment host iP address service docker01192.168.1.11consul+consul-template+nginxdocker02192.168.1.13consul+registratordocker03192.168.1.20consul+registrator

The three hosts turn off the firewall, disable selinux, and change the hostname as described above.

four。 Deploy consul service (1) docker01 go to the official website https://www.consul.io/downloads.html to download the consul service [root@docker01 ~] # unzip consul_1.5.1_linux_amd64.zip / / now it is a local import package Need to extract the [root@docker01 ~] # mv consul/ usr/local/bin/// mobile service to the bin directory [root@docker01 ~] # chmod + x / usr/local/bin/consul// to give an executable permission (2) launch consul agent [root @ docker01 ~] # consul agent-server-bootstrap-ui-data-dir=/var/lib/consul-data-bind=192.168.1.11-client=0.0.0.0-node=master

PS: / /-bootstrap: when this option is added, it is usually used when server is a single node, and it is elected to leader since.

Parameter explanation:

-server: add a service

-bootstrap: it is generally used when server is a single node, and it is elected as leader since.

-data-dir:key/volume specifies the directory where the data is stored

-ui: open the internal web interface

-bind: specify the ip to enable the service

-client: specify the client to access

-node: the name used for communication within the cluster. The default is the hostname.

Now this ip is for external use.

PS: open port

8300 Cluster nodes

8301 access within the cluster

8302 Communication across data centers

8500 web ui interface

8600 Port that uses dns protocol to view node information

Please refer to the following figure to see the meaning of the port:

At this point, the command to start consul will occupy the terminal, and you can use the nohup command to keep it running in the background.

[root@docker01 ~] # nohup consul agent-server-bootstrap-ui-data-dir=/var/lib/consule-data-bind=192.168.1.11-client=0.0.0.0-node=master & (3) View consul port information * * [root@docker01 ~] # consul info

five。 Docker01 download and deploy consul-template

On https://github.com/hashicorp/consul-template, download consul-template

[root@docker01 ~] # unzip consul-template_0.19.5_linux_amd64.zip// extract the installed consul-template package [root@docker01 ~] # mv consul-template / usr/local/bin/// move to the command directory [root@docker01 ~] # chmod + x / usr/local/bin/consul-template / / give a brief description of six and seven steps to give executable permission

Operate on docker01 and docker02

Let's start with a general idea of how to operate on a docker server:

Create a registrator container on both docker servers and notice the consul service center

Run two nginx containers (ports randomly generated) on docker01 and two nginx containers (ports randomly generated) on docker02

Modify the index.html page content in these 4 nginx containers to (xgp-web01, xgp-web02, xgp-web03, xgp-web04)

Access the consul web interface to verify

Access the nginx server address http://192.168.1.11:8000 for verification

six。 Docker02,docker03, join the consul cluster

Here we use a container to run the consul service.

(1) download the l image required by consu [root@docker02] # docker pull consul (2) Open a container based on consul image [root@docker02] # docker run-d-name consul-p 8301-p 8301:8301/udp-p 8500 advertise 8600-p 8600:8600/udp-- restart always progrium/consul-join 192.168.1.11-advertise 192.168.1.13-client 0.0.0.0-node=node01

Parameter explanation:

-d: daemon

-- name: container name

-- restart: the container runs along with the docker service

-advertise: declare the local address

-join: declare the server address

-name in the node:consul cluster

(3) docker to view the information of consul cluster members [root@docker01 ~] # consul members

(4) after the container is opened by two docker, docker01 can view it.

(5) browsers access http://192.168.1.11:8500

seven。 Deploy registrator services on docker02 and docker03

Registrator is a tool that automatically discovers the services provided by docker container and registers or cancels services in the back-end service registry, which supports conusl, etcd, skydns2, zookeeper and so on.

(1) download registrator image [root@docker02 ~] # docker pull registrator// download registrator image (2) based on registrator image, open a container [root@docker02 ~] # docker run-d-name registrator-v / var/run/docker.sock:/tmp/docker.sock-- restart always gliderlabs/registrator consul://192.168.1.13:8500

Parameter description:

-- network: sets the running docker container to host network mode

-v / var/run/docker.sock: mounts the Unix domain socket listened to by default by the Docker daemon (Docker daemon) of the host to the container

-- ip: network has just been specified in host mode, so we specify IP as the IP of the host.

The last option for consul:j is to configure the IP and port of the consul server.

(3) Open a nginx container [root@docker02] # docker run-d-P-- name nginx nginx:latest (4) browser to view http://192.168.1.11:8500/ui/dc1/nodes

VIII.Docker01 deploy a nginx service

Configure nginx. The general idea of configuration is as follows:

Create a directory consul in / usr/local/nginx/conf with a custom directory name

Create a nginx.ctmpl template in the consul directory

Add an include entry to the nginx.conf configuration and point to the consul directory

Restart the nginx service

(1) install and enable nginx service

Install the nginx dependency package

[root@docker01 ~] # yum-y install pcre pcre-devel openssl openssl-devel zlib zlib-devel

Compile and install nginx

[root@docker01] # cd nginx-1.14.0/ [root@docker01 nginx-1.14.0] #. / configure-- user=nginx-- group=nginx-- with-http_stub_status_module-- with-http_realip_module-- with-pcre-- with-http_ssl_module & & make & & make install

Create the required user and linked command directory

[root@docker01 nginx-1.14.0] # useradd-M-s / sbin/nologin nginx [root@docker01 nginx-1.14.0] # ln-s / usr/local/nginx/sbin/* / usr/local/bin/

Check if there is a problem with nginx and turn on nginx

[root@docker01 nginx-1.14.0] # nginx- tnginx: the configuration file / usr/local/nginx/conf/nginx.conf syntax is oknginx: configuration file / usr/local/nginx/conf/nginx.conf test is successful [root@docker01 nginx-1.14.0] # nginx

PS: here nginx acts as a reverse proxy to proxy the container service of nginx on the backend docker02 and docker03, so let's go to docker02 and docker03. Deploy some services on the, in order to see the effect of the load later, so after we have run the container, we will make a distinction between the contents of the main interface.

(2) after installation, the local test visits [root@docker01 nginx-1.14.0] # curl 127.0.0.1

(3) deployment environment host service docker02nginxweb01,web02docker03nginxweb03,web04 downloads nginx image (both docker02,docker03) [root@docker02 ~] # docker pull nginx// downloads nginx image docker01 operation

Run the container mentioned above based on the nginx image and set up the test page

Web01 [root@docker02 ~] # docker run-itd-- name web01-P nginx:latest [root@docker02 ~] # docker exec-it web01 / bin/bashroot@44b59d07202f:/# cd / usr/share/nginx/html/root@44b59d07202f:/usr/share/nginx/html# echo web01 > index.htmlweb02 [root@docker02 ~] # docker run-itd-name web02-P nginx:latest [root@docker02 ~] # docker exec-it web02 / bin/bashroot@44b59d07202f:/# cd / usr/share/ Nginx/html/root@44b59d07202f:/usr/share/nginx/html# echo web02 > index.htmldocker02 operation

Run the container mentioned above based on the nginx image and set up the test page

Web03 [root@docker03 ~] # docker run-itd-- name web03-P nginx:latest [root@docker03 ~] # docker exec-it web03 / bin/bashroot@fd8e8b2df136:/# cd / usr/share/nginx/html/root@fd8e8b2df136:/usr/share/nginx/html# echo web03 > index.htmlroot@fd8e8b2df136:/usr/share/nginx/html# exitweb04 [root@docker03 ~] # docker run-itd-- name web04-P nginx:latest [root@docker03 ~] # docker exec -it web04 / bin/bashroot@fd8e8b2df136:/# cd / usr/share/nginx/html/root@fd8e8b2df136:/usr/share/nginx/html# echo web04 > index.htmlroot@fd8e8b2df136:/usr/share/nginx/html# exit (4) docker01 change nginx configuration file [root@docker01 ~] # cd / usr/local/nginx/// enter nginx configuration file directory [root@docker01 nginx] # mkdir consul// create consul directory [root@docker01 nginx] # cd consul/// Enter the consul directory to create a nginx.ctmpl template [root@docker01 consul] # vim nginx.ctmplupstream http_backend {range service "nginx"}} server {{.Address}}: {{.Port}} {{end} server {listen 8000; server_name localhost; location / {proxy_pass http://http_backend;}}

Nginx.ctmpl template content is two paragraphs of meaning, familiar with nginx friends can understand: the first definition of nginx upstream a simple template, the second definition of a server, listening to port 8000, reverse proxy to upstream.

Modify the nginx configuration file, use the include parameter to include the newly created file [root@docker01 consul] # cd / usr/local/nginx/conf/ [root@docker01 conf] # vim nginx.conf include / usr/local/nginx/consul/*.conf; # file and finally add (to be in curly braces) to generate a vhost.conf configuration file, and restart nginx (which will occupy the terminal)

Use the consul-template command to produce a new configuration file based on the template and reload the configuration file for nginx.

[root@docker01 conf] # consul-template-consul-addr 192.168.1.11 template-template "/ usr/local/nginx/consul/nginx.ctmpl:/usr/local/nginx/consul/vhost.conf:/usr/local/bin/nginx-s reload"

Parameter description:

-- consul-addr: specify the ip and port of the consul service

. / nginx.ctmpl: this is to start the process with the template nginx.ctmpl, which is either a relative path or an absolute path for writing

The file name after the vhost.conf:nginx.ctmpl template is generated, which can also write the absolute path. If you do not write the absolute path, the file will be generated in the current directory (/ usr/local/nginx/consul/)

At this point, this command will occupy the terminal, so you can use the nohup command to keep it running in the background and restart the nginx service.

[root@docker01 conf] # nohup consul-template-consul-addr 192.168.1.11 template-template "/ usr/local/nginx/consul/nginx.ctmpl:/usr/local/nginx/consul/vhost.conf:/usr/local/sbin/nginx-s reload" &

Check to see if the file is generated and if there is content in it

[root@docker01 ~] # cd / usr/local/nginx/consul/ [root@docker01 consul] # lsnginx.ctmpl vhost.conf

[root@docker01 consul] # cat vhost.conf

At this point, you should be able to see that the newly produced vhost.conf configuration file is in effect, and you can access the native port 8000 to get the services provided by different containers.

Test access to [root@docker01 consul] # curl 127.0.0.1:8000web01

At this point, you can see the effect of load balancing!

If the access is not successful

Check to see if port 8000 is open

[root@docker01 consul] # ss-lnt

Check the nginx configuration file

[root@docker01 consul] # nginx-tnginx: the configuration file / usr/local/nginx/conf/nginx.conf syntax is oknginx: configuration file / usr/local/nginx/conf/nginx.conf test is successful

Check the nginx configuration file written by yourself

[root@docker01 consul] # cd / usr/local/nginx/consul/ [root@docker01 consul] # cat nginx.ctmpl upstream http_backend {{range service "nginx"}} server {{.Address}: {{.Port}; {{end} server {listen 8000; server_name localhost; location / {proxy_pass http://http_backend;}}

If the nginx configuration file is fine, restart nginx

[root@docker01 consul] # nginx-s reload Test automatic Discovery

Docker02 creates a test container

[root@docker02] # docker run-itd-- name web05-P nginx:latest [root@docker02 ~] # docker exec-it web05 / bin/bashroot@44b59d07202f:/# cd / usr/share/nginx/html/root@44b59d07202f:/usr/share/nginx/html# echo web02 > index.html [root@docker02 ~] # docker ps

Docker01 View

[root@docker01 consul] # cd / usr/local/nginx/consul/ [root@docker01 consul] # cat vhost.conf

Docker01 test access

[root@docker01 consul] # curl 127.0.0.1:8000

/ / ditto

At this point, you can see the effect of load balancing!

At this point, you do not need to consider that the backend web server will be updated automatically if you add or delete it. This is because the function of the / usr/local/sbin/nginx-s reload added after running the consul-template command!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.