Shulou(Shulou.com)06/02 Report--

Using Nginx+Keepalived+iis+Memcached to build highly available cluster services

The general architecture is shown in the figure above.

1. Set up the Windows server

The company uses a lot of IIS as a web service, which wastes a lot of time where session shares.

At first I planned to use Sqlserver database sharing, but later I found that IIS comes with session sharing.

As shown in the above figure, the connection string in session state can be connected using the session of another server IIS service. Of course, 3 servers and 4 servers can also be set up. As long as they can form a circle, they will share and converge with each other. To set the method, run regedit → to open the registry → and find the node HKEY_LOCAL_MACHINE\ SYSTEM\ CurrentControlSet\ Services\ aspnet_state\ Parameters node → sets the key value of AllowRemoteConnection to "1" (1 to allow connections to remote computers, 0 means forbidden)

Personal test is effective.

Another is when setting up a site, because we have multiple sites on an iis, which is usually used as shown below.

Create a new site, using port 80, and then there are a lot of virtual directories or applications under the site (which must be set as applications here). Because we create a new directory on disk and put a lot of sites under the directory, because the program must be written in a first-level directory, it can only be converted to the program to be properly accessed.

Going back to the main question, because nginx is now used for load balancing, it involves the back-end detection module of Nginx. Here, it is recommended that a site is a website. If there are more sites, you can use different ports, and it cannot be like the above. The specific reasons are described in detail later.

This is the process of setting up the windows system.

2. Start building Linux services

The first is to set up Nginx, which uses the reverse proxy module of Nginx and the health check module of the back-end server.

1) build nginx, turn off the firewall, shut down selinux and restart the server

2), because the backend detection module is a third-party module, you need to compile and install Nginx

The version of Nginx used is nginx-1.14.0.tar.gz, and the third-party module is nginx_upstream_check_module-master.zip

Why do you use nginx-1.14.0.tar.gz, because the check_1.14.0+.patch only sees 1.14 (the highest) in the third-party module, which may not be supported if you use the new version?

3) patch Nginx first

Because mine has already been called. All will be prompted like this, and the normal prompt is the following:

If it appears as shown in the figure, it means that the patch is successful.

4), compile Nginx

First of all, install the development kit group directly, and there are fewer errors in later compilation.

Yum groupinstall Development-tools-y has a long waiting time.

Additional installation: yum-y install perl-devel perl-ExtUtils-Embed this package is not usually used, but if it is not installed, it will report an error when compiling the third-party module, so install it directly

Nginx compilation parameters:

/ configure-- prefix=/usr/local/nginx1.14-- conf-path=/usr/local/nginx1.14/nginx.conf-- error-log-path=/usr/local/nginx1.14/logs/error.log-- http-log-path=/usr/local/nginx1.14/logs/access.log-- with-http_stub_status_module-- add-module=/home/nginx_upstream_check_module-master-- with-http_gunzip_module-- with-http_gzip_static_ Module-with-http_ssl_module-with-http_v2_module-with-http_mp4_module-with-http_gunzip_module-with-http_gzip_static_module-with-http_secure_link_module-with-http_auth_request_module-with-http_sub_module with-http_perl_module-with-perl_modules_path=/usr/local/nginx1.14/modules/

The above are the nginx compilation parameters. Enough compilation is done according to the above. If you don't know the compilation and installation process, you can stop talking about it here under Baidu or google.

If it passes smoothly, you will see the folder of ngnx1.14 under / usr/local/.

As shown in the figure

Let's start to configure nginx. The configuration file nginx.conf recommends backing up nginx.conf.back first.

5), configure nginx.conf

The user used by user www; to start nginx

Worker_processes auto; limit supports auto, so there is no need to write the number of cpu.

# error_log logs/error.log

# error_log logs/error.log notice

Error_log logs/error.log info

Pid logs/nginx.pid

Events {

Worker_connections 1024

}

Http {

Include mime.types

Default_type application/octet-stream

# log_format main'$remote_addr-$remote_user [$time_local] "$request"'

'$status $body_bytes_sent "$http_referer"' "$http_user_agent" $http_x_forwarded_for "; # access_log logs/access.log main;sendfile on;#tcp_nopush on;keepalive_timeout 65; set timeout gzip on;upstream ERP {A total of two back-end servers are defined here, but these are all port 80 used (default). You need to modify ip_hash later. Server 192.168.3.100; server 192.168.0.229; check interval=3000 rise=2 fall=5 timeout=1000 type=http; uses the patch you just made. The backend health check is recommended.

One success per second is success, and one failure is failure. For those with higher requirements, it is suggested that you can further reduce it. Here is a pit. If the setting time is too long, the error page will be brushed out when refreshing. Because the stopped IIS has not been removed from the Nginx, Nginx will also dispatch the request to the stopped Server at the backend within this period of time.

"A brief introduction to the back-end health check module. This module nginx_upstream_check_module-master.zip should be made on Taobao. Taobao's tengine supports this module by default, which is explained on its official website.

Tengine is a Web server project initiated by Taobao. On the basis of Nginx, it adds a lot of advanced functions and features to meet the needs of high-volume websites. The performance and stability of Tengine have been well tested in large websites such as Taobao and Tmall Mall. Its ultimate goal is to create an efficient, stable, secure and easy-to-use Web platform.

Since December 2011, Tengine has become an open source project, and the Tengine team is actively developing and maintaining it. The core members of the Tengine team come from Internet companies such as Taobao and Sogou. Tengine is the result of community cooperation, and we welcome all of you to participate and contribute to it. "

The meaning of the parameter is:

Interval: check the interval between requests.

Fall (fall_count): the server is marked as shut down after the fall_count failed check.

Rise (rise_count): after a successful rise_count check, the server is marked.

Timeout: check the timeout of the request.

Default_down: specifies the initial state of the backend server, which is turned off by default.

Type: check the protocol type:

Tcp: a simple TCP socket connects and looks at a byte.

Ssl_hello: sends client SSL hello packets and receives server SSL hello packets.

Http: sends a http request packet, receives and parses the http response to diagnose whether the upstream server is active.

Mysql: connect to the mysql server and receive a greeting response to diagnose if the upstream server is active.

Ajp: sends AJP Cping packets, receives and parses the AJP Cpong response to diagnose whether the upstream server is active.

Port: specify the check port in the back-end server. It can be different from the original server port. The default port is 0, which is the same as the original back-end server. Add this option after tengine-1.4.0.

Check_http_send "GET / HTTP/1.0\ r\ n\ r\ n"

I didn't use tengine, because I configured a module command for more than half an hour and always reported an error, so I had no choice but to patch nginx at the beginning.

}

Server {

Listen 80

Server_name localhost

# charset koi8-r

Location / nginxcon {

Stub_status

}

# access_log logs/host.access.log main

Location ^ ~ / XQZC {

Proxy_pass http://ERP/XQZC;

}

Location ^ ~ / BJ_CS {

Proxy_pass http://ERP/BJ_CS;

}

Location / status {

Check_status

Access_log off

Allow SOME.IP.ADD.RESS; # deny all;} error_page 500 502 503 504 / 50x.htl; location = / 50x.html {root html;}

There are two other things worth explaining.

As shown in the following figure:

Of course, it will check all the time.

So there is nothing wrong with the check interval=1000 rise=1 fall=1 timeout=1000 type=http; interval set above.

This is the complete configuration of Nginx, but there is a very big problem, that is, when checking the back-end server, because only the health check of port 80 http is set here, and we have added a lot of applications to a website.

So here comes the problem: if we modify one of the applications (hereinafter referred to as small websites), such as updating, we must stop this iis in order to replace one of the small websites. If we stop IIS, all session will be forwarded to another server, and we only maintain one of the small websites, adding that there are 10 small websites, then the remaining 9 will be no problem at all.

This results in a waste of resources.

All to change to, a small website to set up a site, using different style slogans

Well, in this way, when maintaining a small website, you only need to stop one of them without affecting the other. When defining a Upstream, you must define it separately according to the port number, and it will look like this.

As shown in the figure, this is much better, and it can only detect one, so it becomes as shown in the following figure

If you can't use that, let's stop that. It's for the best.

Finally configure keepalived

This is relatively simple, directly yum to install keepalived

Layer layer 3 detection: perform ICMP ping packet detection to confirm whether the host is alive. If abnormal, the host will be removed from the server cluster.

Layer layer 4 detection: performs port detection, such as 80, 3306, etc., and removes the server from the cluster when the port is blocked.

Layer layer 7 detection: this is based on the application, such as whether the http error code is 200, confirm whether the host is normal.

Here we use the third..

Let's talk about the pit we encountered first.

1),

Here is the command that has been used directly at the beginning, and all tutorials on the Internet allow you to use the command directly.

However, the new version does not seem to support direct command checking in this configuration section, and I don't know if I set it incorrectly or really cannot, so I use script check here. The content of the script still needs to be optimized. For the time being, the content is as follows:

According to the keepalived health inspection right reduction rule, if the check result fails by default, the right will be reduced if the failure occurs. Of course, we can also modify the configuration file to reduce the right if successful, and use the default for the time being. As shown below:

If both servers have vip, please continue to clear the firewall and shut down selinux because of them.

Backup keepalived directly copies the past master configuration file and modifies state and prioriy.

Write in a bit of a hurry, bad places welcome to leave a message to correct, thank you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.