Shulou(Shulou.com)06/03 Report--

Cluster concept:

By two or more service entities to coordinate and cooperate to complete a series of work mode, external performance as a whole.

Characteristics

Assign user request

Fail-over

Shared storage

Structure:

Agent load scheduler

Business tier server pool

Storage shared storage

1. Vertical expansion

Add more resources to the same computing resource pool, such as more memory, disks, or virtual cpu, to cope with the increased application load

two。 Horizontal expansion

More machines or equipment need to be added to the computing platform to deal with the growing demand

Upward expansion: upgrade, replacement and upgrade of hardware

Disadvantages:

Performance: there is a limit to improvement

Disadvantages:

DNS does not have the ability to detect the health status of backend devices, that is, when WEB2 devices fail, DNS still assigns tasks to WEB2 according to its own cached records.

Defect of dns: unable to achieve health check

Cache dns failed to implement allocation user request

The parsing cache of DNS results in uneven distribution of tasks, resulting in excessive pressure on a single server.

Algorithm:

RR (Round-Robin): polling

WRR (weightd-Round-Robin): weight (weighted polling)

Using the agent agent mechanism, we can detect the health status of the back-end devices and effectively improve the work efficiency.

The most basic cluster topology:

Load balancing Cluster (LBC Load Balance Cluster)

Function: reduce the pressure on a single server

Load Balancing load balancing, different nodes are independent of each other and do not share any resources; through a certain algorithm, the client access requests are evenly distributed to each node of the cluster, making full use of the resources of each node. Load balancing expands the bandwidth of network equipment and servers, increases throughput, and strengthens network data processing capacity.

Achieve:

Software: LVS RAC

Hardware: F5

1. Share the load of the system

two。 Monitor the operation of the node

Under one network, you can build multiple load balancing clusters to disperse the pressure:

Difference:

The later maintenance of the hardware is better, but the price is on the high side.

The software is flexible and the price is relatively cheap.

High availability cluster (HA): ensure the continuous operation of the server

Provide persistent access to the application

Free switchover in the event of failure

No matter which one is implemented to access web1 or web2, the result is the same and there will be no deviation.

In order to use web1 and web2 content consistency:

The mechanism used is: heartbeat detection

Method to realize:

RS232 serial port line (heartbeat line)

Send a packet every few seconds to see if the other party has a response, and if there is no response, it will judge that there is a problem with the target.

Existing network card

Detect through the eth2 port between the two servers (send packets at regular intervals for confirmation)

Eth0 is for users to access

Server availability requirements criteria:

99% 2 91 years 87.6 hours (downtime) 99.9% 3 91 years 8.8 hours (downtime) 99.99% 4 91 years 53 minutes (downtime) 99.99% 5 91 years 3-5 minutes (downtime)

If the heartbeat line fails:

Both hosts think that the other is out of order (dead), and both will rush to take over the other's work, respond to customer requests, and snatch shared resources and data. (this phenomenon is called "brain fissure".)

Consequences: incomplete data, even disaster level

In the case of a brain fissure:

Both hosts think that the other is dead, and both will read and write data to the shared storage server, so that writing to a file at the same time will result in data loss and damage to integrity.

Impact on services: making access unreachable

Under normal circumstances, web1 is working and web2 is not working. In the case of brain fissure, both hosts think that the other is dead. At this time, the web2 server will snatch the IP address of web1 and configure it for itself. Under the same network, because web1 and web2 use the same IP, it will cause network failure and users cannot access the server normally.

Solution:

Prevention first: increase redundant heartbeats

Forced quarantine:

Both servers are connected to the power switch. When web2 detects a problem with web1 and wants to take over, and in order to avoid brain fissure, web2 will send an instruction to the power switch, and the power switch will disconnect the power cable connected to web1, so that there will be no conflict between web1 and web2.

Stonish: head shot

Shoot in other node in the head

Implement with tools: ipfail

Set the reference IP for both servers (usually the gateway IP)

When two servers think there is something wrong with each other, there will be a reference IP set by ping. If the reference IP is not connected to ping, the server will decide that the problem is not caused by the other server, but that there is a problem within itself. At this time, it will detect its own interior and take corresponding actions.

Highly available cluster software:

High performance computing cluster:

A contains 10 subtasks: A1, a2, a3. A10

It takes 1 hour for each subtask to complete

Load balancer cluster: (subtasks processed per unit time)

It takes 10 hours for one computer to complete the task (10 subtasks)

It takes 10 hours for 10 computers to complete the task (a total of 100 subtasks)

High-performance computing cluster: (shorten the execution time of a single task to improve efficiency)

It takes 10 hours for one computer to complete the task (10 subtasks)

It takes 1 hour for 10 computers to complete the task (a total of 10 subtasks)

High-performance cluster concept:

High-performance computing cluster is to split a large computing task, each node calculates part of the operation content, and finally summarizes the processing results of each computer to get the answer we want. this mode of operation is high-performance computing cluster.

3PB data needs to be processed, and AGENT will tell the client how to distribute 100G to each storage server. After that, the client will find AGENT2 and ask AGENT2 how to handle the distributed data, and AGNET2 will run on each node separately.

The program processes the data by operation, and when the data is processed, it will summarize the data and feed it back to the client. This process is called high-performance computing cluster.

Load scheduler:

Hardware: F5 citrix array

Software:

Layer 4 load scheduler layer 4 switch

High efficiency

LVS Linux virtual server

Layer 7 load dispatcher layer 7 switch

Mysql proxy

Nginx

Haproxy

Advanced featur

Layer 4 switch: IP+ port

Layer 7 switch: advanced features contained in URL or hostname or page content

Difference:

1. Trigger conditions are different

Layer 4: works in the transport layer, forwarding data depends on layer 3 ip and layer 4 port

Seventh floor: working in the application layer. Forwarding data depends on URL or hostname

two。 The principle of implementation is different.

Layer 4 load scheduler: TCP connection is established once, between client and RS host

How layer 4 switches work: (forwarding)

Layer 7 load scheduler: TCP connection is established twice, client and load scheduler; load scheduler and RS host

How the layer 7 switch works: (acts as a proxy server)

Layer 7 switch can make a judgment according to the access request initiated by the customer and give the corresponding content feedback to the customer!

3. Different application scenarios

Four-tier load scheduler: TCP application is the main OA ERP

Seven-layer load scheduler: based on HTTP protocol

4. Security is different.

Layer 4 load Scheduler: forwarding *

Seven-layer load scheduler: intercept *

IPVS: hook function, kernel mechanism, function that captures and acquires priority control before a request reaches the destination address.

IPVSADM: works in user space and is responsible for writing rules for the ipvs kernel framework that defines who is the cluster service and who is the real server at the back end

LVS Overview:

First of all, let's briefly introduce what LVS (Linux Virtual Server) is. In fact, it is a Cluster technology, using IP load balancing technology and content-based request distribution technology. The scheduler has a good throughput, it transfers requests to different servers evenly, and the scheduler automatically shields the failure of the server, thus forming a group of servers into a high-performance and highly available virtual server. The structure of the whole server cluster is transparent to customers, and there is no need to modify client-side and server-side programs.

For example:

Explanation: when the external network client submits the request to the external network card of the load reader, the network card will process the request and submit the request to the INPUT function module provided by the kernel. Normally, this function will forward the customer's application request for access to local port 80 to the HTTP server installed by the local application, but our real goal is If you want the load scheduler to forward the user's request to the backend web site node, we need IPVS,IPVS to play the role of hook function here (normally, the accepted request will be forwarded to the corresponding local service, but the hook function role played by IPVS will force the data request packet text to come over before the request is forwarded to the local service, and check it for yourself. If it is found that the user is accessing the cluster node, it will go through the eth2 port, because IPVS will save certain policies and can judge by itself, and then the request will be forwarded to web1 accordingly, the second time to web2, and polling in turn. )

At this point, we will wonder why the IPVS kernel module knows to forward the request to web1 and web2 respectively, because there is an IPVSadm tool to implement it.

CIP:client IP client IP

DIP:director IP load scheduler internal network port IP

VIP:virtual IP Virtual IP Cluster IP

RIP:realserver IP server node IP real server IP

NAT: address translation

Note:

1. A maximum of 10 RS are supported.

two。 Cluster nodes (director and RS) must be in the same network

The gateway for 3.RS must be between the client and the RS

4.RS can use any operating system

5.RS gateway must be set to director

DR: route pattern

Advantages:

Greatly reduces the pressure on the load scheduler

Can support more server nodes RS

Note:

1.RS must be on the same network as director

2.RS can set public network IP or private network IP (when the load scheduler is broken, you can directly allow customers to access the cluster through public network IP)

3. The load scheduler is only responsible for inbound requests

4. Gateways of cluster nodes cannot use director

TUN: tunnel mode

Advantages:

Not restricted by geographical location

Note:

1. All nodes must have a public network IP,RS and must be configured on RIP

2.RS must not use the load scheduler as the default gateway

4. Tunneling must be supported

Load scheduler algorithm:

Static algorithm: only consider the algorithm itself, not the server state

Rr (round robin): stateless scheduling algorithm from 1 to n (number of N=RS), regardless of server performance

Wrr (weighted Round Robin): schedule according to the proportion of weights. The greater the weight, the more requests are responsible (weight = priority).

Sh (source address hash): source address hash or source address hash, which implements session binding and retains previously established session information. Send requests from the same ip address to a real server.

Dh (destination address hash): sends a request for the same destination address to the same server node. Improve cache hit ratio

Dynamic algorithm: consider not only the algorithm itself, but also the server state (principle: record the connection status through the hash table-active/inactive)

LC (minimum connections): assigns new connection requests to the server with the least number of connections currently. Formula: active connection * 256 + inactive connection

WLC (weighted least join): a special mode of least connection. Formula: (active connection * 256 + inactive connection) / weight

SED (minimum expected delay): a special mode with weighted minimum connections. Formula: (active connection + 1) * 256 / weight

NQ (never queue): a special mode of sed. When a real server has a connection of 0, it is allocated directly and is not counted.

LBLC (locality-based minimum connections): a special mode of dh that not only increases the cache hit ratio, but also considers the number of connections. First find all the servers used according to the target IP address of the request. If the server is still available and has the ability to process the request, the scheduler will try to choose the same server, otherwise it will continue to choose other feasible servers.

LBLCR (locality-based minimum connections with replication): LBLCR=LBLC+ cache sharing mechanism

Experiment: building a load cluster based on NAT pattern

Topology presentation:

Background: five virtual machines are required. The IP in the picture is used for demonstration. The following operations are different.

Configure NFS on a shared storage server

Install the Apache service on RS1 and RS2 respectively

RS1's web page by mounting the home page on the NFS server

Mount-t nfs 192.168.115.179:/share / var/www/html/

Failed after mounting and starting the Apache service

Chcon-R-h-t httpd_sys_content_t / var/www/html/

It'll be all right after that.

The gateway of RS1 points to the private network port IP of the load scheduler

Route add default gw 192.168.115.175

The web pages of RS2 are created by yourself, and the home pages of the two services are different, so it is easy to distinguish between them.

The gateway of RS2 points to the private network port IP of the load scheduler

Load scheduler: (dual network card)

Enable routing forwarding function

And perform a sysctl-p refresh

Install IPVSadm softwar

Check whether ipvsadm is installed successfully

Configuration

Public network client verification: (the page will appear every time it is refreshed)

Load cluster based on DR mode:

Principle and topology display:

The lab topology shows:

The IP address on the figure is for reference only:

How ARP works:

Arp_ignore: the response level after receiving an ARP request from another host

If the following parameter is set to 0:

Respond as long as the local configuration has a responsive IP address

If the following parameter is set to 1:

Only the destination address that responds to the request is configured on the network interface where the request arrives

Arp_announce: sets the level of ARP advertisements

0: advertise any address information for all interfaces on the network

2: advertise only the matching network information to the target network

Configuration:

Load scheduler:

Configure the ifcfg0:1 mapping subport with the address of cluster IP

Optimize environment configuration: (/ etc/sysctl.conf)

Net.ipv4.conf.all.send_redirects = 0

Net.ipv4.conf.default.send_redirects = 0

Net.ipv4.conf.eth0.send_redirects = 0

Sysctl-p

Query whether IPVS is installed:

Grep-I "ip_vs" / boot/config-2.6.32-431.el6.x86_64

Install ipvsadm

Rpm-ivh ipvsadm-1.26-2.el6.x86_64.rpm

Set up an increased load scheduler. Polling is used by default.

Ipvsadm-A-t 192.168.115.210 80-s rr

Set to use DR mode

Ipvsadm-a-t 192.168.115.210 80-r 192.168.115.178V 80-g

Ipvsadm-a-t 192.168.115.210VRO 80-r 192.168.115.173RV 80-g

Ipvsadm-L-n

Service ipvsadm save

Shared Storage Server:

RS1 configuration:

Configure the local loopback interface IP

Optimize environment variables (/ etc/sysctl.conf)

Net.ipv4.conf.all.arp_ignore = 1

Net.ipv4.conf.all.arp_announce = 2

Net.ipv4.conf.default.arp_ignore = 1

Net.ipv4.conf.default.arp_announce = 2

Net.ipv4.conf.lo.arp_ignore = 1

Net.ipv4.conf.lo.arp_announce = 2

Sysctl-p

Install the Apache service and hang the contents of the home page on the NFS server

Mount-t nfs 192.168.115.179:/share / var/www/html/

Set a route so that any request for information related to the cluster address (115.210) will be handed over to the local loopback interface.

Route add-host 192.168.115.210 dev lo:0

RS2 configuration:

Configure the local loopback mapping port:

Optimize the environment:

Net.ipv4.conf.all.arp_ignore = 1

Net.ipv4.conf.all.arp_announce = 2

Net.ipv4.conf.default.arp_ignore = 1

Net.ipv4.conf.default.arp_announce = 2

Net.ipv4.conf.lo.arp_ignore = 1

Net.ipv4.conf.lo.arp_announce = 2

Sysctl-p

Install the Apache service and manually create a home page to distinguish the content in order to achieve the experimental results.

Set a route so that any request for information related to the cluster address (115.210) will be handed over to the local loopback interface.

Route add-host 192.168.115.210 dev lo:0

Client verification: directly access the IP of the cluster, and refresh will poll the display page in turn.

The detailed meaning of the ipvsadm command parameter options is as follows:

-A (--add-service) adds a new virtual IP record to the list of virtual servers in the kernel. That is to add a new virtual server. The virtual IP is the IP address of the virtual server.

-E (--edit-service) edits a virtual server record in the kernel virtual server list

-D (--delete-service) deletes a virtual server record from the kernel virtual server list

-C (--clear) clears all rules in the kernel virtual server list

-R (--restore) restore virtual server rules

-S (--save) saves virtual server rules and outputs them in a readable format with the-R option

-a (--add-server) adds a new Real Server record to a record in the kernel virtual server list. That is to add a new Real Server to a virtual server

-e (--edit-server) edits a Real Server record in a virtual server record

-d (--delete-server) deletes a Real Server record from a virtual server record

-L |-l-list displays the list of virtual servers in the kernel

-Z (--zero) the virtual server list counter is cleared (clear the current number of connections, etc.)

-- set tcp tcpfin udp sets connection timeout value

-t indicates that the virtual server provides tcp service, and this option is followed by the following format:

[virtual-service-address:port] or [real-server-ip:port]

-u indicates that the virtual server provides udp service, and this option is followed by the following format:

[virtual-service-address:port] or [real-server-ip:port]

-f fwmark description is a service type marked by iptables

-s this option is followed by the scheduling algorithm used by LVS

There are several options: rr | wrr | lc | wlc | lblc | lblcr | dh | sh

The default scheduling algorithm is: wlc

-p [timeout] the duration of service on a Real Server. That is, multiple requests from the same user will be processed by the same Real Server. This parameter is generally used in operations with dynamic requests, and the default value of timeout is 360min. For example:-p 600, indicating that the duration of service is 600 minutes.

-r specifies the IP address of Real Server, followed by the following format:

[real-server-ip:port]

-g (--gatewaying) specifies that the operating mode of LVS is direct routing mode (this mode is the default operating mode of LVS)

-I (- ipip) specifies that the operating mode of LVS is tunnel mode

-m (--masquerading) specifies that the operating mode of LVS is NAT mode

-w (--weight) weight specifies the weight of the Real Server

-c (--connection) displays the current connection information of LVS, such as ipvsadm-L-c

-L-- timeout displays the timeout value of "tcp tcpfin udp", such as: ipvsadm-L-- timeout

-L-- daemon displays the status of the synchronous daemon, for example: ipvsadm-L-daemon

-L-- stats displays statistics, for example: ipvsadm-L-stats

-L-- rate displays rate information, for example: ipvsadm-L-- rate

-L-- sort sorts the output of virtual servers and real servers, for example: ipvsadm-L-- sort

Note: the rules for saving added virtual ip records and ipvsadm can use service ipvsadm save, or-S or-- save. Clear all records and rules except using-C to use-- clear.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.