Shulou(Shulou.com)06/01 Report--

It is believed that many inexperienced people have no idea about how to automatically switch network parameters in heartbeat. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.

I. Environment

Node0

Eth0:192.168.32.30 service vip1:192.168.32.21 VMAC1:00:1E:68:1E:99:5E

Eth2:10.160.100.30 heart RMAC:00:18:8B:8E:04:E2

Node1

Eth0:192.168.32.31 service vip1:192.168.32.21 VMAC1:00:1E:68:1E:99:5E

Eth2:10.160.100.31 heart vip2:192.168.32.22 VMAC2:00:1E:68:1E:88:5E RMAC:00:1E:68:1E:19:5E

Node2

Eth0:192.168.32.32 service vip2:192.168.32.22 VMAC2:00:1E:68:1E:88:5E

Eth2:10.160.100.32 heart RMAC:00:1E:68:1E:16:7E

Experimental requirements:

1. When the node0 network fails, node1 automatically switches over its own ip, hostname, mac addresses to vip1, vman1, nodevir1;, node0 automatically switches back after the failure is restored, and changes its ip, hostname, mac to vip1, vman1, nodevir,node1 to restore its own network parameters.

2. When the network of node2 fails, node1 automatically switches over its own ip, hostname, mac addresses to vip2, vman2, nodevir2;, node0 automatically switches back after the failure is restored, and changes its ip, hostname, mac to vip2, vman2, nodevir2,node1 to restore its own network parameters.

2. Heartbeat installation (the operation of the three node is the same)

1. Install libnet

[root@node1 ~] # tar-zxvf libnet-1.1.4.tar.gz

[root@node1 ~] # cd libnet-1.1.4

[root@node1 libnet-1.1.4] #. / configure

[root@node1 libnet-1.1.4] # make;make install

# libnet is a high-level API tool, which is needed to use heartbeat

2. Install heartbeat

[root@node1 ~] # tar-zxvf heartbeat-2.1.3.tar.gz

[root@node1 ~] # cd heartbeat-2.1.3

[root@node1 heartbeat-2.1.3] # / ConfigureMe configure-- disable-swig-- disable-snmp-subagent

[root@node1 heartbeat-2.1.3] # make; make install

[root@node1 heartbeat-2.1.3] # cp doc/ha.cf doc/haresource doc/anthkeys / etc/ha.d/

[root@node1 heartbeat-2.1.3] # cp ldirectord/ldiretord.cf / etc/ha.d/

[root@node1 heartbeat-2.1.3] # grouadd-g 694 haclient

[root@node1 heartbeat-2.1.3] # useradd-u 694-g 694 hacluster

The ldirectord plug-in is included by default in the installation package of # heartbeat, which is mainly used to manage load balancers and install perl-Mailtools to ensure the availability of ldirectord.

III. Configuration

1 node0 configuration

1.1 create a network parameter modification script (/ etc/ha.d/resource.d/Remac1)

[root@node0 resource.d] # vim / etc/ha.d/resource.d/Remac1 # Remac1 can also be stored in the / etc/rc.d/init.d directory

#! / bin/bash

# discription: Start Real Server

VMAC=00:1E:68:1E:99:5E

RMAC=00:1E:68:1E:19:5E

GATEWAY=192.168.32.254

. / etc/rc.d/init.d/functions

Case "$1" in

Start)

Echo "Replace eth0's Real Mac addr to Virtual Mac addr"

/ sbin/ifconfig eth0 down

/ sbin/ifconfig eth0 hw ether $VMAC

/ sbin/ifconfig eth0 up

/ sbin/ifconfig eth0 192.168.32.21 broadcast 192.168.32.255 netmask 255.255.255.0

/ bin/hostname nodevir

/ sbin/route add default gw $GATEWAY

Stop)

Echo "Restore eth0's Real Mac addr"

/ sbin/ifconfig eth0 down

/ sbin/ifconfig eth0 hw ether $RMAC

/ sbin/ifconfig eth0 up

/ sbin/ifconfig eth0 192.168.32.31 broadcast 192.168.32.255 netmask 255.255.255.0

/ bin/hostname node1

/ sbin/route add default gw $GATEWAY

*)

Echo "Usage: $0 {start | stop}"

Exit 1

Esac

1.2 heartbeat main configuration file (/ etc/ha.d/ha.cf)

[root@node1 ha.d] # vim / etc/ha.d/ha.cf

# debugfile / var/log/ha-debug

Logfile / var/log/ha-log

# specify the location where the log is stored under the name heartbeat.

# crm yes

# whether to enable Cluster Resource Manager (Cluster Resource Management) function.

Bcast eth2

# indicates that the heartbeat is broadcast over Ethernet and is broadcast on the eth2 interface.

Keepalive 2

# specifies that the heartbeat interval is 2 seconds (that is, a broadcast is sent on the eth2 every two seconds).

Deadtime 60

# specifies that after the standby node does not receive the heartbeat signal of the primary node within 30 seconds, it will immediately take over the service resources of the primary node.

Warntime 20

# specifies that the heartbeat delay is ten seconds. When the backup node cannot receive the heartbeat signal of the primary node within 10 seconds, a warning log is written to the log, but the service is not switched at this time.

Initdead 240

# on some systems, it takes some time for the network to work properly after the system is started or rebooted. This option is used to resolve the time interval caused by this situation. The value is at least twice that of deadtime.

Udpport 694 # sets the port used for broadcast communication, which is the default port number.

# baud 19200 # sets the baud rate for serial communication.

# serial / dev/ttyS0 # Select a serial communication device for dual computers connected by a serial cable. If both computers use Ethernet.

# ucast eth0 192.168.1.2 # the udp unicast of the network card eth2 is used to organize the heartbeat, followed by the IP address of each other's IP address.

# mcast eth0 225.0.0.169410 # Udp Multicast of the network card eth0 is used to organize the heartbeat, which is generally used when there is more than one backup node. Bcast, ucast and mcast represent broadcast, unicast and multicast, respectively, and are the three ways to organize the heartbeat.

Auto_failback on

# is used to define whether the service will be automatically switched back when the primary node is restored. The two hosts of heartbeat are the primary node and the backup node respectively. Under normal circumstances, the primary node occupies resources and runs all the services. in case of failure, the resources are handed over to the backup node and the service is run by the backup node. When this option is set to on, once the primary node resumes operation, it automatically acquires resources and replaces the backup node. If this option is set to off, then when the primary node is restored, it becomes the backup node, and the original backup node becomes the primary node.

# stonith baytech / etc/ha.d/conf/stonith.baytech

The main function of # stonith is to detach the nodes with problems from the cluster environment, thereby releasing cluster resources and avoiding the occurrence of two nodes competing for one resource. Ensure the security and integrity of shared data.

# watchdog / dev/watchdog

# this option is optional and uses Heartbeat to monitor the running status of the system. To use this feature, you need to load the "softdog" kernel module into the kernel to generate the actual device files.

If you do not have this kernel module in the system, you need to specify this module and recompile the kernel. After compilation, enter "insmod softdog" to load the module. Then enter "grep misc / proc/devices" (which should be 10) and "cat / proc/misc | grep watchdog" (which should be 130). Finally, the device file is generated: "mknod / dev/watchdog c 10130". You can use this feature.

Node node0 # host node hostname, which can be viewed with the command "uanme-n".

Node node1 # alternate node hostname.

Ping 192.168.32.254

# choose the node of ping. The better the ping node, the stronger the HA cluster. You can choose a fixed router as the ping node, but it is best not to select the members of the cluster as the ping node. The ping node is only used to test the network connection.

Ping_group group1 192.168.12.251 192.168.12.239 # is similar to ping.

# respawn hacluster / usr/local/ha/lib/heartbeat/ipfail

# apiauth pingd gid=haclient uid=hacluster

Apiauth pingd gid=root uid=root

Respawn hacluster / usr/local/ha/lib/heartbeat/pingd-m 100-d 5s

# this option is an optional configuration that lists the processes started and shut down with heartbeat, which are typically plug-ins integrated with heartbeat and can be automatically restarted if they fail. The most commonly used process is pingd, which is used to detect and monitor the status of the network card and needs to cooperate with the ping node specified in the ping statement to check the connectivity of the network. Where hacluster indicates the identity of starting the pingd process.

# the following configuration is the key, that is, to activate crm management and start using v2 style format

Crm respawn

# Note, you can also use crm yes, but if you write it this way, if there is a problem with the later cib.xml configuration

# will cause heartbeat to restart the server directly, so it is recommended to use respawn when testing

# the following is the compression of the transmitted data, which is optional

Compression bz2

Compression_threshold 2

# Note: v2 style does not support ipfail, so pingd must be used instead.

1.3Resource files (/ etc/ha.d/haresources)

[root@node0 ha.d] # vim / etc/ha.d/haresources

Node0 Remac1

# haresources file is used to specify cluster resources such as master node, cluster IP, subnet mask, broadcast address and started service of a dual-machine system. Each line of the file can contain one or more resource script names, separated by spaces between resources, and parameters separated by two colons. The file must be identical on two HA nodes. The general format of this file is as follows:

Node-name network

# node-name refers to the hostname of the master node, which must be the same as the node name specified in the ha.cf file. Network is used to set the IP address, subnet mask, network setting identity and so on of the cluster. It should be noted that the IP address specified here is the IP address of the external services of the cluster, and resource-group is used to specify the services that need to be hosted by heartbeat, that is, these services can be started and shut down by heartbeat. If you want to host these services, The service must be written as a step that can be started and shut down through start/stop, and then placed in the / etc/init.d/ or / etc/ha.d/resource.d/ directory. Heartbeat will automatically find the appropriate steps to start or shut down in the / etc/init.d or / etc/ha.d/resource.d/ directory according to the name of the script.

# LSB: Linux standard script file (init script), usually placed in the / etc/init.d/ directory, half of the administrative scripts before the heartbeat1.x version are placed in / etc/ha.d/resource.d, in this case / usr/local/ha/etc/ha.d/resource.d

OCF:Open Cluster Framework, which is placed in the / usr/lib/resource.d/heartbeat/ directory by default; in this case / usr/local/ha/etc/ha.d/resource.d

# the following describes the difference between ocf and lsb formats:

The script in LSB format must support status function and must be able to receive start,stop,status with three parameters, while if it is in OCF format, it must support start,stop,monitor with three parameters. Among them, status and monitor parameters are used to monitor resources, which is very important.

For example, for LSB-style scripts, when running. / Mysql status, the return value contains OK or running indicates that the resource is normal, and the return value contains stopped or No indicates that the resource is abnormal. If it is an OCF-style script, when running. / Mysql monitor, 0 indicates that the resource is normal, and 7 indicates that there is a problem with the resource.

# the configuration method is described below:

Node1 IPaddr::192.168.60.200/24/eth0/ Filesystem::/dev/sdb5::/webdata::ext3 httpd tomcat

# where node1 is the master node of the HA cluster, and IPaddr is an execution step that comes with heartbeat. Heartbeat will first perform the operation of / etc/ha.d/resource.d/IPaddr 192.168.60.200 to 24 start, that is, a virtual address with a subnet mask of 255.255.255.255.0 and an IP of 192.168.60.200. This IP is the network address where heartbeat provides services, and specifies the network interface used by this IP as eth0. Heartbeat will mount the shared disk partition, and "Filesystem::/dev/sdf1::/data1::ext3" is equivalent to performing a mount operation under the command line, that is, "mount-t ext3 / dev/sdf1 / data1", and finally start the httpd and tomcat services in turn.

1.4 Certification documents (/ etc/ha.d/authkeys)

[root@node0 resource.d] # vim / etc/ha.d/authkeys

Auth 1

1 crc

# 2 sha1 HI!

# 3 md5 Hello!

# authkeys file is used to set the authentication mode of heartbeat. There are three available authentication methods: crc, md5 and sha1. The security of the three authentication methods is improved in turn, but the system resources occupied are also increased in turn. If the heartbeat cluster runs on a secure network, you can use crc. If the hardware configuration of each node in HA is very high, it is recommended to use sha1. This authentication method has the highest security level. If it is between network security and system resources, you can use md5 authentication. Here we use the crc authentication method, which is set as follows:

Auth 1

1 crc

# 2 sha1 sha1_any_password

# 3 md5 md5_any_password

# it is important to note that no matter what number is specified after auth, it must reappear as a keyword on the next line, for example, if "auth 6" is specified, there must be a line "6 authentication type" below. Finally, make sure that the permissions for this file are 600 (that is,-rw-).

2 node1 configuration

2.1 create a network parameter modification script (/ etc/ha.d/resource.d/Remac1&2)

[root@node1 ~] # vim / etc/ha.d/resource.d/Remac1 # backup node0 network parameter modification script

#! / bin/bash

# discription: Start Real Server

VMAC=00:1E:68:1E:99:5E

RMAC=00:1E:68:1E:19:5E # Note: different from RMAC in Remac1 in node0

GATEWAY=192.168.32.254

. / etc/rc.d/init.d/functions

Case "$1" in

Start)

Echo "Replace eth0's Real Mac addr to Virtual Mac addr"

/ sbin/ifconfig eth0 down

/ sbin/ifconfig eth0 hw ether $VMAC

/ sbin/ifconfig eth0 up

/ sbin/ifconfig eth0 192.168.32.21 broadcast 192.168.32.255 netmask 255.255.255.0

/ bin/hostname nodevir

/ sbin/route add default gw $GATEWAY

Stop)

Echo "Restore eth0's Real Mac addr"

/ sbin/ifconfig eth0 down

/ sbin/ifconfig eth0 hw ether $RMAC

/ sbin/ifconfig eth0 up

/ sbin/ifconfig eth0 192.168.32.31 broadcast 192.168.32.255 netmask 255.255.255.0

/ bin/hostname node1

/ sbin/route add default gw $GATEWAY

*)

Echo "Usage: $0 {start | stop}"

Exit 1

Esac

[root@node1 ~] # cat / etc/ha.d/resource.d/Remac2 # backup node2 network parameter modification script

#! / bin/bash

# discription: Start Real Server

VMAC=00:1E:68:1E:88:5E

RMAC=00:1E:68:1E:19:5E # Note: different from RMAC in Remac1 in node

GATEWAY=192.168.32.254

. / etc/rc.d/init.d/functions

Case "$1" in

Start)

Echo "Replace eth0's Real Mac addr to Virtual Mac addr"

/ sbin/ifconfig eth0 down

/ sbin/ifconfig eth0 hw ether $VMAC

/ sbin/ifconfig eth0 up

/ sbin/ifconfig eth0 192.168.32.22 broadcast 192.168.32.255 netmask 255.255.255.0

/ bin/hostname nodevir2

/ sbin/route add default gw $GATEWAY

Stop)

Echo "Restore eth0's Real Mac addr"

/ sbin/ifconfig eth0 down

/ sbin/ifconfig eth0 hw ether $RMAC

/ sbin/ifconfig eth0 up

/ sbin/ifconfig eth0 192.168.32.31 broadcast 192.168.32.255 netmask 255.255.255.0

/ bin/hostname node1

/ sbin/route add default gw $GATEWAY

*)

Echo "Usage: $0 {start | stop}"

Exit 1

Esac

2.2 main configuration file (/ etc/ha.d/ha.cf)

[root@node1 ~] # vim / etc/ha.d/ha.cf

Debugfile / var/log/ha-debug

Logfile / var/log/ha-log

Logfacility local0

Keepalive 2

Deadtime 60

Warntime 20

Initdead 240

Udpport 694

Auto_failback on

Node node0 # main node0, the higher the order, the higher the priority.

Node node2 # Master node2

Node node1 # backup node1

Ping 192.168.32.254

Respawn root / usr/lib/heartbeat/ipfail

Apiauth ipfail gid=root uid=root

Deadping 60

# Note: since it takes a lot of time to restart the Nic to modify the network parameters, all the time parameters are set here for a long time.

2.3 Resource file (/ etc/ha.d/haresource)

[root@node1 ~] # cat / etc/ha.d/haresources

Node0 Remac1

Node2 Remac2

# backup nod0 with Remac1 script, backup node2 with Remac2 script

2.4Certification documents (/ etc/ha.d/authkeys)

Same as node0

3 node2 configuration

3.1 create a script for modifying network parameters (/ etc/ha.d/resource.d/Remac2)

[root@node2 ~] # cat / etc/ha.d/resource.d/Remac2

#! / bin/bash

# discription: Start Real Server

VMAC=00:1E:68:1E:88:5E

RMAC=00:1E:68:1E:16:7E

GATEWAY=192.168.32.254

. / etc/rc.d/init.d/functions

Case "$1" in

Start)

Echo "Replace eth0's Real Mac addr to Virtual Mac addr"

/ sbin/ifconfig eth0 down

/ sbin/ifconfig eth0 hw ether $VMAC

/ sbin/ifconfig eth0 up

/ sbin/ifconfig eth0 192.168.32.22 broadcast 192.168.32.255 netmask 255.255.255.0

/ bin/hostname nodevir2

/ sbin/route add default gw $GATEWAY

Stop)

Echo "Restore eth0's Real Mac addr"

/ sbin/ifconfig eth0 down

/ sbin/ifconfig eth0 hw ether $RMAC

/ sbin/ifconfig eth0 up

/ sbin/ifconfig eth0 192.168.32.32 broadcast 192.168.32.255 netmask 255.255.255.0

/ bin/hostname node2

/ sbin/route add default gw $GATEWAY

*)

Echo "Usage: $0 {start | stop}"

Exit 1

Esac

3.2Master configuration file (/ etc/ha.d/ha.cf)

[root@node1 ~] # vim / etc/ha.d/ha.cf

Debugfile / var/log/ha-debug

Logfile / var/log/ha-log

Logfacility local0

Keepalive 2

Deadtime 60

Warntime 20

Initdead 240

Udpport 694

Auto_failback on

Node node2 # main node2, the higher the order, the higher the priority.

Node node1 # backup node1

Ping 192.168.32.254

Respawn root / usr/lib/heartbeat/ipfail

Apiauth ipfail gid=root uid=root

Deadping 60

# Note: since it takes a lot of time to restart the Nic to modify the network parameters, all the time parameters are set here for a long time.

3.3Resource files (/ etc/ha.d/haresource)

[root@node1 ~] # cat / etc/ha.d/haresources

Node2 Remac2

3.4 Certification document (/ etc/ha.d/authkeys)

Same as node0

4. Test the high availability function of heartbeat

How to know whether the HA cluster is working properly? simulation environment testing is a good method. Before putting the Heartbeat high availability cluster into the production environment, you need to do the following steps to determine whether the HA is working properly:

1 shut down and restart the heartbeat of the primary node normally

First, execute "service heartbeat stop" on the primary node node1 to shut down the Heartbeat process of the primary node normally. At this time, check the network card information of the primary node through the ifconfig command. Normally, you should see that the primary node has released the service IP address of the cluster, as well as the mounted shared disk partition, and then check the backup node. Now the backup node has taken over the service IP of the cluster. At the same time, the shared disk partition is automatically mounted.

In this process, using the ping command to test the cluster service IP, you can see that the cluster IP is uniformly reachable without any delay and blocking, that is, when the master node is shut down normally, the switching between the master and slave nodes is seamless, and the services provided by HA can run uninterrupted.

Next, the primary node heartbeat is started normally, and after heartbeat is started, the backup node will automatically release the cluster service IP and uninstall the shared disk partition, while the primary node will once again take over the cluster service IP and mount the shared disk partition. In fact, the backup node releases resources at the same time as the primary node binds resources. Therefore, this process is also a seamless switch.

2 unplug the network cable from the primary node

After unplugging the network cable from the primary node to connect to the public network, the heartbeat plug-in ipfail can immediately detect the network connection failure through the ping test, and then automatically release resources, and at this time, the ipfail plug-in of the standby node will also detect the network failure of the primary node. After waiting for the primary node to release resources, the standby node immediately takes over the cluster resources, thus ensuring the continuous operation of network services.

Similarly, when the primary node network returns to normal, the cluster resource will automatically switch to the primary node from the standby node because the "auto_failback on" option is set.

3 shut down the system of the primary node

After the primary node is unplugged, the heartbeat process of the standby node will immediately receive the message that the primary node has shutdown, and the standby node will begin to take over the resources, which is actually similar to the network failure of the primary node.

4 crash the kernel of the master node system

When the primary node system crashes, the network also loses its response, then the heartbeat process of the standby node will immediately detect the primary node network failure and then switch resources, but because the kernel of the primary node system crashes, it can not unload the resources it owns, such as shared disk partition, cluster service IP, etc., so if there are no similar Stonith devices, there will be resource contention However, if there is a Stonith standby, the Stonith device will first power off or restart the failed master node, thus allowing the master node to release the cluster resources. when the Stonith device completes all operations, the backup node will take over the ownership of the master node resources, thus taking over the master node resources.

After reading the above, have you mastered the method of how to automatically switch network parameters in heartbeat? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.