Network Security Internet Technology Development Database Servers Mobile Phone Android Software Apple Software Computer Software News IT Information

In addition to Weibo, there is also WeChat

Please pay attention

WeChat public account

Shulou

Lvs+srs (dr model)

2025-01-18 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Servers >

Share

Shulou(Shulou.com)06/03 Report--

Reference:

Https://my.oschina.net/fengjihu/blog/416883 (srs Cluster Mode)

Http://blog.csdn.net/reblue520/article/details/50857078 (lvs load balancing)

Http://www.cnblogs.com/lgfeng/archive/2012/10/16/2726308.html (arp_ignore and arp_announce)

Http://www.linuxvirtualserver.org/docs/arp.html (lvs arp problem)

Http://www.linuxvirtualserver.org/VS-DRouting.html (lvs dr mode)

Objective:

Push streams to multiple streaming servers through lvs to relieve server pressure. All streams can be accessed by using the feature of edge server, and bandwidth can be saved. Because it is live broadcasting, the pushed streams are all of different ip, so you can use the function of lvs hash ip to forward the same first-class packets to the same first-class server, thus solving the problem of stream storage.

Introduction to the principle of lvs dr mode:

In dr mode, director, real server and vip are all in the same network segment, which brings several problems:

1. In this mode, the request received by vip cannot be simply forwarded to real server, because when the request is simply forwarded, because all three servers can be connected to the public network, the real server will reply directly to the requester when it is forwarded, and the requester will discard the request when it receives the ip and finds that it is not its own request.

Therefore, each real server is bound with vip, and when receiving a request packet from vip, real server replies with vip as the source address.

two。 We have solved the problem of 1, but this brings a new problem: both director and realserver have vip, and they are in the same network segment, so when the vip request arrives, the gateway will broadcast the request, so whether it is director or realserver, they will receive the request. Suppose our director receives it first, how does it schedule the load balance?

It is not possible to use ip. If you use the real ip of realserver, you will go back to the problem of 1, so you can only use vip as the destination ip. In fact, in dr mode, lvs schedules load balancers based on mac addresses. Because the mac address of each server is different, only the rs (realserver abbreviation, too lazy to type) of the corresponding mac will receive the request, and can respond to the request because it does have vip.

3. So how do we make sure director is the first to receive it? In the same network segment, each server will receive a vip request broadcast packet. How can we ensure that rs can respond to these requests without first responding? This requires the use of arp_ignore and arp_announce or hidden interface functions.

In dr mode, the configuration of rs is something special. One of the more common is to bind vip to the lo port, because most linux kernel versions do not respond to the arp request of the loop ip, so when the vip broadcast request message is received, the rs will not respond. Because director binds vip to a non-loop, director can be the first to receive and respond to the vip request message. Another common practice is that rs binds vip to an interface that is not in use, and then uses the function of hidden interface to disable the arp response of this interface. Because there is no arp response, this port must not be used, otherwise normal communication will be affected.

Our case uses the third common method: set arp_ignore to 1, which answers only ARP query requests where the destination IP address is the local address of the visiting network interface

For example, suppose that the ip bound on eth0 is 1.1.1.1 and the binding on eth2 is 2.2.2.2, for example, if arp_ignore is set to 1, eth0 will not respond when it receives the arp request message of 2.2.2.2. Similarly, eth2 will not respond to the arp request message of 1.1.1.1 (eth0:0 subnet card is not sure whether it is possible, to be tested)

Then, set arp_announce to 2, which uses the mac of the egress port as the source mac.

For example, suppose the native machine has eth0:1.1.1.1,eth2:2.2.2.2, and if the machine receives the request packet of 1.1.1.1 but sends the packet from the eth2 port, should the source mac of the data frame be the mac of writing eth0 or the mac of eth2? Under normal circumstances is to write eth2, but linux is quite special, the default is to write eth0, set to 2, is to let it write eth2 by default.

Why do you need it here? Because director is the first to respond to the VIP packet, the arp table of the gateway retains the mac of director. Assuming that rs,eth0 matches the real ip,eth2 with vip, then, when director rewrites the vip request packet of mac, it responds and sends back the packet from eth0 (we should not let eth2 receive the packet, otherwise it goes back to 2). If eth2 is filled in at this time, that is, the mac where vip is located, the gateway will update the arp table when it finds that this is vip's new mac. Resulting in vip preemption (this is someone else's view, I don't quite agree, because even the mac using eth0 will be updated, just as preemptive, until I find a reliable answer to update this question)

Architecture:

Director ip:192.168.2.119 (lvs Scheduler Server ip)

Virtual ip:192.168.2.111 (virtual ip for scheduling)

Real1 ip:192.168.2.127 (srs source server ip1)

Real2 ip:192.168.2.138 (srs source server ip2)

Srs ip:192.168.2.221 (this is srs Edge Server ip)

For the convenience of experiment, iptables and selinux are turned off by default

Real1:

Echo 1 > / proc/sys/net/ipv4/conf/eth0/arp_ignore

Echo 1 > / proc/sys/net/ipv4/conf/all/arp_ignore

Echo 2 > / proc/sys/net/ipv4/conf/eth0/arp_announce

Echo 2 > / proc/sys/net/ipv4/conf/all/arp_announce

Cd / etc/sysconfig/network-scripts/

Cp ifcfg-lo ifcfg-lo:0

The vi ifcfg-lo:0 content is as follows:

DEVICE=lo:0

IPADDR=192.168.2.111

NETMASK=255.255.255.255

BROADCAST=192.168.2.111

BROADCAST=127.255.255.255

ONBOOT=yes

NAME=loopback

Service network restart

The vi / usr/local/srs/conf/edge.conf content is as follows:

Listen 1935

Max_connections 1000

Pid objs/edge.pid

Srs_log_file. / objs/edge.log

Vhost _ _ defaultVhost__ {

}

Service srs restart

Route add-host 192.168.2.111 dev lo:0 # this is added to ensure that the source ip of the native response package is vip.

Real2:

Configuration is the same as real1, slightly

Configuration of srs Edge Server:

Vi / usr/local/srs/conf/edge.conf

Listen 1935

Max_connections 1000

Pid objs/edge.pid

Srs_log_file. / objs/edge.log

Vhost _ _ defaultVhost__ {

Mode remote

Origin 192.168.2.138Viru 1935 192.168.2.127 origin 1935; (tested that when there are different streams between 192.168.2.138 and 192.168.2.127

} with this configuration, the edge server can access all streams on both servers)

Service srs restart

Configuration of direct:

Vi / etc/sysctl.conf

Net.ipv4.ip_forward = 1 (lvs forwarding packet, so this should be 1, enable forwarding function)

Sysctl-p

Yum install ipvsadm-y (lvs is managed through the ipvsadm package, similar to the relationship between netfiter and iptables)

Cd / etc/sysconfig/network-scripts/

Cp ifcfg-eth0 ifcfg-eth0:0

The vi ifcfg-eth0:0 content is as follows:

DEVICE=eth0:0

BOOTPROTO=static

HWADDR=00:0C:29:5B:58:99

ONBOOT=yes

IPADDR=192.168.2.111

GATEWAY=192.168.2.1

NETMASK=255.255.255.0

Service network restart

Route add 192.168.2.111 dev eth0:0

Ipvsadm-A-t 192.168.2.111sh 1935-s sh (add service,-s specify scheduling algorithm, which means source_hash, dispatched to the same server according to source ip)

Ipvsadm-a-t 192.168.2.111VR 1935-r 192.168.2.138VR 1935-m (add back-end server)

Ipvsadm-a-t 192.168.2.111v 1935-r 192.168.2.127v 1935-m

Ipvsadm-Ln (this command can view the current status of ipvs)

Push test:

/ usr/local/src/srs/trunk/objs/ffmpeg/bin/ffmpeg-re-I test.flv-vcodec copy-acodec copy-f flv-y rtmp://192.168.2.111/live/livestream1

/ usr/local/src/srs/trunk/objs/ffmpeg/bin/ffmpeg-re-I test2.flv-vcodec copy-acodec copy-f flv-y rtmp://192.168.2.111/live/livestream2

Check rtmp://192.168.2.221/live/livestream1 and rtmp://192.168.2.221/live/livestream2 to confirm that the video can be played normally.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.

Views: 0

*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.

Share To

Servers

Wechat

© 2024 shulou.com SLNews company. All rights reserved.

12
Report