Shulou(Shulou.com)06/03 Report--

[background]

The matter of ipv6 has been put on the agenda again this year, and the Ministry of Industry and Information Technology has been pushing it. I wonder if everyone has started it. The request we have received here is as follows:

At first, it just said that Q4 only needed to ensure that the main business functions in the test environment could operate smoothly in the ipv6 environment, but suddenly it turned into at least 10% of the online traffic to the ipv6 environment.

During this period of time have been messing with ipv6 things, various environmental deployment joint adjustment and so on, read a lot of information, but the online information about this piece is really too little, during the deployment of the environment has stepped on a lot of holes, here is a simple record.

[deployment]

Lb-01

Lb-02

Rs-01

Rs-02

Vip

Ipv4

10.1.1.111

10.1.1.112

10.1.1.113

10.1.1.114

10.21.5.7

Ipv6

240e:45e:1111:1ff::1

240e:45e:1111:1ff::2

240e:45e:1111:1ff::3

240e:45e:1111:1ff::4

240e:97d:1111:2ff::2

1. Ospf

1. Turn on ospf6d process listening

Compared with the original ipv4 in ipv6 environment, you need to open ospf6d extra.

Root@node-01:/etc/quagga# cat daemonszebra=yesbgpd=noospfd=yesospf6d=yesripd=noripngd=noisisd=no

2. Configure ospf

Root@node-01:/etc/quagga# cat / etc/quagga/ospf6d.conf!! Zebra configuration saved from vty! 2019-11-21 11:55:20!hostname ospf6dpassword zebralog stdoutlog file / var/log/quagga/ospf6d.log!!interface eth0ipv6 ospf6 hello-interval 1ipv6 ospf6 dead-interval 3ipv6 ospf6 priority 0ipv6 ospf6 instance-id 0interface loipv6 ospf6 hello-interval 1ipv6 ospf6 dead-interval 3ipv6 ospf6 priority 0ipv6 ospf6 instance-id 0router ospf6 router-id 10.1.1.111 interface eth0 area 0.0.0.0 interface lo area 0.0.0.0!line vty

The following is the ospf configuration of ipv4. You can compare the differences between configurations.

!! Zebra configuration saved from vty! 2019-10-15 16:51:09!hostname ospfdpassword zebralog stdoutlog file / var/log/quagga/ospf.log!!interface eth0!ip ospf hello-interval 1ip ospf dead-interval 3ip ospf priority 0interface eth2!ip ospf hello-interval 1ip ospf dead-interval 3ip ospf priority 0!router ospfospf router-id 10.1.1.111network 10.21.5.7/32 area 0.0.0.0network 10.1.1.0/24 area 0.0.0.0!line vty

Note:

Log file recommends setting an additional log path to print separately from the original ipv4 ospfd log to facilitate troubleshooting in the future.

Change router ospf configuration to router ospf6

Ospf process id generally uses machine IP (guaranteed to be unique)

Another biggest difference is that the machine IP IP address range and VIP do not need to be declared in router ospf. Ipv6 only needs to specify which interfaces need to be declared.

3. Start quagga

Root@node-01:/etc/quagga# / etc/init.d/quagga restart [ok] Restarting quagga (via systemctl): quagga.service.

After startup, you will see that there is a multi-watch ospf6d process

Root@node-01:/etc/quagga# ps aux | grep quaggaquagga 25820 0.00.0 24496? Ss 15:15 0:00 / usr/lib/quagga/zebra-- daemon-A 127.0.0.1quagga 25824 0.0 26980 2732? Ss 15:15 0:00 / usr/lib/quagga/ospfd-- daemon-A 127.0.0.1quagga 25828 0.0 24556 628? Ss 15:15 0:00 / usr/lib/quagga/ospf6d-- daemon-A:: 1root 25833 0.0 15428? Ss 15:15 0:00 / usr/lib/quagga/watchquagga-- daemon zebra ospfd ospf6d

Telnet local 2606 port

Root@node-01:/etc/quagga# telnet:: 1 2606Trying:: 1...Connected to:: 1.Escape character is'^] '.Hello, this is Quagga (version 0.99.24.1). Copyright 1996-2005 Kunihiro Ishiguro, et al.User Access Verificationospf6d > show ipv6 ospf6 neighborNeighbor ID Pri DeadTime State/IfState Duration I / F [State] 10.1.1.1 255 00:00:02 Full/DR 00:00:09 eth0 [DROther]

Note: there is a hole here. Because we are running LVS in TUNNEL mode, we need to set the MTU to small. After we get up the ospf, we find that we cannot establish a neighbor with the switch. The switch side log shows that the switch interface MTU needs to be set to the same value (1440 in our environment).

Under the ipv4 environment, the switch side does not need to set up MTU.

4. Configure VIP

Start ipv6 vip on LB

There are two ways to configure: (IPV6-related operation commands are at the end of the article)

Method 1:

Root@node01:/etc/quagga# ip addr add 240E:97D:1111:2FF::2/64 dev lo:vip1 label lo:vip1

Method 2:

Root@node01:/etc/quagga# / sbin/ifconfig lo:vip3 inet6 add 240E:97D:1111:2FF::2/64

5. Test ipv6 vip connectivity

Root@ubuntu:/usr/local/named/etc# ping6 240e:97d:1111:2ff::2-c 3PING 240e:97d:1111:2ff::2 (240e:97d:1111:2ff::2) 56 data bytes64 bytes from 240e:97d:1111:2ff::2: icmp_seq=1 ttl=51 time=28.4 ms64 bytes from 240e:97d:1111:2ff::2: icmp_seq=2 ttl=51 time=28.4 ms64 bytes from 240e:97d:1111:2ff::2: icmp_seq=3 ttl=51 time=28.3 ms

Note:

The machine tested must also have an ipv6 address, otherwise it will return that the network is unreachable

2. Configure LVS

1. Compile and install a newer version of keepalived (here I compiled keepalived-2.0.18)

Note: it is recommended to compile in ubuntu16.04 or above. Some packages of ubuntu cannot be found in lower versions such as 12.04 and 14.04.

Root@ubuntu:/usr/local/src/keepalived-2.0.18# apt-get install libnftnl-dev libmnl-dev

Root@ubuntu:/usr/local/src/keepalived-2.0.18# apt-get install iptables-dev libipset-dev libnl-3-dev libnl-genl-3-dev libssl-dev

Root@ubuntu:/usr/local/src/keepalived-2.0.18#. / configure-- prefix=/usr/local/keepalived

Root@ubuntu:/usr/local/src/keepalived-2.0.18# make & & make install

2. Keepalived configuration

Method 1: command line

Root@node-01:/etc/quagga# ipvsadm-A-t [240e:97d:2014:1ff::2]: 80-s rrroot@node-01:/etc/quagga# ipvsadm-a-t [240e:97d:2014:1ff::2]: 80-r 10.21.41.43 rrroot@node-01:/etc/quagga# ipvsadm 80-iroot@node-01:/etc/quagga# ipvsadm-a-t [240e:97d:2014:1ff::2]: 80-r 10.21.41.44 rrroot@node-01:/etc/quagga# ipvsadm 80-I

Method 2:

Virtual_server 240e:97d:1111:2ff::2 80 {delay_loop 6 lb_algo wrr lb_kind TUN persistence_timeout 0 protocol TCP real_server 240e:45e:1111:1ff::3 80 {weight 10 TCP_CHECK {connect_port 80 connect_timeout 8 nb_get_retry 3 delay_before_retry 3}} real _ server 240e:45e:1111:1ff::4 80 {weight 10 TCP_CHECK {connect_port 80 connect_timeout 8 nb_get_retry 3 delay_before_retry 3}

Root@node-01:/etc/quagga# ipvsadm-lnIP Virtual Server version 1.2.1 (size=1048576) Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP [240e:97d:1111:2ff::2]: 80 rr-> [240e:45e:1111:1ff::3]: 80 Tunnel 100-> [240e:45e:1111:1ff::4]: 80 Tunnel 100 0

3. Configure RS VIP

In LVS tunnel mode, two things need to be done on the RS: one is to start VIP, and the other is to establish a TUN tunnel with LB.

Root@node-03:~ # ip-6 tunnel add lvs6tun0 mode ip6ip6 local 240e:45e:1111:1ff::3 remote 240e:45e:1111:1ff::2 dev eth0root@node-03:~ # ip link set dev lvs6tun0 uproot@node-03:~ # ip-6 addr add 240e:97d:1111:2ff::2/64 dev lvs6tun0

Create ipv6 tunnel command description. Since it is a peer-to-peer tunnel, you need to specify that local is the local RSIP,remote peer IP, that is, the IP of LB

Ip-6 tunnel add lvs6tun0 mode ip6ip6 local $rs-ip remote $lb-ip dev $interface

Peer-to-peer, as the name implies, means that peer,n LB nodes, m RS nodes and n * m tunnel nodes need to be established between each node.

The big difference between ipv6 and ipv4 is that ipv6 does not have a broadcast address and cannot build an one-to-many tunnel, so it can only build a peer-to-peer tunnel based on ip6ip6 protocol. The following is how tunnel is created in ipv4 environment, which can be compared:

/ sbin/ifconfig tunl0 $vip broadcast $vip netmask 255.255.255.255 up/sbin/route add-host $vip dev tunl0

4. Service testing

On a machine that has ipv6 (which cannot be a machine in the above cluster), try to access it through ipv6 vip

Root@ubuntu:~ # for i in {0.999}; do nc-6-v-w 1 240e:97d:1111:2ff::2 80 DoneConnection to 240e:97d:1111:2ff::2 80 port [tcp/http] succeeededconnection to 240e:97d:1111:2ff::2 80 port [tcp/http] succeeded connection to 240e:97d:1111:2ff::2 80 port [tcp/http] succeededconnection... root@ubuntu:~ # curl http://[240e:97d:1111:2ff::2]/-H "Host:ipv6-test.aaa.com" Test Page

At this point, the test environment can communicate normally.

In addition, the LVS NAT/DR mode has also been verified during the test, which can be natively supported. It is not as troublesome as TUNNEL, such as DR mode. Just configure a VIP on RS. DR is not adopted because there is a restriction that LB and RS must be on the same network segment in DR mode. Our production environment cannot guarantee that LB and RS must be in the same network segment. And NAT performance is poor, FullNAT also has part of the business in use, the performance is not particularly good, rely on the nf_conntrack table, and finally choose the TUNNEL mode.

Note:

In the process of deployment, the configuration of ipv6 tunnel is the most laborious, and it has been tried in a variety of ways, but not in any way. The phenomenon is that LB sends SYN packets to RS, but does not receive ACK, and the connection on LB is in a SYNC_RECV state (the connection status can be seen through ipvsadm-lnc)

Finally, through the above peer-to-peer tunnel solution, for the time being, there is no other more suitable way, later found and then updated.

The above is only a basic available test environment, and all kinds of performance and parameter tuning are needed to go online.

Ipv6 actually involves a lot of things, not only the above mentioned, but also DNS, GSLB, business, network, CDN and other levels will involve related transformation.

Attached:

1. IPV6 environment testing website

For a pure ipv6 environment, please use: http://ipv6.test-ipv6.com

For dual stack environment, please use: http://www.test-ipv6.com/

2. Ipv6 related instructions

Http://tldp.org/HOWTO/Linux+IPv6-HOWTO/ch06s02.html

Http://tldp.org/HOWTO/Linux+IPv6-HOWTO/ch07s02.html

Http://tldp.org/HOWTO/Linux+IPv6-HOWTO/ch04s03.html

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.