Shulou(Shulou.com)06/01 Report--

Demand put forward

We know that the native Managed network of Rancher establishes a secure tunnel through IPsec to ensure the reachability between service. Compared with some other × × technologies, the biggest advantage of IPSec lies in its security; this security not only refers to the encrypted transmission of business data, but also includes a set of security mechanism for secret key exchange, which is unmatched by other L3 × × technologies such as VXLAN. However, in order to provide this layer of security mechanism, IPSec needs to encrypt the messages one by one in the process of data exchange, which is bound to consume CPU resources and affect the transmission efficiency.

We have done some tests: compared with the iperf test of the Host network, the performance loss of the Managed network is more serious (of course, this is also related to the configuration of the host running Rancher). Unfortunately, in the current official Release version of Rancher, there is no other Overlay network available as an option except the Managed network.

The reality is that some users have self-built data centers, Rancher has been deployed inside the data center, and network security has been relatively guaranteed. Affected by the nature of business, some scenarios have higher requirements for network transmission efficiency. At this point, the Managed network seems to be unable to cover the requirements. Therefore, we consider whether we can make some modifications to Rancher's Managed network without considering security, so as to improve the efficiency of network transmission.

A more direct idea is to remove the encryption process of business messages.

Basic concepts of IPSec

This section is not going to explain the principle of IPSec in detail, but will only point to some important concepts so far. Interested students can come down and study the RFC of IETF.

Let's first use a diagram to understand the IPSec framework:

IPSec mode

Transmission mode: suitable for data protection between two hosts.

Tunnel mode: suitable for establishing a secure tunnel for site-2-site. (the Rancher Managed network is clearly intended to open an encrypted tunnel for all Containers on Host, using tunnel mode.)

IPSec security protocol

Security protocols including AH, ESP and their combined AH-ESP,Rancher use ESP. Through the following figure, we can vividly understand the relationship between IPSec patterns and security protocols:

In addition, some students who have grasped the packet will find that the IPSec message of Rancher is somewhat different from the packet format based on Tunnel mode + ESP security protocol here. The actual captured message contains a UDP header using port 4500 after the outer IP header and in front of the ESP domain. This is the NAT-T function, its implementation mechanism is to first determine whether the devices at both ends support NAT-T, and then detect whether there is NAT on the link, start NAT-T once the two conditions are met, and transmit all business messages using port 4500 of UDP.

Protect privacy

IPSec supports the use of multiple encryption algorithms to encrypt transmitted business data. Encrypt the data from plaintext to unreadable ciphertext, so as to ensure the privacy of the data.

Encryption algorithms are divided into symmetric encryption algorithms (DES/AES/3DES, etc.) and asymmetric encryption algorithms (such as RSA)

Guarantee integrity

Through the HASH operation of the data, a data summary similar to fingerprint is generated and transmitted to the opposite end together with the data to confirm that the data has not been illegally tampered with and ensure the integrity of the data.

Common HASH algorithms include MD5/SHA and so on.

Guarantee authenticity

Both symmetric encryption and HASH require that both sides of the communication have the same key, so a set of key exchange algorithm is needed to transfer the key safely between them.

Identity authentication can ensure the authenticity of the data and ensure that the data is indeed sent by a specific peer.

Common authentication methods include: Pre-shared key pre-shared key, RSA Signature digital signature and so on.

The following figure shows the ESP packet flow in tunnel mode, which reflects how encryption and authentication algorithms are used to generate the final encrypted message.

IKE

In addition, IPSec also includes IKE. IKE is a security mechanism, which provides dynamic authentication between end to end, provides services for IPsec to automatically negotiate and exchange keys and establish SA, and simplifies the use, management (configuration and maintenance) of IPsec. IKE does not transmit the key directly on the network, but calculates the shared key through the exchange of a series of data. With IKE,IPsec, many parameters (such as keys) can be established automatically, reducing the complexity of manual configuration.

IKE uses UDP port 500 for secret key exchange by using a specific key exchange algorithm. The common algorithms are DH and RSA.

After all this theoretical stuff, let's go back to Rancher and see how Rancher's Managed network works.

Principle of Rancher Managed network

By going to the agent-instance container to view the process information, you can find that the following processes are started:

Rancher-metadata

Rancher-dns

Rancher-net

Charon

Haproxy

Host-api

Rancher-metadata launches a web server in response to the metadata request of the containers under the jurisdiction of agent.

Rancher-dns implements a skydns instance listening on the IP address: 169.254.169.250, which is used to respond to the dns request of the container under the jurisdiction of agent.

Haproxy is actually a load balancer that comes with healthcheck for the specified port of member. Here, it is mainly used to check the health of containers configured with healthcheck.

Host-api is used in the container to report the data collected by haproxy to cattle.

Let's focus on rancher-net and charon:

The configuration of rancher-net is mainly based on the file "/ var/lib/cattle/etc/cattle/ipsec/config.json", which contains Container address information for all Managed Network within Environment.

The specific format is as follows:

Rancher-net maintains the link and policy of the IPSec by initiating charon and sending configurations to and from the xfrm.

Charon is used to implement the negotiation between IKE and send rule to xfrm. In the next chapter, the modification of IPSec encryption method will be realized by calling the API of charon.

In addition to sending xfrm state through charon, rancher-net also needs to configure xfrm policy. The implementation of this block is implemented by directly executing "ip xfrm policy add" in rancher-net.

In addition, there is a command line interface swanctl that can view the status of strongswan:

In addition to IPSec, rancher-net needs to listen to all ARP requests and respond to ARP requests whose destination IP address is reachable through IPSec tunnel but not in this Host.

To sum up, the features of rancher-net include:

Listen on port 8111 to respond to HTT requests from reload and ping.

When there is a reload request, read the configuration file and verify the paths to all IP in the configuration file:

a. If a new host is added, add an ipsec tunnel

b. If there is a new container, but IPSec already exists, update the xfrm policy

c. If there is a deletion, do the opposite.

Listen for ARP requests on eth0 and respond to ARP requests that need to reach the IP of the IPSec peer.

The topology of the entire IPSec tunnel is as follows:

Managed network practice

Having said so much, you have all been bewildered. Do not say much, modify the code link solemnly on the stage.

The modification of the code is relatively simple, as long as the first encryption method of proposals is changed to null.

For example, the representative of "null-sha1-modp2048":

Use symmetric encryption for null (no encryption)

HASH algorithm using sha1

Use modp2048's asymmetric encryption algorithm for secret key exchange.

Then recompile the code using Dockerfile.dapper to make sure there are no compilation errors.

We know that all packages on agent are pulled from cattle at startup; therefore, it is useless to replace rancher-net by updating docker p_w_picpath.

In order to test, we first directly replace the rancher-net in the agent-instance with the compiled rancher-net executable file through docker cp.

Replace and run, found that the charon negotiation IKE could not succeed, prompting "ENCRYPTION_ALGORITHM NULL (key size 20) not supported!". This is because we set the encryption algorithm to NULL, which is not supported by charon in the system.

The query official website found that NULL needs the support of openssl in IKE, which may be due to the fact that rancher did not specify the compilation openssl plugin when compiling charon:

The modification method is to specify parameters when charon compiles. For more information on enable-openssl, please see [https://lists.strongswan.org/pipermail/dev/2011-February/000253.html]].

The analysis code shows that agent-instance gets charon by installing the package generated by "rancher/strongswan-package" during initialization. So, go to rancher/strongswan-package to reassign. / configure-- enable-openssl to compile charon.

After generating a new executable file, execute "ip xfrm state flush" and "ip xfrm policy flush" first, and then restart rancher-net and charon;. If you do not wipe out policy and state first, the newly added ipsec will not take effect.

After modification, the environment is as follows:

The two Network Agent containers are the endpoints of IPSec, and in addition, there is an Iperf container running on each host for testing.

Through swanctl-- list-algs, you can see that all the supported encryption algorithms, of which we have load openssl, have been able to support NULL.

Check the ipsec rules in xfrm. Encryption is already ciper_null, that is, no encryption.

The policy in xfrm lists the strategy to reach the peer iperf container, that is, take IPSec tunnel.

Cross-host testing

The iperf container on two hosts, one as an iperf server and the other as a client for bandwidth testing.

During the test, you need to grab the package to confirm whether the modification has taken effect, as follows:

As you can see, ESP sequence is followed by 0X45 0X00, which is obviously an IP header of RAW.

To confirm this information, we decode out the src IP and dst IP in the IP header, which are:

0x0A 0X2A 0XF4 0XFA and 0x0A 0x2A 0xB4 0xF9

0x0A 0X2A 0XF4 0XFA = 10.42.224.250

0x0A 0x2A 0xB4 0xF9 = 10.42.180.249

These two IP addresses are the IP addresses of our iperf server and client container; therefore, we can judge that the message is not encrypted at this time; our modification has taken effect.

Before Chan

After Chan

After 5-minute iperf test on the servers of two gigabit network cards, the bandwidth of the native Managed network using Rancher is 600+Mbps. After changing the encryption method to NULL, the bandwidth of iperf test is increased to 800+Mbps.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.