Shulou(Shulou.com)06/01 Report--

It is believed that many inexperienced people do not know what to do about the working principle and how to realize the Linux bridge in the foundation of Docker network. therefore, this paper summarizes the causes and solutions of the problem. I hope you can solve this problem through this article.

This article is reproduced from the official account "Linux Kernel things" on Wechat, author songsong001. To reprint this article, please contact the official account of the Linux kernel.

The Linux bridge is a virtual device (implemented by software) that connects multiple network interfaces within the Linux, as shown in the following figure:

The result of connecting network interfaces is that when one network interface receives a network packet, it is copied to other network interfaces, as shown in the following figure:

As shown in the figure above, when network interface A receives the packet, the bridge copies and sends the packet to other network interfaces connected to the bridge (such as network card B and network card C in the figure above).

Docker uses bridges to communicate between containers. Let's take a look at how Docker uses bridges to communicate between containers. The principle is shown below:

When Docker starts up, it creates a bridge called docker0 and sets its IP address to 172.17.0.1 Universe 16 (private IP address). Then use the virtual device pair veth-pair to connect the container to the bridge, as shown in the figure above. For packets of 172.17.0.0 Docker with 16 network segments, Docker defines a rule for iptables NAT to translate the IP addresses of these packets into public network IP addresses, and then send them out through the real network interface (such as the ens160 interface in the figure above).

Next, we mainly analyze the implementation of the bridge through the code.

Realization of Network Bridge

1. Creation of Brid

We can add a bridge device object named br0 with the following command:

[root@vagrant] # brctl addbr br0

Then, we can view a list of all the bridge devices in the system by using the command brctl show, as follows:

[root@vagrant] # brctl show bridge name bridge id STP enabled interfaces br0 8000.000000000000 no docker0 8000.000000000000 no

When a new bridge device is created with the command, the kernel is triggered to call the br_add_bridge () function, which is implemented as follows:

Int br_add_bridge (char * name) {struct net_bridge * br; if ((br = new_nb (name)) = = NULL) / / create a bridge device object return-ENOMEM; if (_ _ dev_get_by_name (name)! = NULL) {/ / has the device name been registered? Kfree (br); return-EEXIST; / / returned an error and cannot re-register devices with the same name} / / add to the bridge list br- > next = bridge_list; bridge_list = br;. Register_netdev (& br- > dev); / / register the bridge with the network device return 0;}

The br_add_bridge () function mainly accomplishes the following tasks:

Call the new_nb () function to create a bridge device object.

Call the _ _ dev_get_by_name () function to check whether the device name has been registered, and return an error message if so.

Bridge device objects are added to the bridge_list linked list, and the kernel uses the bridge_list linked list to hold all bridge devices.

Call register_netdev () to register the bridge device with the network device.

As you can see from the above code, the bridge device is described using the net_bridge structure, which is defined as follows:

Struct net_bridge {struct net_bridge * next; / / Connect all bridge objects in the kernel rwlock_t lock; / / lock struct net_bridge_port * port_list; / / bridge port list struct net_device dev / / Bridge device information struct net_device_stats statistics; / / Information statistics rwlock_t hash_lock; / / used to lock CAM table struct net_bridge_fdb_entry * hash [Br _ HASH_SIZE]; / / CAM table struct timer_list tick; / * STP * /.}

In the net_bridge structure, the more important fields are port_list and hash:

Port_list: a list of bridge ports that holds a list of network interfaces bound to the bridge.

Hash: holds a hash table with the MAC address of the network interface as the key and the bridge port as the value.

The bridge port is described using the structure net_bridge_port, which is defined as follows:

Struct net_bridge_port {struct net_bridge_port * next; / / points to the next port struct net_bridge * br; / / Bridge device object struct net_device * dev; / / Network Interface device object int port_no; / / Port number / * STP * /.}

The net_bridge_fdb_entry structure is used to describe the corresponding relationship between the MAC address of the network interface device and the bridge port, which is defined as follows:

Struct net_bridge_fdb_entry {struct net_bridge_fdb_entry * next_hash; struct net_bridge_fdb_entry * * pprev_hash; atomic_t use_count; mac_addr addr; / / Network Interface device MAC address struct net_bridge_port * dst; / / Bridge Port.}

The correspondence of these three structures is shown in the following figure:

It can be seen that to bind a network interface device to a bridge, you need to use the net_bridge_port structure to associate it. Let's analyze how to bind a network interface device to a bridge.

The bridge works at the second layer of the TCP/IP stack, that is, the bridge can broadcast or unicast packets based on the destination MAC address. When the destination MAC address can find the corresponding bridge port in the hash table of the bridge, it indicates that the packet is unicast, otherwise it is broadcast.

two。 Bind the network interface to the bridge

To bind a network interface device to a bridge, use the following command:

[root@vagrant] # brctl addif br0 eth0

The above command binds the network interface eth0 to the bridge br0.

When the command is called to bind the network interface device to the bridge, the kernel triggers a call to the br_add_if () function, with the following code:

Int br_add_if (struct net_bridge * br, struct net_device * dev) {struct net_bridge_port * p;... Write_lock_bh (& br- > lock); / / create a new bridge port object and add it to the bridge's port_list linked list if ((p = new_nbp (br, dev)) = = NULL) {write_unlock_bh (& br- > lock); dev_put (dev); return-EXFULL } / / set the network interface device to promiscuous mode dev_set_promiscuity (dev, 1); add br_fdb_insert (br, p, dev- > dev_addr, 1) to the hash table of the network interface MAC address corresponding to the bridge port. Write_unlock_bh (& br- > lock); return 0;}

The br_add_if () function mainly does the following:

Call the new_nbp () function to create a new bridge port and add it to the bridge's port_list list.

Set the network interface device to promiscuous mode.

Call the br_fdb_insert () function to insert the newly created bridge port into the hash table corresponding to the network interface MAC address.

In other words, the br_add_if () function mainly establishes the relationship between the network interface device and the bridge.

3. The network interface in the bridge receives data

When a network interface receives a packet, it determines whether the network interface is bound to a bridge, and if so, calls the handle_bridge () function to process the packet. The handle_bridge () function is implemented as follows:

Static int _ _ inline__ handle_bridge (struct sk_buff * skb, struct packet_type * pt_prev) {int ret = NET_RX_DROP;... Br_handle_frame_hook (skb); return ret;}

Br_handle_frame_hook is a function pointer that points to the br_handle_frame () function. Let's analyze the implementation of the br_handle_frame () function:

Void br_handle_frame (struct sk_buff * skb) {struct net_bridge * br; br = skb- > dev- > br_port- > br; / / gets the bridge object read_lock (& br- > lock) connected by the device; / / locks _ _ br_handle_frame (skb) on a pair of bridges; / / calls _ _ br_handle_frame () function to process packet read_unlock (& br- > lock);}

The implementation of the br_handle_frame () function is relatively simple. First, lock the bridge, and then call _ _ br_handle_frame () to process the data packet. Let's analyze the implementation of the _ _ br_handle_frame () function:

Static void _ _ br_handle_frame (struct sk_buff * skb) {struct net_bridge * br; unsigned char * dest; struct net_bridge_fdb_entry * dst; struct net_bridge_port * p; int passedup; dest = skb- > mac.ethernet- > html destination; / / destination MAC address p = skb- > dev- > br_port; / / Port bound to the network interface br = p-> br; passedup = 0 Insert the learned MAC address into the hash table of the bridge if (p-> state = = BR_STATE_LEARNING | | p-> state = = BR_STATE_FORWARDING) br_fdb_insert (br, p, skb- > mac.ethernet- > h_source, 0);. If (dest [0] & 1) {/ / if it is a broadcast packet br_flood (br, skb, 1); / / send the packet to all network interfaces connected to the bridge if (! passedup) br_pass_frame_up (br, skb); else kfree_skb (skb); return } dst = br_fdb_get (br, dest); / / get the bridge port corresponding to the destination MAC address. If (dst! = NULL) {/ / if there is a br_forward (dst- > dst, skb) in the bridge port corresponding to the destination MAC address; / / then only forward the packet to this port br_fdb_put (dst); return;} br_flood (br, skb, 0); / / otherwise send to all network interfaces return connected to this bridge ...}

The _ _ br_handle_frame () function mainly accomplishes the following tasks:

First insert the MAC address learned from the packet into the hash table of the bridge.

If the packet is a broadcast packet (the first bit of the destination MAC address is 1), the br_flood () function is called to send the packet to all network interfaces connected to the bridge.

Call br_fdb_get () to get the bridge port corresponding to the target MAC address, and if the bridge port corresponding to the destination MAC address exists, then call the br_forward () function to forward the packet to this port.

Otherwise, the br_flood () function is called to send the packet to all network interfaces connected to the bridge.

The function br_forward () is used to send the packet to the specified bridge port, which is implemented as follows:

Static void _ br_forward (struct net_bridge_port * to, struct sk_buff * skb) {skb- > dev = to- > dev; dev_queue_xmit (skb);} void br_forward (struct net_bridge_port * to, struct sk_buff * skb) {if (should_forward (to, skb)) {/ / Port can receive data? _ _ br_forward (to, skb); return } kfree_skb (skb);}

The br_forward () function sends data to the specified bridge port by calling the _ _ br_forward () function. The _ _ br_forward () function first sets the output interface device of the packet to the device bound by the bridge port, and then calls the dev_queue_xmit () function to send the packet out.

The br_flood () function is used to send packets to all network interface devices bound to the bridge, which is implemented as follows:

Void br_flood (struct net_bridge * br, struct sk_buff * skb, int clone) {struct net_bridge_port * p; struct net_bridge_port * prev;. Prev = NULL; p = br- > port_list; while (p! = NULL) {/ / traverse all network interface devices if (should_forward (p, skb)) {/ / ports bound to the bridge can receive packets? If (prev! = NULL) {struct sk_buff * skb2; / / Clone a packet if ((skb2 = skb_clone (skb, GFP_ATOMIC)) = = NULL) {br- > statistics.tx_dropped++; kfree_skb (skb); return } _ br_forward (prev, skb2); / / send the packet to the device} prev = p;} p = p-> next;} if (prev! = NULL) {_ br_forward (prev, skb); return;} kfree_skb (skb) }

The implementation of the br_flood () function is also relatively simple, mainly traversing all the network interface devices bound to the bridge, and then calling the _ _ br_forward () function to forward the packet to the corresponding port of the device.

After reading the above, have you mastered the working principle of Docker bridge in Linux network foundation and how to realize it? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.