Shulou(Shulou.com)06/02 Report--

What is the network model and working mechanism of the OpenStack cloud platform? in order to solve this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible method.

The network is the most problematic part of the deployment of OpenStack, and it is also a difficult part of its structure. I often receive e-mails about OneStack deployment network issues and OpenStack network structure issues. According to my own understanding, let's talk about the virtual network, network topology and network flow of OpenStack. Personal understanding is limited, just throw a brick to attract jade, please correct any questions, thank you.

1. Related concepts and key points (can be skipped)

1. The role of nova-network in OpenStack

There are two types of physical nodes in the OpenStack platform, the control node and the computing node. Control nodes include network control, scheduling management, api services, storage volume management, database management, identity management and mirror management. Computing nodes mainly provide nova-compute services. The service of the control node can be separated in multiple nodes, and the node providing nova-network service is called the network controller.

OpenStack's network is managed by nova-network (Network Controller), which creates virtual networks that allow hosts to access each other and external networks.

OpenStack's API server distributes commands provided by nova-network through message queues, which are then processed by nova-network. The main operations are: assigning ip addresses, configuring virtual networks, and communicating.

Distinguish between the following two concepts: control node and network controller

In the simplest case, all services are deployed on a single host, which is all-in-one

A little more complicated, all services except nova-compute are deployed on one host, which carries out various kinds of control and management, so it is the control node (in this article, the deployment of 2 or more nodes is called "multi-node").

However, in many cases (for example, for high availability), various management services need to be deployed on different hosts (such as database cluster services, message queues, mirror management, network control, etc.). At this point, the network controller (running nova-network) is only part of the cluster of control nodes.

2. 2 kinds of ip and 3 kinds of management modes of network in OpenStack

Nova has the concept of fixed IP and floating IP. The fixed IP is distributed to the created instance, and the floating IP is some IP address that can be dynamically bound and released to the instance.

Nova supports three types of networks, corresponding to three types of "network management": Flat management mode, FlatDHCP management mode and VLAN management mode. The VLAN model is used by default.

These three types of network management modes can coexist in a single deployment of penStack, can be different in different nodes, and can be configured to achieve high availability.

These three management modes are briefly introduced and analyzed in detail later.

Flat (flat): all instances are connected to the same virtual network, and the bridge needs to be set up manually.

FlatDHCP: similar to the Flat (flat) management model, in which all instances of the network are bridged to the same virtual network, a flat topology. The difference is that, as with the name difference, the ip of the instance provides dhcp acquisition (the nova-network node provides dhcp services) and can automatically help build the bridge.

VLAN: provide a protected network segment (virtual LAN) for each project.

Second, the working mechanism of the three network modes

Flat mode

1) specify a subnet that specifies the range of ip used by the virtual machine, that is, an ip pool (

The allocation of ip will not exceed this range, that is, the fixed_range in the configuration, such as 10.0.0.1 ip 27, then there are 32 available CPUs.

This network can be changed, for example, after configuring the nodes nova.conf and interfaces, nova-manage network delete 10.0.0.1According to 27 1 32 management network create192.168.1.0/24 1 255.

2) when creating an instance, take an IP from the valid ip address pool, assign it to the virtual machine instance, and then inject the virtual machine image (file system) when the virtual machine starts.

3) the bridge (br100) must be manually configured, and all system instances are connected to the same bridge; the bridge and the instances connected to the bridge form a virtual network, and the node where the nova-network is located is the default gateway. For example, the ip of flat_interface=eth2;eth2 is 10.0.0.1, and the ip of other networks is within 10.0.0.1. Flat interface-- > br100-- > flat network

4) after that, the network controller (nova-network node) performs NAT conversion to the virtual machine instance to realize the communication with the outside.

Note: at present, it seems that configuration injection can only work for operating system instances of Linux type, and the network configuration is saved in the / etc/network/interfaces file.

Flat DHCP mode

As in Flat mode, the ip is taken from the ip pool and assigned to the virtual machine instance, all of which are associated with a bridge in the compute node. However, in this mode, the control node does more configuration, tries to establish a bridge with the Ethernet device (default is eth0), and automatically allocates the fixed ip of the flat network to the instance through dhcp, which can reclaim and release ip.

1) the network controller (the node running the nova-network service) runs dusmasq as the DHCP server to listen to the bridge

2) the instance performs a dhcp discover operation to send a request

3) the network controller responds to the IP address obtained from a specified subnet to the virtual machine instance

4) the instance accesses each other through the network controller and the outside.

VLAN network model

The default network management mode of OpenStack is not set-network_manager=nova.network.manager.FlatDHCPManager or FlatManager defaults to vlan. In order to install multiple machines, VLAN network mode requires a switch (switch) that supports VLAN tags (IEEE 802.1Q).

In this mode, VLAN and bridges are created for each project. All instances belonging to a project are connected to the same VLAN, and Linux bridges and VLAN are created if necessary.

Each project gets some private IP addresses that can only be accessed from within VLAN, that is, private network segments. Each project has its own VLAN,Linux bridge and subnet. Subnets specified by the network administrator are dynamically assigned to a project when needed.

1) the DHCP server on the network controller is started by all VLAN, and the IP address is obtained from the subnet assigned to the project and transferred to the virtual machine instance.

2) in order for users to obtain instances of the project and access the private network segment, you need to create a special VPN instance (code name is cloudpipe, which is used to create integer, key and vpn access instances).

3) the compute node generates certificates and key for the user, so that the user can access the VPN, and the compute node automatically starts the VPN.

4) vpn access.

Comparison between Flat and vLAN

In both Flat modes, the network controller acts as the default gateway, and the instances are assigned a common IP address (flat structure, both in a bridged network).

The vLAN pattern is rich in functionality and is suitable for internal deployment in the enterprise. However, switches supporting vLAN is needed to connect, and it is relatively complex, so FlatDHCP mode is often used in small-scale experiments.

Third, explain the FlatDHCP mode in detail (the Flat mode is similar, except that the part of dhcp is missing, so we will skip it)

There can be a variety of deployment methods, such as multiple network cards, external gateways, multi_host, and so on, in order to achieve high availability. This paper mainly introduces the basic deployment method (a control node, or a network controller).

1. Network card and node

Due to the difference of network card and the number of nodes, it can be simply divided into single node (all-in-one) single network card and multi-node multi-network card.

In the case of a single node, the network controller (running nova-network) and computing (running nova-compute, or rather, running a virtual machine instance) are deployed on the same host. In this way, there is no need for communication between the control node and the computing node, and a lot of network concepts are missing, which is also a common way for beginners.

In the case of multi-node, the network controller and the computing node are respectively under different hosts and ordinary deployment modes (not multi_host), only nova-network controls the network, and it only runs on the control node. Therefore, all instances of computing nodes need to communicate with the external network through the control node.

When a single network card is used, the network card needs to be used as the interface of the public network as well as the interface of the flat network, so it needs to be in hybrid mode. However, the established network is similar to the dual network card, which is divided into flat network and public network.

To use a single network card, you need to make both public_interface and flat_interface eth0 in nova.conf.

2. Network flow

As analyzed above, in a normal deployment mode, there is only one control node (or network controller), through which both dhcp and external network access are required.

When dhcp:

1) the network controller (the node running the nova-network service) has been running dusmasq as the DHCP server listening bridge (br100)

2) the instance performs a dhcp discover operation to send a request

3) the network controller responds to the IP address obtained from a specified subnet to the virtual machine instance.

When the instance accesses the public network:

1) the instance passes through the flat_interface of the host (this is a flat network), and connects to the host where the nova-network resides (the control node)

2) the network controller forwards the outgoing network flow.

When accessing an instance via public network:

1) the network controller nat the floating ip

2) the incoming data is routed to the corresponding instance through the flat network.

Figures 1 and 2 below can compare the network flow (traffic) of single network card and dual network card, and figures 2 and 3 can compare the network flow of single node and multi-node.

Figure 1: dual network card multi-node OpenStack network flow

Figure 3: single Nic single node OpenStack network flow

3. The working principle of multi-node control node and computing node.

Control node:

1) create a br100 on the host and assign the gateway ip to the bridge; if there is already an ip, automatically assign the ip to the bridge as the gateway and repair the gateway

2) set up dhcp server, listen to the bridge, and record the allocation and release of ip in the database to determine that the virtual machine is released and close dhcp normally.

3) when listening for an ip request, take the ip from the ip pool and send the ip to the instance in response to the ip.

4) establish iptables rules to restrict and open communication with external networks or access to other services.

Compute nodes:

1) establish a bridge (br100) of the corresponding control node on the host, and bridge the instance (virtual machine) on it to a network (the network where the br100 resides)

2) since then, the bridge, the bridge of the control node and the virtual network card of the instance are all on the same virtual network and are accessed through the control node.

It can be seen that this approach has the following characteristics:

1) all instances communicate with the public network through the network controller, which is called SPoF (single failure point)

2) the control node provides dhcp services, nat, and establishes subnets as the gateway of the virtual network.

3) A computing node may not have a public network ip. Like the instances on it, the control node can be accessed as a gateway.

4) too much communication between the instance and the public network will cause congestion or high load on the control node network.

IV. The characteristics of VLAN mode.

The purpose of the VLAN model is to provide protected network segments for each project, with the following characteristics:

Implementing public ip with NAT

There is no other way to enter every lan except public NAT

Limited outflow network, project-admin can control

Access between restricted projects, also controlled by project-admin

So the connection between the instance and api is via vpn

You know little about vLAN schemas, so borrow a diagram from the OpenStack document for comparison:

Figure 4:VLAN mode OpenStack network structure

V. Network deployment

1. Network configuration

Apt-get install bridge-utils

The purpose of installing bridge-utils is to establish a virtual bridge and realize a virtual network. OpenStack automatically creates the br100 bridge, so you don't have to create it yourself.

-- network_manager=nova.network.manager.FlatDHCPManager

Set the network management mode, generally use FlatDHCP, but also can cooperate with multi_host to achieve high availability.

# Network Configuration--dhcpbridge_flagfile=/etc/nova/nova.conf--dhcpbridge=/usr/bin/nova-dhcpbridge--flat_network_bridge=br100--flat_interface=eth2--flat_injected=False--public_interface=eth0

Dhcpbridge_flagfile specifies the configuration file, and flat_injected implements the injection of ipv6 addresses, so it is turned off.

Flat_network_bridge specifies the bridge.

Flat_interface specifies the network card, which is the host node (usually the control node) used to build bridges, bridge instances and virtual networks, as well as public networks. The single network card is set to eth0, which is the same as public.

# Block of IP addresses that are fixed IPs--fixed_range=10.0.0.1/27

Specify the scope of the ip pool, which is the ip pool that takes the ip from the specified ip pool and assigns it to the instance mentioned many times in this article.

2. High availability of the network in OpenStack (HA)

In basic network management, the network flow of all instances passes through the network controller. When there is a problem with the network controller, the network fails, and the network controller is a SPoF (single failure point). "Building High availability for OpenStack (HA,High Availability)" briefly introduces four methods and the future Quantum.

The main deployment method is FlagDHCP + multi_host:

1) install nova-network on each compute node and set multi_host to true. In this way, flat_interface on each compute node serves as a bridge, providing dhcp and dns as gateways (gateway) for all instances on it. Instances no longer pass through the control node, and problems with the control node will not affect the network.

2) the flat_interface of each computing node provides switch connection to realize the transmission and communication of virtual network between instances.

3) each computing node has a public_interface, which is connected to the external network.

4) assign floating ip to each instance as the second virtual ip of the instance to communicate with the public network.

That is, the following changes have taken place:

Figure OpenStack network flow when 5:multi_host is deployed

It is worth looking forward to the future Quantum and Melarge to provide better network services. The Quantum project implements layer 2 network-related functions, such as creating and managing virtual networks, ports, etc. Melange is responsible for layer 3 network correlation, and its main tasks are IP address management (IPAM), DHCP, NAT and even load balancing. However, because its implementation takes a certain amount of time and multiple stages, we still need to understand the various network models and deployments mentioned above.

This is the answer to the question about the network model and working mechanism of the OpenStack cloud platform. I hope the above content can be of some help to you. If you still have a lot of doubts to be solved, you can follow the industry information channel for more related knowledge.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.