Shulou(Shulou.com)06/02 Report--

1. Why Metadata Service

OpenStack Metadata Service provides configuration information for instance (collectively referred to as metadata). When instance starts, it requests from Metadata Service and gets its own metadata,instance cloud-init to complete the personalized configuration work according to metadata.

2. Metadata Service architecture

Nova-api-metadata

Nova-api-metadata is a sub-service of nova-api, and it is the actual provider of metadata. When a virtual machine is started, you can choose to obtain metadata information through the REST API of nova-api-metadata.

The nova-api-metadata service runs on the control node, service port 8775 (can be modified through the configuration of nova)

Enable the metadata service

Nova-api-metadata and nova-api services are merged, and you can specify whether to enable nova-api-metadata through the enabled_apis configuration of nova.conf

Neutron-metadata-agent

The nova-api-metadata takes the management network, and the virtual machine goes through the business network, so the virtual machine can not communicate with nova-api-metadata directly. At this time, we need to use the neutron-metadata-agent service running on the network node.

Running two components, L3 agent and dhcp agent, on the network node, they will create a haproxy process, run in their respective namespace, receive the metadata request sent by the virtual machine, and forward the request to the neutron-metadata-agent service through Unix Domain Socket, and then forwarded by neutron-metadata-agent to nova-api-metadata.

The whole process can be summarized as follows:

A, instance sends the metadata request to router or the haproxy process created by dhcp agent

B. The haproxy process sends the request to neutron-metadata-agent through Unix Domian Socket

C. Neutron-metadata sends the request to the nova-api-metadata service through the internal management network

3 、 L3 agent or DHCP agent

As mentioned in the above description, both L3 agent and dhcp agent can create haproxy processes to forward metadata requests, so how to choose the two options?

In fact, each of them has its own application scenario, and even the two schemes can coexist. The difference between the two will be analyzed in detail below.

Previously, l3-agent and dhcp-agent have different ways to access metadata.

For 169.254.169.254:

An and l3-agent are forwarded using iptables rules.

B, dhcp-agent is to configure this IP on your own interface.

4. Analysis of L3 agent implementation

Create a network test1, enable dhcp, temporarily disconnect to router, start an instance, and observe the startup log of instance:

From the startup log above, we can see that after intance obtained the ip address from dhcp, it sent a request request to 169.254.169.254, but failed 20 times.

So what on earth is 169.254.169.254?

In fact, this ip address is the ip address of metadata service, and instance sends a metadata request to it when it starts.

Now that we find out that the instance request is a failure, why?

As mentioned above, OpenStack creates a haproxy process through L3 agent or dhcp agent to forward metadata. Let's first check the haproxy process on the network node.

It is found that no haproxy process is running, which explains why the instance request failed. But what went wrong?

The root cause is that by default, the haproxy process is created by the L3 agent (dhcp agent is turned off). Since the current test1 network is not mounted on the router, no haproxy process is created.

After you create the router and mount the test1 on the router. We found that the haproxy process has been started on the network node.

Restart instance test1 to see what happens.

According to the instance log, the virtual machine has successfully obtained the metadata from 169.254.169.254.

What happened in the process?

A, instance will initiate a metadata request to port 80 of 168.254.169.254 after startup, according to the

The route to instance can find that the request is sent from the eth0 of instance to the qr- device on the router.

B. The metadata request arrives at the route, enters the PREROUTING chain, and enters the qr- port with the destination address 169.254.169.254. The request with the destination port of 80 is redirected to 9697.

C. Why 9697?

Because the haproxy process created by router is listening on port 9697

D. It's simple later: the haproxy process forwards the request to neutron-metadata-agent via Unix Domain Socket, which then forwards it to nova-api-metadata through the administrative gateway.

5. Analysis of DHCP agent implementation.

OpenStack manages the implementation of metadata through L3 agent by default, and then communicates with nova-api-metadata, but not all environments have L3 agent, such as scenarios that directly use physical router, so how can metadata be implemented?

The answer is DHCP agent.

A. Turn on the metadata function of dhcp agent

Vim / etc/neutron/dhcp_agent.ini

Restart the dchp agent service

B. Check the haproxy process on the network node and find that the network with dhcp function enabled will create a corresponding haproxy process and configure 169.254.169.254 on the corresponding dhcp port.

C. Restart instance,instance will send a metadata request to port 80 of 169.254.169.254. According to the internal route of instance, the request will reach the dhcp_port port.

C. When the metadata request arrives at the namespace of dhcp, it will be heard by the haproxy process (the haproxy process listens on port 80 in dhcp namespace)

D, the latter process is exactly the same as L3 agent: the haproxy process forwards the request to neutron-metadata-agent through Unix Domain Socket, which is then forwarded to nova-api-metadata through the management gateway.

6. How does Instance get its own Metadata

To get the metadata from nova-api-metadata, you need to specify the id of the instance

When instance starts, it cannot know its id, so id will not be included in the http request.

Instance id is added by neutron-metadata-agent. For L3 agent and DHCP agent

The implementation of the two schemes is slightly different, which will be explained below.

The process to obtain metadata is as follows:

Instance-> haproxy- > neutron-metadata-agent-> nova-api-metadata

L3 agent

A. The haproxy process receives the metadata request and adds ip and router id to the head of the http before forwarding it to the neutron-metadata-agent.

B. After neutron-metadata-agent receives the request, it queries the id of instance, then adds the instance id to the head of http, and then forwards it to nova-api-metadata.

C, nova-api-metadata responds to the request to the specified instance.

DHCP agent

A. The haproxy process receives the metadata request and adds ip and network id to the head of the http before forwarding it to the neutron-metadata-agent.

B. After neutron-metadata-agent receives the request, it queries the id of instance, then adds the instance id to the head of http, and then forwards it to nova-api-metadata.

C, nova-api-metadata responds to the request to the specified instance.

In this way, whether instance sends the request to l3-agent or dhcp-agent,nova-api-metadata will eventually know the id of instance and return the correct metadata.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.