Shulou(Shulou.com)06/01 Report--

This article will explain in detail the example analysis of NAT message tracking on network address translation. The editor thinks it is very practical, so I share it with you as a reference. I hope you can get something after reading this article.

Introduction

Network address Translation (NAT) is a way to expose containers or virtual machines to the Internet. The incoming connection request rewrites its destination address to another address and is then routed to the container or virtual machine. The same technique can also be used for load balancing, where incoming connections are distributed to different servers.

When the network address translation does not work as expected, the connection request will fail, the wrong service will be exposed, the connection will eventually appear in the wrong container, or the request timeout, and so on. One way to debug such problems is to check whether the incoming request matches the expected or configured transformation.

Connection tracking

NAT is not just about changing the IP address or port number. For example, when mapping addresses X to Y, there is no need to add new rules to perform reverse translation. A netfilter system called "conntrack" can recognize reply messages that are already connected. Each connection has its own NAT state in the conntrack system. The reverse conversion is done automatically.

Rule matching tracking

The nftables tool (and to a lesser extent iptables) allows you to check for a message how it is processed and which rule in the set of rules the message matches. To use this special feature, you can insert a "tracking rule" in the appropriate place. These rules select the messages to be tracked. Suppose a host from IP address C is accessing a service whose IP address is S and port is P. We want to know which NAT conversion rules the message matches, which rules are checked by the system, and whether the message is discarded or not.

Since we are dealing with incoming connections, we add the rule to the prerouting hook. Prerouting means that the kernel has not yet decided where to send the message. Changing the destination address usually causes the message to be forwarded by the system rather than processed by the host itself.

Initial configuration # nft 'add table inet trace_debug'# nft' add chain inet trace_debug trace_pre {type filter hook prerouting priority-200000;}'# nft "insert rule inet trace_debug trace_pre ip saddr $C ip daddr $S tcp dport $P tcp flags syn limit rate 1/second meta nftrace set 1"

The first rule adds a new rule table, which makes it easier to delete and debug rules in the future. All rules and chains temporarily added to the table during debugging can be deleted with a single nft delete table inet trace_debug command.

The second rule creates a basic hook before the system makes routing (the prerouting hook) and sets its priority to a negative number to ensure that it is executed before the connection tracking process matches the NAT rule.

However, the only most important part is the last paragraph of the third rule: meta nftrace set 1. This rule causes the system to log events associated with all messages that match the rule. In order to view the tracking information as efficiently as possible (improve the signal-to-noise ratio), consider adding a rate limit to the tracked events to ensure that the number of tracked events is within a manageable range. A good option is to limit up to one message per second or one message per minute. The above case records all SYN and SYN/ACK messages from terminal $C and destined for port $P of terminal $S. The rate-limiting configuration statement can prevent the risk of flooding caused by too many events. In fact, in most cases, only one message is enough.

For iptables users, the configuration process is similar. The equivalent configuration rules are similar to:

# iptables-t raw-I PREROUTING-s $C-d $S-p tcp--tcp-flags SYN SYN-- dport $P-m limit-- limit 1 move s-j TRACE to get tracking events

Users of the native nft tool can directly run nft to enter nft trace mode:

# nft monitor trace

This command prints out the received message and all rules that match the message (stop the output with CTRL-C):

Trace id f0f627 ip raw prerouting packet: iif "veth0" ether saddr..

We will analyze the results in detail in the next chapter. If you are using iptables, first check the installed version with the iptables-version command. For example:

# iptables-versioniptables v1.8.5 (legacy)

(legacy) means that tracked events are logged to the kernel's ring buffer. You can use the dmesg or journalctl command to view these events. These debug outputs lack some information, but are conceptually similar to those provided by the new tool. You will need to first look at the line number recorded by the rule and manually associate it with the active iptables rule set. If the output is displayed (nf_tables), you can use the xtables-monitor tool:

# xtables-monitor-trace

If the above command only shows the version number, you still need to look at the output of dmesg/journalctl. The xtables-monitor tool and the nft monitoring and tracking tool use the same kernel interface. The only difference between them is that the xtables-monitor tool prints events using the syntax of iptables, and if you use both iptables-nft and nft, it will not be able to print rules that use maps/sets or other features that only nftables supports.

Example

Let's assume that we need to debug a problem where the port to the virtual machine / container is not working. The ssh-p 1222 10.1.2.3 command should be able to remotely connect to a container on that server, but the connection request timed out.

You have login rights to the host running that container. Now log in to the machine and add a tracking rule. You can use the above case to see how to add a temporary table of debugging rules. The tracking rule looks something like this:

Nft "insert rule inet trace_debug trace_pre ip daddr 10.1.2.3 tcp dport 1222 tcp flags syn limit rate 6/minute meta nftrace set 1"

After adding the above rules, run nft monitor trace, start nft in trace mode, and then retry the ssh command that just failed. If the rule set is large, there will be a large amount of output. Don't worry about the output, we'll do a line-by-line analysis in the next section.

Trace id 9c01f8 inet trace_debug trace_pre packet: iif "enp0" ether saddr.. Ip saddr 10.2.1.2 ip daddr 10.1.2.3 ip protocol tcp tcp dport 1222 tcp flags = = syntrace id 9c01f8 inet trace_debug trace_pre rule ip daddr 10.2.1.2 tcp dport 1222 tcp flags syn limit rate 6/minute meta nftrace set 1 (verdict continue) trace id 9c01f8 inet trace_debug trace_pre verdict continuetrace id 9c01f8 inet trace_debug trace_pre policy accepttrace id 9c01f8 inet nat prerouting packet: iif "enp0" ether saddr.. Ip saddr 10.2.1.2 ip daddr 10.1.2.3 ip protocol tcp tcp dport 1222 tcp flags = = syntrace id 9c01f8 inet nat prerouting rule ip daddr 10.1.2.3 tcp dport 1222 dnat ip to 192.168.70.10 oif 22 (verdict accept) trace id 9c01f8 inet filter forward packet: iif "enp0" oif "veth31" ether saddr. Ip daddr 192.168.70.10.. Tcp dport 22 tcp flags = = syn tcp window 29200trace id 9c01f8 inet filter forward rule ct status dnat jump allowed_dnats (verdict jump allowed_dnats) trace id 9c01f8 inet filter allowed_dnats rule drop (verdict drop) trace id 20a4ef inet trace_debug trace_pre packet: iif "enp0" ether saddr. Ip saddr 10.2.1.2 ip daddr 10.1.2.3 ip protocol tcp tcp dport 1222 tcp flags = = syn analyzes the tracking results line by line

The first line of the output is the message number that triggers the subsequent output. The syntax of this line is the same as the syntax of the nft rule, but also includes the first field information of the received message. You can also find the name of the interface that received the message (enp0 in this case), the source and destination MAC addresses of the message, the source IP address of the message (which may be important-the person reporting the problem may have chosen an incorrect or unexpected host), and the source and destination ports of the TCP. You can also see a "tracking number" at the beginning of the line. The number identifies a specific message that matches the tracking rule. The second line includes the first tracking rule for the match of the message:

Trace id 9c01f8 inet trace_debug trace_pre rule ip daddr 10.2.1.2 tcp dport 1222 tcp flags syn limit rate 6/minute meta nftrace set 1 (verdict continue)

This is the trace rule you just added. The first rule shown here is always the rule that activates message tracking. If there are other rules before that, they will not be displayed here. If there is no trace output, the trace rule has not been reached or the match has not been successful. The following two lines indicate that there are no subsequent matching rules, and the trace_pre hook allows the message to continue to be transmitted (determined to be accepted).

The next matching rule is:

Trace id 9c01f8 inet nat prerouting rule ip daddr 10.1.2.3 tcp dport 1222 dnat ip to 192.168.70.10 verdict accept

This DNAT rule sets a mapping to other addresses and ports. The parameter 192.168.70.10 in the rule is the address of the virtual machine that needs to be received, and so far there is no problem. If it is not the correct virtual machine address, either the address is entered incorrectly or the wrong NAT rule is matched.

IP forwarding

From the following output, we can see that the IP routing engine tells the IP protocol stack that the message needs to be forwarded to another host:

Trace id 9c01f8 inet filter forward packet: iif "enp0" oif "veth31" ether saddr.. Ip daddr 192.168.70.10.. Tcp dport 22 tcp flags = = syn tcp window 29200

This is another form of presentation of the received message, but there are some interesting differences from before. Now the result is a collection of output interfaces. This did not exist before, because the previous rule was before the routing decision (prerouting hook). The tracking number is the same as before, so it is still the same message, but the destination address and port have been modified. Assuming that there are rules that match tcp dport 1222, they will not have any impact on messages at this stage.

If the line does not contain an output interface (oif), the routing decision routes the message to the local machine. Debugging the routing process belongs to another topic, which is no longer covered in this article.

Trace id 9c01f8 inet filter forward rule ct status dnat jump allowed_dnats (verdict jump allowed_dnats)

This output shows that the message matches a rule that jumps to the allowed_dnats chain. The next line explains the root cause of the connection failure:

Trace id 9c01f8 inet filter allowed_dnats rule drop (verdict drop)

This rule discards the message unconditionally, so there is no subsequent log output about the message. The next line is the output of another message:

Trace id 20a4ef inet trace_debug trace_pre packet: iif "enp0" ether saddr.. Ip saddr 10.2.1.2 ip daddr 10.1.2.3 ip protocol tcp tcp dport 1222 tcp flags = = syn

The tracking number is different from before, and then the content of the message is the same as before. This is a retransmission attempt: the first message is discarded, so TCP tries to retransmit. The rest of the output can be ignored because it does not provide new information. Now it's time to check that chain.

Rule set analysis

In the previous section, we found that the message was discarded in a chain called allowed_dnats in the inet filter table. Now let's look at it:

# nft list chain inet filter allowed_dnatstable inet filter {chain allowed_dnats {meta nfproto ipv4 ip daddr. Tcp dport @ allow_in accept drop}}

The rules for accepting packets from the @ allow_in set are not shown in the trace log. By listing the elements, we check again whether the destination address of the above message is in the @ allow_in set:

# nft "get element inet filter allow_in {192.168.70.10. 22}" Error: Could not process rule: No such file or directory

As expected, the address-service pair does not appear in the collection. We add it to the collection.

# nft "add element inet filter allow_in {192.168.70.10. 22}"

Now run the query command, which will return the newly added elements.

# nft "get element inet filter allow_in {192.168.70.10. 22}" table inet filter {set allow_in {type ipv4_addr. Inet_service elements = {192.168.70.10. 22}

The ssh command should now work, and the trace results reflect this change:

Trace id 497abf58 inet filter forward rule ct status dnat jump allowed_dnats (verdict jump allowed_dnats) trace id 497abf58 inet filter allowed_dnats rule meta nfproto ipv4 ip daddr. Tcp dport @ allow_in accept (verdict accept) trace id 497abf58 ip postrouting packet: iif "enp0" oif "veth31" ether. Trace id 497abf58 ip postrouting policy accept

This indicates that the message passed the last hook in the forwarding path-postrouting.

If you are still unable to connect, the problem may be at a later stage of the message flow and may not be within the scope of the nftables rule set.

This is the end of the article on "sample Analysis of NAT message tracking in Network address Translation". I hope the above content can be helpful to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.