Shulou(Shulou.com)05/31 Report--

What this article shares with you is about the method and implementation of ICMP tunnel detection based on statistical analysis. The editor thinks it is very practical, so I share it with you to learn. I hope you can get something after reading this article. Let's take a look at it.

I. Overview

In the intranet environment, ICMP protocol is one of the essential network communication protocols, which is used to detect the network connectivity status. Usually, the firewall will put this protocol by default. Because the firewall is open to the ICMP protocol, malicious attackers often use the ICMP protocol for illegal communication. For example, a situation that often occurs in hacker attacks is that the hacker obtains the permission of a host in a certain way and gets some files, such as domain hash, password files, and so on, which need to be sent back to the local computer for cracking, but the firewall blocks the request initiated by the intranet, only the icmp protocol is not blocked, and the hacker needs to return the file. At this time, if the hacker can ping the remote computer. You can try to establish an ICMP tunnel, which encapsulates traffic into ping packets and is designed to use ping data to penetrate the firewall. Now there are many similar tools on the market, such as icmptunnel, ptunnel, icmpsh and so on.

The following will introduce a simple and effective icmp tunnel detection technology. We will use Spark Streaming to help us detect ICMP tunnels.

Second, the working mechanism of ICMP tunnel 2.1,

ICMP (Internet ControlMessages Protocol, Internet Control message Protocol) is a sub-protocol of the TCP/IP protocol family, and it is a connectionless protocol. The structure of the ICMP protocol is shown in figure 1:

Figure 1

The frequently used ping command is based on ICMP protocol. The default transmission of ping under windows system is: abcdefghijklmnopqrstuvwabcdefghi, a total of 32bytes, as shown in figure 2:

Figure 2

Under linux system, ping transmits 48bytes by default. The former 8bytes changes with time, and the latter is fixed. The content is! "# $% &'() +, -. / 01234567, as shown in figure 3:

Figure 3

In addition, the packet size of ping, that is, the data size, can be modified. Take windows as an example, using ping baidu.com-l 223 and changing it to 223bytes. From the point of view of the content of the package, the rule is the same, just repeating, as shown in figure 4:

Figure 4

Can we change these data to populate our own data? The answer is yes!

This is how ICMP covert tunnels work, changing the default Data filled in by the operating system and replacing it with our own data.

For example, using an icmp tunnel, you can construct a custom data package that contains a www.facebook.com string, as shown in figure 5:

Figure 5

2.2 ICMP tunnel packet inspection

Under the windows system, the default transmission of ping is: the content of abcdefghijklmnopqrstuvwabcdefghi,16 is:

6162636465666768696a6b6c6d6e6f7071727374757677616263646566676869

The default content transmitted by ping under linux system is:! "# $% &'() +, -. / 01234567 8bytes hexadecimal content after removing the 8bytes with a variable beginning:

101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f3031323334353637

For the custom length ping, using ping-s 500 baidu.com under linux produces a 492bit data, and the hexadecimal content with variable beginning is removed as follows:

101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f505152535455565758595a5b5c5d5e5f606162636465666768696a6b6c6d6e6f707172737475767778797a7b7c7d7e7f808182838485868788898a8b8c8d8e8f909192939495969798999a9b9c9d9e9fa0a1a2a3a4a5a6a7a8a9aaabacadaeafb0b1b2b3b4b5b6b7b8b9babbbcbdbebfc0c1c2c3c4c5c6c7c8c9cacbcccdcecfd0d1d2d3d4d5d6d7d8d9dadbdcdddedfe0e1e2e3e4e5e6e7e8e9eaebecedeeeff0f1f2f3f4f5f6f7f8f9fafbfcfdfeff000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f505152535455565758595a5b5c5d5e5f606162636465666768696a6b6c6d6e6f707172737475767778797a7b7c7d7e7f808182838485868788898a8b8c8d8e8f909192939495969798999a9b9c9d9e9fa0a1a2a3a4a5a6a7a8a9aaabacadaeafb0b1b2b3b4b5b6b7b8b9babbbcbdbebfc0c1c2c3c4c5c6c7c8c9cacbcccdcecfd0d1d2d3d4d5d6d7d8d9dadbdcdddedfe0e1e2e3e4e5e6e7e8e9eaebecedeeeff0f1f2f3

The custom length ping under windows is similar. The hexadecimal content is as follows:

6162636465666768696a6b6c6d6e6f70717273747576776162636465666768696a6b6c6d6e6f70717273747576776162636465666768696a6b6c6d6e6f707172737475

Therefore, the data generated by ping in a normal operating system is converted to hexadecimal and incremented from 00 to one or more segments of the repeated sequence that goes all the way to ff.

000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f505152535455565758595a5b5c5d5e5f606162636465666768696a6b6c6d6e6f707172737475767778797a7b7c7d7e7f808182838485868788898a8b8c8d8e8f909192939495969798999a9b9c9d9e9fa0a1a2a3a4a5a6a7a8a9aaabacadaeafb0b1b2b3b4b5b6b7b8b9babbbcbdbebfc0c1c2c3c4c5c6c7c8c9cacbcccdcecfd0d1d2d3d4d5d6d7d8d9dadbdcdddedfe0e1e2e3e4e5e6e7e8e9eaebecedeeeff0f1f2f3f4f5f6f7f8f9fafbfcfdfeff

For custom data packets generated by icmp tunnels, the contents are irregular when converted to hexadecimal. For example, construct a custom data package with the content of R5 * ê and wwwfacebookcom string, whose hexadecimal content is as follows:

4500003e377d400040119c2f0a0001020a0052019d800035002a1a14eacd01000001000000000000037777770866616365626f6f6b03636f6d0000010001

Therefore, according to the data packets generated by ping, whether the content after conversion to hexadecimal is regular or not can be distinguished.

Here, AC automaton string matching method is used for feature matching. The specific methods are as follows:

(1) the repetitive sequences generated by the normal operating system are as follows:

000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f505152535455565758595a5b5c5d5e5f606162636465666768696a6b6c6d6e6f707172737475767778797a7b7c7d7e7f808182838485868788898a8b8c8d8e8f909192939495969798999a9b9c9d9e9fa0a1a2a3a4a5a6a7a8a9aaabacadaeafb0b1b2b3b4b5b6b7b8b9babbbcbdbebfc0c1c2c3c4c5c6c7c8c9cacbcccdcecfd0d1d2d3d4d5d6d7d8d9dadbdcdddedfe0e1e2e3e4e5e6e7e8e9eaebecedeeeff0f1f2f3f4f5f6f7f8f9fafbfcfdfeff

Every 4 bits is divided into a feature array, and the resulting feature array is as follows:

0001, 0203, 0405. "feff"

For data data generated by ping, such as data generated by the following linux:

E7cd0a0000000000101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f505152535455565758595a5b5c5d5e5f606162636465666768696a6b6c6d6e6f707172737475767778797a7b7c7d7e7f808182838485868788898a8b8c8d8e8f909192939495969798999a9b9c9d9e9fa0a1a2a3a4a5a6a7a8a9aaabacadaeafb0b1b2b3b4b5b6b7b8b9babbbcbdbebfc0c1c2c3c4c5c6c7c8c9cacbcccdcecfd0d1d2d3d4d5d6d7d8d9dadbdcdddedfe0e1e2e3e4e5e6e7e8e9eaebecedeeeff0f1f2f3f4f5f6f7f8f9fafbfcfdfeff000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445464748494a4b4c4d4e4f505152535455565758595a5b5c5d5e5f606162636465666768696a6b6c6d6e6f707172737475767778797a7b7c7d7e7f808182838485868788898a8b8c8d8e8f909192939495969798999a9b9c9d9e9fa0a1a2a3a4a5a6a7a8a9aaabacadaeafb0b1b2b3b4b5b6b7b8b9babbbcbdbebfc0c1c2c3c4c5c6c7c8c9cacbcccdcecfd0d1d2d3d4d5d6d7d8d9dadbdcdddedfe0e1e2e3e4e5e6e7e8e9eaebecedeeeff0f1f2f3

Every 4 bits is divided into a feature array, and the resulting feature array is as follows:

"e7cd", "0a00", "1011", "1213", "1415", "feff"

(2) the ac automaton algorithm is used to match the feature array generated by the normal operating system, and the matching degree is calculated by taking the number of matched features * 4/payload length as the matching degree.

Matching degree algorithm:

Matching degree = number of features matched * 4 / length of payload

For the data generated by normal ping data, the calculated matching degree is more than 0.9. the custom data packet generated by Icmp tunnel usually has a low matching degree, so we can distinguish whether it is the packet generated by the normal operating system according to the matching degree.

Third, ICMP tunnel detection based on statistical analysis.

For data generated by normal ping commands, there are the following characteristics:

The number of packets sent per second is relatively small, and a maximum of two packets are usually sent per second.

The content of the request packet is the same as the corresponding response packet

The size of the payload in the packet is fixed, 32bytes in windows, 48bytes in Linux

The content of payload in the packet is fixed. Under windows, it is! "# $% &'() +, -. / 01234567. If you specify the length to be sent by ping, it is a fixed string that is repeated over and over again.

There are only two types of type, 0 and 8. 0 is request data and 8 is response data.

For data generated by ICMP tunnels, there are the following characteristics:

A large number of packets are sent per second, and hundreds of ICMP packets are generated at the same time.

The content of the request packet is different from that of the corresponding response packet

The size of the payload in the packet can be any size

There are some malformed packets with payload whose type is 13-15-17.

The contents of packets generated by individual ICMP tunneling tools are preceded by a 'TUNL' tag to identify the tunnel.

Therefore, according to the characteristics of packets generated by normal ping and ICMP tunnels, ICMP tunnels can be detected by the following characteristics:

Detect the number of packets from the same source. A normal ping sends only 2 packets per second, while an ICMP tunnel can send many packets per second

Detect the size of the payload in the packet. The size of payload packets generated by normal ping is fixed, while the size of ICMP tunnel packets can be arbitrary.

Detect whether the payload in the response packet is inconsistent with the request packet. The packet request response generated by normal ping has the same content, while the ICMP tunnel request response packet can be consistent or inconsistent.

Detect the contents of the payload in the packet. The payload generated by a normal ping is a fixed string, and the payload of an ICMP tunnel can be any

Check if the type of the ICMP packet is 0 and 8. For a packet with payload generated by a normal ping, the type has only 0 and 8 type for the ICMP tunnel can be 13-15-17.

IV. Specific implementation of 4.1 Technology selection

When the statistics-based ICMP tunnel detection is deployed to the production environment, we need to consider the impact of big data scale on the timeliness, throughput and other performance indicators of the model. After many considerations, Spark Streaming is finally selected for on-line detection.

Spark Streaming is an extension of Spark core API, which can achieve high throughput and fault-tolerant real-time streaming data processing. Support to obtain data from a variety of data sources, including Kafka, Flume, Twitter, ZeroMQ, Kinesis and TCP sockets. After obtaining data from the data source, you can use advanced functions such as map, reduce, join and window to process complex algorithms. Finally, the processing results can be stored in the file system, database and field dashboard. On the basis of "One Stack rule them all", you can also use other sub-frameworks of Spark, such as cluster learning, graph computing, etc., to process streaming data.

4.2 testing process

The whole process is mainly divided into three stages:

Data collection = > summary statistics = > check out and filter. The specific detection logic is shown in figure 6:

Figure 6

4.2.1 data collection

The real-time traffic data is read into the Spark Streaming, and the data is processed once in 15 seconds. After sorting and splicing according to the source ip and destination ip size of the packet, the request / response packets sent by the same ip to the destination ip are divided into a group. The packet field time_sort_sip_dip is added with a timestamp timestamp, in order to filter out the packets between the source ip and the destination ip in the last 15 seconds. Ensure that only the data of the same ip and destination ip within the current 15 seconds are processed within each 15 seconds.

4.2.2 Summary Statistics

According to the grouped data, the following characteristic indicators are counted as the basis for final detection:

1. The number of packets requested and responded by ICMP in a time window

two。 Whether the payload part of the captured ICMP request response is the normal length of the operating system

3. ICMP packets with the same sequence number should belong to a pair of ICMP request response messages. Whether their payload is the same.

4. Whether the payload content of the ICMP request response is normal (multi-mode matching)

5. Whether the ICMP request is a malformed Ping message (whether the Type is abnormal)

On 1-2-5, the number of payload in the statistical group, the value of payload_len length, and whether there are any packets whose type is not 0ap8 can be obtained directly.

The judgment of "and 3" needs to be based on the difference between normal ping and abnormal data, and after the duplicates are removed from the id_seq set and payload set obtained by statistics in the group, the size comparison is made:

For a normal ping, the id_seq in a series of packets sent by the source ip to the destination ip is all different, where identify means that the sequence number randomly generated by the current request is not duplicated. Like the same batch of identify, seq is a number that increases one at a time based on the number of packets, and the id_seq of the same request and response packet is the same as the payload content. For example: ping baidu.com generates 10 packets, including 5 requests and 5 responses, then the id_seq is deduplicated and the payload is deduplicated. After the duplicates are removed, it is 1 in win and 5 in Linux.

For Icmp tunnels, the id_seq in a series of packets sent by the source ip to the destination ip can be the same or different. The id_seq of the same request and response packet can be the same or different, and the payload content is different. For example, if 10 packets are generated in the tunnel, the id_seq will be 5 (seq increments sequentially) or 1 (seq will not be incremented), and the payload will be 10. In extreme cases, only packets will be sent and no response packets will be sent, then id_seq will be 10 after de-duplication.

It can be seen that regardless of some extreme cases, you can simply use payloads_num > id_seq_num to consider that the content of the request response is different, and the extreme cases can be checked out according to other conditions.

For the judgment of 4, use the method described in 2.2 to determine whether the payload is abnormal.

4.2.3 check-out filtering

In order to reduce false positives, filter out packets sent by some normal routers, and use a combination of three conditions to determine whether or not to check out:

There are payload with illegal type and the number of deduplicated payload is greater than the threshold (tested in the intranet, individual routers will send packets with malformed type with the same payload content, and filter by increasing the number of de-duplicated payload to be greater than the threshold)

Request response packet inconsistency

The number of packets within 15 seconds, the number of abnormal length packets, the number of abnormal content packets, and the number of deduplicated payload are greater than the self-defined threshold (after intranet testing, individual routers will send payload heartbeats with abnormal content but only a few characters different, by adding weight removal (using hamming distance to remove weight). The difference of no more than 3 characters is considered the same) the number of payload is greater than the threshold to filter).

In view of the diversity of Icmp tunnel characteristics, as long as one of the three is satisfied, it will be detected. For packets generated by some extremely abnormal ICMP tunnels, for example, if only request packets are sent out without response packets in the tunnel, the id_seq is 10 (10 times are all request packets and seq increments), and payload is 10 after weight removal, so it cannot be checked out through 2 alone, but it can also be checked out and will not fail to report.

Fifth, model effect 5.1 negative sample detection effect

At present, the known negative samples used in the testing process are as follows:

IcmpTunnel samples, samples made using the icmpTunnel tunneling tool

Icmptransmitter samples, samples made using the icmptransmitter tunneling tool

Icmpsh samples, samples made using the icmpsh tunneling tool

Badping sample, using the python script to manually create icmp exception packets.

5.2 positive sample detection effect

At present, there are several known positive samples used in the testing process:

Normal Ping data packet, data packet generated by Ping command, platform including linux/win/mac/solaris/android/ios,payload content as follows: abcdefg12345 and so on.

Network equipment heartbeat packets, routers and other network devices will regularly send some heartbeat packets, the payload content is as follows: DataBuffer 0000001, DataBuffer 0000002 and so on.

The above is the method and implementation of ICMP tunnel detection based on statistical analysis. The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.