Shulou(Shulou.com)06/01 Report--

This article will explain in detail the example analysis of tcpdump crawling HTTP package in linux. The editor thinks it is very practical, so I share it with you for reference. I hope you can get something after reading this article.

Tcpdump

Tcpdump is a package grabbing tool that comes with the linux system. It is more suitable for online servers to grab packets mainly through the command line. If windows or ubuntu can choose some graphical tools, ubuntu recommends wireshark. The installation method is very simple sudo apt.

Command line format:

Tcpdump [- adeflnNOpqStvx] [- c quantity] [- F file name] [- I network interface] [- r file name] [- s snaplen] [- T type] [- w file name] [expression]

Commonly used parameters:

-l turns standard output into buffered line form

-n does not convert network addresses into names

-c after receiving the specified number of packets, tcpdump will stop

-I specify the network interface for listening; (if not specified, it is possible to listen on the default network card, you need to specify a network card bound with a specific IP)

-w writes the package directly to the file and does not analyze and print it

-s specifies the size of the record package. Common-s 0 represents the maximum value of 65535, and half of the linux transmission minimum unit MTU is 1500, which is enough.

-X outputs package data data directly. It is not set by default and can only be exported through-w specified files.

Common expressions:

Keywords about types, mainly including host,net,port

Keywords of transmission direction, including src, dst, dst or src, dst and src

Keywords of the protocol, including fddi,ip, arp,rarp,tcp,udp and other types

Logical operation, take non-operation as' not'!', and operation is' and','&&';'or operation is' or','| |'

Other important keywords are as follows: gateway, broadcast,less,greater

Practical examples:

1. Http packet capture (output package data directly in the terminal)

Tcpdump tcp port 80-n-X-s 0 specifies port 80 for output

two。 Grab the specified file of http package data to output package

Tcpdump tcp port 80-n-s 0-w / tmp/tcp.cap

The corresponding / tmp/tcp.cap basically depends on the naked eye to see the information, such as http Header, content information, etc.

3. Combined pipeline flow

Tcpdump tcp port 80-n-s 0-X-l | grep xxxx

This allows real-time string matching filtering of packets.

4. Mod_proxy reverse proxy grabs the package

Online server apache+jetty, a reverse proxy through apache mod_proxy, 80 apache port, 7001 jetty port

Apache port data capture package: tcpdump tcp port 80-n-s 0-X-I eth0 Note: specify eth0 network interface

Jetty port data capture package: tcpdump tcp port 7001-n-s 0-X-I lo Note: specify Loopback network interface

5. Monitor only specific ip hosts

Tcpdump tcp host 10.16.2.85 and port 2100-s 0-X

You need to use a combination of tcp expressions, where host instructs to listen only to the ip

Tips:

1. Can be combined with tcpdump (command) + wireshark (graphical)

Action:

Tcpdump-w / tmp/tcp.cap on the server specifies the output of external files

Scp / tmp/tcp.cap copy files to your local location

Wireshark & start wireshark

Open the copied file through File-> Open so that you can use it for packet analysis

The rest will be very convenient.

Tcpdump-XvvennSs 0-I eth0 tcp [20:2] = 0x4745 or tcp [20:2] = 0x4854

0x4745 is the first two letters "GE" of "GET".

0x4854 is the first two letters "HT" of "HTTP".

Description: usually: a normal TCP connection has three stages: 1, TCP three-way handshake; 2, data transfer; 3. TCP waving four times.

Here are a few concepts:

SYN: (synchronous sequence number, Synchronize Sequence Numbers)

ACK: (confirmation number, Acknowledgement Number)

FIN: (closing flag, FINish)

TCP three-way handshake (create OPEN)

The client initiates a request to create a TCP link with the service, here is SYN (J)

After receiving a creation request from the client, the server terminal returns two messages: SYN (K) + ACK (junk 1)

The client returns a message: ACK (ACK 1) after receiving the ACK message from the server successfully (J and Jig1).

At this time, the server receives the ACK information from the client after the verification is successful (K and Kroom1), no longer returns the information, and then enters the data communication phase.

Data communication

Client / server read/write packet

TCP four-way handshake (turn off finish)

The client initiates a shutdown request and sends a message: FIN (M)

After receiving the message, the server first returns ACK (masking 1) to indicate that it has received the message.

Before the server is ready to shut down, it finally sends a FIN (N) message to the client, asking if the client is ready to shut down.

After receiving the message sent by the server, the client returns a confirmation message: ACK (Number1)

Finally, when both the server and the client are confirmed, each closes or reclaims the corresponding TCP link.

Detailed status description (and linux related parameter adjustment)

SYN_SEND

The client tries to link the server through the open method. After the first step in the TCP three-way handshake, notice that the client state

Sysctl-w net.ipv4.tcp_syn_retries = 2. As the client, you can set the number of retries for the SYN package. By default, 5 times (about 180 s) quote the headmaster: only 2 retries, the modern network is enough.

SYN_RECEIVED

After the service accepts the SYN of the creation request, which is step 2 of the TCP three-way handshake, before sending the ACK packet

Note that it is the server status. Generally, about 15 are normal. If it is very large, it is suspected to be attacked by SYN_FLOOD.

Sysctl-w net.ipv4.tcp_max_syn_backlog=4096, which sets the number of waiting queues for this state. Default is 1024, which can be adjusted to prevent syn-flood. For more information, please see man 7 tcp

Sysctl-w net.ipv4.tcp_syncookies=1, open syncookie, and provide a mechanism to temporarily swap out syn links when the syn backlog queue is insufficient.

Sysctl-w net.ipv4.tcp_synack_retries = 2, the number of retries that the server returns the ACK packet. By default, it is 5 times (about 180 s) to quote the headmaster: only 2 retries, the modern network is enough.

ESTABLISHED

The status of the client after receiving the ACK packet of the server, and the server is ESTABLISHED after sending out the ACK for a certain period of time

Sysctl-w net.ipv4.tcp_keepalive_time = 1200, default is 7200 seconds (2 hours). The system will check the heartbeat for idle links. If you exceed the net.ipv4.tcp_keepalive_probes * net.ipv4.tcp_keepalive_intvl = default 11 points, terminate the corresponding tcp links, and adjust the heartbeat check frequency appropriately.

Current online monitoring waring:600, critial: 800

FIN_WAIT1

The party that closes voluntarily, after issuing the FIN request, that is, the first step of the four-way handshake in TCP.

CLOSE_WAIT

The passively closed party, after receiving the FIN from the client, is the second step of the four-way handshake in TCP.

FIN_WAIT2

The active closing party, after receiving the ACK of the passive closing party, is the second step of the TCP four-way handshake.

Sysctl-w net.ipv4.tcp_fin_timeout=30, you can set the timeout after the passive shutdown party returns FIN, effectively reclaim the link and avoid syn-flood.

LASK_ACK

The party that shuts down passively initiates another FIN request after a period of time after sending the ACK (to ensure that the client has received it). That's the third step of TCP's four-way handshake.

TIME_WAIT

The party that closes actively sends ACK after receiving the passively closed FIN packet. That's the fourth step of TCP's four-way handshake.

Sysctl-w net.ipv4.tcp_tw_recycle = 1, open Fast Recycling TIME_WAIT,Enabling this option is not recommended since this causes problems when working with NAT (Network Address Translation)

Sysctl-w net.ipv4.tcp_tw_reuse = 1, quickly recycle and reuse TIME_WAIT links, seems to be in conflict with tw_recycle, can not be reused?

Net.ipv4.tcp_max_tw_buckets: the maximum number of links in the time_wait state. Default is 180000.

Related instructions

After receiving the FIN request from the passive shutdown party, the active closing party successfully sends an ACK to the other party, then changes its status from FIN_WAIT2 to TIME_WAIT, and has to wait for twice the time of MSL (Maximum Segment Lifetime,MSL is the time for a Datagram to exist in internetwork) before both parties can change the status to CLOSED to close the connection. Currently, the time to maintain TIME_WAIT status in RHEL is 60 seconds.

KeepAlive strategy can effectively avoid three-way handshake and four closes.

Other important network parameters

Net.ipv4.tcp_rmem parameter

Default value: min=4096 default=87380 max=4194304

Net.ipv4.tcp_wmem parameter

Default value: min=4096 default=16384 max=4194304

This is the end of this article on "sample analysis of tcpdump crawling HTTP packages in linux". I hope the above content can be helpful to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.