Shulou(Shulou.com)06/03 Report--

This article mainly introduces "how the client request arrives at the server". In the daily operation, I believe that many people have doubts about how the client request reaches the server. The editor consulted all kinds of data and sorted out the simple and easy-to-use operation method. I hope it will be helpful to answer the doubt of "how the client request reaches the server". Next, please follow the editor to study!

1. Data packet

I. HTTP request and response steps

The whole process of http request

Request

Response

The above fully represents the seven steps of the HTTP request and response. Let's understand how the HTTP request and response are delivered from the perspective of the TCP/IP protocol model.

2. Overview of TCP/IP

We use the diagram in RFC 1180 as a reference

The above figure shows the four-layer TCP/IP protocol diagram, in which network applications is the application program and belongs to the application layer; TCP and UDP mainly transmit data and belong to the transport layer, TCP ensures end-to-end reliable transmission and ensures the healthy operation of the network as far as possible, while UDP is simple and unreliable transmission; IP mainly solves routing problems and belongs to the network layer ARP is a network address translation, which is mainly used to translate IP addresses and MAC addresses, which is between the data link layer and the network layer, and can be regarded as layer 2.5. here, ENET is the data link layer, and the Nic driver belongs to this layer, mainly for specific media transmission. The advertisement request packet capture in the previous example is captured at the data link layer.

It is worth noting that ARP belongs to the network layer in the linux system, while in RFC it is between the data link layer and the network layer. In the book detailed explanation of TCP/IP, ARP is put into the data link layer. When solving practical problems, we should put ARP on the network layer.

The figure above shows how to use TCPCopy to send packets at different layers. If TCPCopy sends packets from the data link layer, users need to add an additional MAC address when using TCPCopy because they do not enjoy ARP services, while if TCPCopy sends packets from the IP layer, there is no need to specify a MAC address.

This course focuses on TCP-related cases. Without special instructions, TCP refers to the TCP of the transport layer.

3. What is TCP

TCP, the transmission control protocol, is a connection-oriented, reliable, byte-stream-based communication protocol. The main work of TCP is to define the identity of port identification applications, achieve reliable end-to-end communication, and carry out congestion control to prevent the collapse of the Internet.

TCP has the following key features:

Because TCP is a connection-oriented protocol, it is a stateful protocol, and stateful protocols are often complex, so it is difficult for TCP to learn.

Take a look at the complexity of the TCP state through the state diagram:

The image above shows an intricate TCP state diagram, but the reality is more complex.

The real TCP state diagram actually looks like this:

In the figure, not only can the SYN_RCVD state receive reset packets (the RST,reset packets in the figure are packets that reset the connection, which can instantly change the TCP state to the CLOSED state, while the CLOSED state is untraceable), but FIN_WAIT1, FIN_WAIT2, ESTABLISHED, SYN_SENT, and CLOSE_WAIT can all be returned to the CLOSED state by reset packets. Not only that, the TCP state is also affected by timeouts. For example, in a Linux system, once the connection is in FIN_WAIT_2, the system will change the connection state FIN_WAIT_2 directly to CLOSED state if no peer FIN packet is received within 60 seconds (default). Although Linux does this to prevent attacks, this practice of killing the connection state on a timeout is likely to kill a lot of normal connections, making the problem more elusive.

In the field of the Internet, many weird questions have something to do with the status of TCP, and there will be more cases to talk about these weird problems later in the course.

4. Port

When we connect to the server program, we need to specify the server port. Why do you need a port? Because the system distinguishes different applications by port, TCP finds the upper application through the port.

It should be noted that the port number is limited, the maximum port number is 65535, and the number of ports that can be utilized depends on the configuration. For example, during stress testing, the number of TCP ports that a machine can use is limited, and the connections that can be utilized (client port, client IP, server application port, server IP address) are limited. To solve this problem, you can configure multiple IP addresses to expand the number of available connections. The system supports the same port, different IP addresses to bind different applications. Under the Linux high version system, we can also use the REUSEPORT mechanism to make different applications share the same listening port when the same IP address is bound, which is very useful for high-performance server development. The database middleware cetus we developed uses this mechanism to solve the problem of short link storm.

5 、 IP

The main work of the network layer is to define network addresses, distinguish network segments, MAC addressing in subnets, and route packets from different subnets. The main function of IP is to send packets to the final destination address in a complex network environment.

IP is connectionless and is a stateless protocol. Why is IP designed to be stateless?

1. Stateless protocol processing is simple

two。 There is no need to establish a connection before communicating

3. TCP is already connection-oriented, and the IP layer can delegate TCP to solve the connection-oriented problem.

4. Without status, the Internet is more free to route and more fault-tolerant.

It is worth noting that in reality, the IP layer is often equipped with security filtering, and even some routers, firewalls and other midway devices will interfere with applications (such as interfering with TCP sessions through reset packets). In order to better do security detection, the IP layer also adds connection tracking to track upper layer connections on top of stateless protocols. This approach improves security, but sometimes it brings new problems, and we have a case that tells the story of the hole brought about by connnection tracking.

6 、 TCP Socket

The application invokes the TCP service through the TCP socket interface, thus achieving the purpose of transferring data. Each TCP socket is bound to a port, and the TCP socket can communicate in both directions, sending and receiving data at the same time.

It is worth noting that after the application sends the data, it only means that the work entrusted to the TCP through TCP socket has been completed, it does not mean that it has been sent to the peer, and the data sent by the application and the data transmitted by TCP are not synchronized.

7 、 How TCP/IP Works

When the user sends the request through the TCP socket interface, the TCP protocol module takes over the request delivery. TCP first splits the request into smaller data segments (assuming that TCP offload is not enabled) and sends it through the IP layer. At the IP layer, these data segments are encapsulated into IP packets that are sent to the Internet through the data link layer (see figure below). These packets reach their destination through multiple routers on the Internet. Because the IP network is a stateless protocol, each packet may take a different path and arrive in a different order, which requires the peer TCP to reassemble the packet to ensure that the data transmitted to the application layer is a user request that the user can recognize, so that the server application can process the request made by the user.

In the following figure, assume that the user request is split into two IP packets

The first IP packet may pass through A _ Magi B ~ P C ~ G, as shown in the figure below.

The second packet may pass through A _ Magi B _ E ~ G (the E node is selected at point B, resulting in a different path), as shown in the following figure.

There may be many reasons for the packets to go to different paths in the B node, for example, the C node is not as unobstructed as the E node or temporary network congestion, which is similar to the principle of driving on the highway.

Due to the changeable network environment, it is possible that the second packet arrives at the server first, and TCP will handle the out of order situation. If a router is too busy and loses the first packet during network transmission, the TCP on the client side will retransmit the first packet to ensure that the TCP on the server side will not miss the first packet because of packet loss.

If the user requests a large content, such as uploading a large file, it will be split into a large number of data segments, and when TCP transmits these data segments, it will often consider the extent to which the whole Internet can receive and the other party can receive. Too greedy in sending data will not only affect the whole Internet, the other party may not be able to receive it, but may also make itself slower, which is a bit like road driving. Can not be too selfish, obey certain traffic rules in order to make the road smooth. In the aspect of data transmission on the Internet, these traffic rule algorithms are famous network congestion control algorithms, and whether the other party can receive it or not, it is controlled by the way of sending window. Generally speaking, the size of a transmission is determined comprehensively according to the receiving window size and congestion control algorithm of the other party.

As can be seen from the above, IP is responsible for transmitting data over the Internet, while TCP is responsible for reliable data transmission and making the network run healthily as far as possible. The two cooperate to complete the transmission of requests, which is also a common way of Internet application work.

It should be noted that TCP is responsible for interacting with TCP, and the application layer does not need to realize the function of TCP, but only needs to delegate to TCP to complete the data transmission. This isolation method brings convenience to the development / operation and maintenance / testing of the application layer. In addition, when there are TCP-related problems, the difficulty of solving the problem is also greatly increased.

8. TCP experience and knowledge

In the course of many years of actual combat, we have found the following TCP experience very helpful. The summary is as follows:

1. The farther the distance is, the greater the delay is, the greater the retransmission probability is.

two。 The quality of the network directly affects the performance of the application.

3. Different congestion algorithms are used in different environments

4. Congestion control algorithm is the essence of the Internet and one of the key factors for the great success of the Internet.

5. TCP is a stateful protocol and uses asynchronous processing.

6. Packet analysis is a sharp tool to find the root cause of TCP-related problems.

7. The interaction between TCP client and TCP server is the common interaction part of all applications in the application layer. Understanding this principle can solve a large number of TCP-related problems.

9. IP experience and knowledge

In terms of IP experience and knowledge, we roughly summarize as follows:

1. The packet selection path is not fixed, and the order of arrival may be out of order.

two。 Safety filter, where there are many holes

3. Stateless protocols, which simplify the Internet architecture, are one of the key factors in the great success of the Internet.

4. The IP layer seldom fragments the packets transmitted by TCP.

5. The midway device not only has the routing function, but also interferes with the TCP session (a hotbed of psychic problems)

10. Concluding remarks

TCP/IP network is the cornerstone of the Internet, understanding how TCP/IP works is very helpful for us to solve problems.

At this point, the study on "how the client request reaches the server" is over. I hope to be able to solve everyone's doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.