Shulou(Shulou.com)11/24 Report--

This article comes from the official account of Wechat: developing Internal skills practice (ID:kfngxl). Author: Zhang Yanfei allen

Hello, everyone. I'm Brother Fei!

In the entry interview for the relevant positions in the back-end, the frequency of the three-way handshake is so high that it is not too much to say that it is a required exam. The general answer is how the client initiates the SYN handshake to enter the SYN_SENT state, the server responds to the SYN and replies to the SYNACK, and then enters the SYN_RECV,. Oh, bar, bar, and so on.

But I want to give a different answer today. In fact, in the implementation of the kernel, the three-way handshake is not only a simple flow of state, but also includes key operations such as semi-connection queue, syncookie, full connection queue, retransmission timer and so on. If you can deeply understand these, your online grasp and understanding will be further. If an interviewer asks about your three handshakes, I'm sure this answer will help you win a lot of points in front of the interviewer.

In TCP-based service development, the main flow chart of the three-way handshake is as follows.

The core code in the server is to create the socket, bind the port, listen to the listen, and finally the accept receives the request from the client.

/ / server core code int main (int argc, char const * argv []) {int fd = socket (AF_INET, SOCK_STREAM, 0); bind (fd,...); listen (fd, 128); accept (fd,...);...} client code is to create socket, and then call connect to connect to server.

/ / client core code int main () {fd = socket (AF_INET,SOCK_STREAM, 0); connect (fd,);} around this three-way handshake diagram, as well as the core code of the client and server, let's deeply explore the internal operation of the three-way handshake. Let's start with listen, which has a lot to do with the three-way handshake!

Friendly hint: there will be more kernel source code in this article. If you can understand it better, and if you find it difficult to understand, just focus on the descriptive text in this article, especially the bold part. In addition, there is a summary picture at the end of the article to summarize and sort out the full text.

First, the listen of the server, we all know that the server needs to listen before starting to provide services. But we seldom think about what is done inside listen.

Today let's take a closer look at the kernel code that was executed directly in the previous listen.

/ / file: net/core/request_sock.cint reqsk_queue_alloc (struct request_sock_queue * queue, unsigned int nr_table_entries) {size_t lopt_size = sizeof (struct listen_sock); struct listen_sock * lopt; / / calculate the length of the semi-connected queue nr_table_entries = min_t (U32, nr_table_entries, sysctl_max_syn_backlog); nr_table_entries =. / / request memory for semi-connected queue lopt_size + = nr_table_entries * sizeof (struct request_sock *); if (lopt_size > PAGE_SIZE) lopt = vzalloc (lopt_size); else lopt = kzalloc (lopt_size, GFP_KERNEL); / / fully connected queue header initialization queue- > rskq_accept_head = NULL; / / semi-connected queue setup lopt- > nr_table_entries = nr_table_entries; queue- > listen_opt = lopt In this code, the kernel calculates the length of the semi-connection queue. Then calculate the actual memory needed for semi-connected queues, and start to apply for memory for managing semi-connected queue objects (semi-connected queues need to be looked up quickly, so the kernel manages semi-connected queues with a hash table, specifically under syn_table under listen_sock). Finally, the semi-connection queue is hung on the receiving queue queue.

In addition, queue- > rskq_accept_head represents a fully connected queue, which is in the form of a linked list. In listen, since there is no connection yet, set the fully connected queue header queue- > rskq_accept_head to NULL.

When there are elements in fully connected queues and semi-connected queues, their structure in the kernel is roughly as follows.

In the server listen, it mainly calculates the length limit of the full / half connection queue, as well as the related memory request and initialization. The full / connection queue is initialized before it can respond to a handshake request from the client.

If you want to know more details about the internal operation of listen, you can read the previous article, "Why do server programs need to listen first?" "

Second, the client connect client initiates the connection by calling connect. The tcp_v4_connect of the kernel source code is entered in the connect system call.

/ / file: net/ipv4/tcp_ipv4.cint tcp_v4_connect (struct sock * sk, struct sockaddr * uaddr, int addr_len) {/ / set the socket status to TCP_SYN_SENT tcp_set_state (sk, TCP_SYN_SENT); / / dynamically select a port err = inet_hash_connect (& tcp_death_row, sk); / / function is used to build a completed syn message based on the information in sk and send it out. Err = tcp_connect (sk);} here you will finish setting the socket state to TCP_SYN_SENT. After dynamically selecting an available port through inet_hash_connect (for detailed port selection process, refer to the previous article, "how is the port number of the client in a TCP connection determined?" "), enter into tcp_connect

/ / file:net/ipv4/tcp_output.cint tcp_connect (struct sock * sk) {tcp_connect_init (sk); / / apply for skb and construct a SYN package. / / add tcp_connect_queue_skb (sk, buff) to the sending queue sk_write_queue; / / actually issue syn err = tp- > fastopen_req? Tcp_send_syn_data (sk, buff): tcp_transmit_skb (sk, buff, 1, sk- > sk_allocation); / / start the retransmission timer inet_csk_reset_xmit_timer (sk, ICSK_TIME_RETRANS, inet_csk (sk)-> icsk_rto, TCP_RTO_MAX);} apply and construct the SYN packet in tcp_connect, and then send it out. At the same time, a retransmission timer is started, which is used to start retransmission when there is no feedback from the server after a certain period of time. The first timeout was 1 s in version 3.10 and 3 s in some older versions.

To sum up, when the client is in connect, it sets the local socket state to TCP_SYN_SENT, selects an available port, and then issues a SYN handshake request and starts a retransmission timer.

Third, the server responds to the SYN on the server side, and all TCP packets (including SYN handshake requests sent by the client) go through the network card and soft interrupt to enter the tcp_v4_rcv. In this function, the socket currently in listen is found according to the destination IP information in the TCP header information of the network packet (skb). Then proceed to the tcp_v4_do_rcv handshake process.

/ / file: net/ipv4/tcp_ipv4.cint tcp_v4_do_rcv (struct sock * sk, struct sk_buff * skb) {/ / the server receives the first handshake SYN or the third step ACK will come here if (sk- > sk_state = = TCP_LISTEN) {struct sock * nsk = tcp_v4_hnd_req (sk, skb);} if (tcp_rcv_state_process (sk, skb, tcp_hdr (skb), skb- > len) {rsk = sk; goto reset) }} after determining whether the current socket is listen in tcp_v4_do_rcv, you will first go to tcp_v4_hnd_req to check the semi-connection queue. When the server responds to SYN for the first time, the semi-connection queue must be empty, so it is equivalent to returning without doing anything.

/ / file:net/ipv4/tcp_ipv4.cstatic struct sock * tcp_v4_hnd_req (struct sock * sk, struct sk_buff * skb) {/ / find the semi-connected queue of listen socket struct request_sock * req = inet_csk_search_req (sk, & prev, th- > source, iph- > saddr, iph- > daddr); return sk;} is processed differently according to different socket states in tcp_rcv_state_process.

/ file:net/ipv4/tcp_input.cint tcp_rcv_state_process (struct sock * sk, struct sk_buff * skb, const struct tcphdr * th, unsigned int len) {switch (sk- > sk_state) {/ / first handshake case TCP_LISTEN: if (th- > syn) {/ / it is judged to be a SYN handshake package. If (icsk- > icsk_af_ops- > conn_request (sk, skb)

< 0) return 1; ......}其中 conn_request 是一个函数指针，指向 tcp_v4_conn_request。服务器响应 SYN 的主要处理逻辑都在这个 tcp_v4_conn_request 里。 //file: net/ipv4/tcp_ipv4.cint tcp_v4_conn_request(struct sock *sk, struct sk_buff *skb){ //看看半连接队列是否满了 if (inet_csk_reqsk_queue_is_full(sk) && !isn) { want_cookie = tcp_syn_flood_action(sk, skb, "TCP"); if (!want_cookie) goto drop; } //在全连接队列满的情况下，如果有 young_ack，那么直接丢 if (sk_acceptq_is_full(sk) && inet_csk_reqsk_queue_young(sk) >

1) {NET_INC_STATS_BH (sock_net (sk), LINUX_MIB_LISTENOVERFLOWS); goto drop;} / assign request_sock kernel object req = inet_reqsk_alloc (& tcp_request_sock_ops); / / construct syn+ack package skb_synack = tcp_make_synack (sk, dst, req, fastopen_cookie_present (& valid_foc)? & valid_foc: NULL) If (likely (! do_fastopen)) {/ / send syn + ack response err = ip_build_and_send_pkt (skb_synack, sk, ireq- > loc_addr, ireq- > rmt_addr, ireq- > opt); / / add to the semi-connection queue and start the timer inet_csk_reqsk_queue_hash_add (sk, req, TCP_TIMEOUT_INIT) } else} here first determines whether the semi-connection queue is full, and if so, enter tcp_syn_flood_action to determine whether the tcp_syncookies kernel parameter is enabled. If the queue is full and tcp_syncookies is not turned on, the handshake packet will be discarded directly!

Then it is necessary to determine whether the full connection queue is full. Because the full connection queue can also lead to an abnormal handshake, it might as well be judged at the first handshake. If the full connection queue is full and there is a young_ack, it is also discarded directly.

Young_ack is a counter held in a semi-connected queue. Record the number of SYN that has just arrived, has not been retransmitted to SYN_ACK by the SYN_ACK retransmission timer, and has not completed a three-way handshake.

The next step is to construct the synack package and then send it out through ip_build_and_send_pkt.

Finally, the current handshake information is added to the semi-connection queue and the timer is started. The function of the timer is that if the third handshake from the client is not received within a certain period of time, the server will retransmit the synack packet.

To sum up, the main job of the server responding to ack is to determine whether the receiving queue is full, if it is full, the request may be discarded, otherwise synack will be issued. Apply for request_sock to be added to the semi-connection queue while starting the timer.

4. When the client responds to the SYNACK client receiving the synack packet sent by the server, it will also enter the tcp_rcv_state_process function. However, because the state of its own socket is TCP_SYN_SENT, it will go into a different branch.

/ / file:net/ipv4/tcp_input.c// except ESTABLISHED and TIME_WAIT The TCP processing in the state goes here int tcp_rcv_state_process (struct sock * sk, struct sk_buff * skb, const struct tcphdr * th, unsigned int len) {switch (sk- > sk_state) / / the server receives the first ACK packet case TCP_LISTEN: / / the client handles the second handshake case TCP_SYN_SENT: / / processes the synack packet queued = tcp_rcv_synsent_state_process (sk, skb, th, len) Return 0;} tcp_rcv_synsent_state_process is the main logic for the client to respond to synack.

/ file:net/ipv4/tcp_input.cstatic int tcp_rcv_synsent_state_process (struct sock * sk, struct sk_buff * skb, const struct tcphdr * th, unsigned int len) {tcp_ack (sk, skb, FLAG_SLOWPATH); / / connection establishment completed tcp_finish_connect (sk, skb) If (sk- > sk_write_pending icsk- > icsk_accept_queue.rskq_defer_accept icsk- > icsk_ack.pingpong) / / delayed confirmation else {tcp_send_ack (sk);}} tcp_ack ()-> tcp_clean_rtx_queue ()

/ / file: net/ipv4/tcp_input.cstatic int tcp_clean_rtx_queue (struct sock * sk, int prior_fackets, U32 prior_snd_una) {/ / delete sending queue. / / delete timer tcp_rearm_rto (sk);} / / file: net/ipv4/tcp_input.cvoid tcp_finish_connect (struct sock * sk, struct sk_buff * skb) {/ / modify socket status tcp_set_state (sk, TCP_ESTABLISHED) / / initialize the congestion control tcp_init_congestion_control (sk); / / Open the if (sock_flag (sk, SOCK_KEEPOPEN)) inet_csk_reset_keepalive_timer (sk, keepalive_time_when (tp));} the client changes its socket status to ESTABLISHED, and then turns on the TCP timer.

/ / file:net/ipv4/tcp_output.cvoid tcp_send_ack (struct sock * sk) {/ / apply and construct ack package buff = alloc_skb (MAX_TCP_HEADER, sk_gfp_atomic (sk, GFP_ATOMIC)); / / send out tcp_transmit_skb (sk, buff, 0, sk_gfp_atomic (sk, GFP_ATOMIC));} construct ack package in tcp_send_ack and send it out.

The client clears the retransmission timer set when the connect is cleared in response to the synack from the server, sets the current socket status to ESTABLISHED, and issues the ack confirmation of the third handshake after turning on the alive timer.

5. The server will also enter the tcp_v4_do_rcv when the ACK server responds to the ack of the third handshake

/ / file: net/ipv4/tcp_ipv4.cint tcp_v4_do_rcv (struct sock * sk, struct sk_buff * skb) {if (sk-sk_state = = TCP_LISTEN) {struct sock * nsk = tcp_v4_hnd_req (sk, skb);} if (tcp_rcv_state_process (sk, skb, tcp_hdr (skb), skb-len)) {rsk = sk; goto reset }} however, since this is the third handshake, the semi-connection information left by the last first handshake will be present in the semi-connection queue. So the execution logic of tcp_v4_hnd_req will be different.

/ / file:net/ipv4/tcp_ipv4.cstatic struct sock * tcp_v4_hnd_req (struct sock * sk, struct sk_buff * skb) {struct request_sock * req = inet_csk_search_req (sk, & prev, th-source, iph-saddr, iph-daddr); if (req) return tcp_check_req (sk, skb, req, prev, false) } inet_csk_search_req is responsible for searching in the semi-connected queue and returning a semi-connected request_sock object when it is found. And then go into the tcp_check_req.

/ file:net/ipv4/tcp_minisocks.cstruct sock * tcp_check_req (struct sock * sk, struct sk_buff * skb, struct request_sock * req, struct request_sock * * prev, bool fastopen) {/ / create sub-socket child = inet_csk (sk)-> icsk_af_ops- > syn_recv_sock (sk, skb, req, NULL); / / Clean up the semi-connection queue inet_csk_reqsk_queue_unlink (sk, req, prev) Inet_csk_reqsk_queue_removed (sk, req); / / add full connection queue inet_csk_reqsk_queue_add (sk, req, child); return child;} 5.1.Creating sub-socketicsk_af_ops- > syn_recv_sock corresponds to the tcp_v4_syn_recv_sock function.

/ / file:net/ipv4/tcp_ipv4.cconst struct inet_connection_sock_af_ops ipv4_specific = .conn _ request = tcp_v4_conn_request, .syn _ recv_sock = tcp_v4_syn_recv_sock,// three-way handshake is almost over Create the sock kernel object struct sock * tcp_v4_syn_recv_sock (struct sock * sk, struct sk_buff * skb, struct request_sock * req, struct dst_entry * dst) {/ / to determine whether the receiving queue is full of if (sk_acceptq_is_full (sk)) goto exit_overflow / / create sock & & initialize newsk = tcp_create_openreq_child (sk, req, skb); * * Note: in the third handshake, continue to determine whether the full connection queue is full. If it is full, modify the counter and discard it. * * if the queue is not satisfied, apply for the creation of a new sock object.

5.2 Delete the semi-connection queue to remove the connection request block from the semi-connection queue.

/ / file: include/net/inet_connection_sock.h static inline void inet_csk_reqsk_queue_unlink (struct sock * sk, struct request_sock * req, struct request_sock * * prev) {reqsk_queue_unlink (& inet_csk (sk)-icsk_accept_queue, req, prev);} reqsk_queue_unlink removes the connection request block from the semi-connection queue.

5.3 add a fully connected queue and then add it to the fully connected queue.

/ / file:net/ipv4/syncookies.cstatic inline void inet_csk_reqsk_queue_add (struct sock * sk, struct request_sock * req, struct sock * child) {reqsk_queue_add (& inet_csk (sk)-icsk_accept_queue, req, sk, child); insert the request_sock object with a successful handshake into the tail of the full connection queue list in reqsk_queue_add.

/ / file: include/net/request_sock.hstatic inline void reqsk_queue_add () {req-sk = child; sk_acceptq_added (parent); if (queue-rskq_accept_head = = NULL) queue-rskq_accept_head = req; else queue-rskq_accept_tail-dl_next = req; queue-rskq_accept_tail = req; req-dl_next = NULL Set the connection to ESTABLISHED//file:net/ipv4/tcp_input.cint tcp_rcv_state_process (struct sock * sk, struct sk_buff * skb, const struct tcphdr * th, unsigned int len) {switch (sk-sk_state) {/ / server third handshake processing case TCP_SYN_RECV: / / change the state to connection tcp_set_state (sk, TCP_ESTABLISHED);}} set the connection to TCP_ESTABLISHED state.

What the server does in response to the third handshake ack is to delete the current semi-connected object, create a new sock and join the full connection queue, and finally set the new connection status to ESTABLISHED.

Server accept the last accept step let's make a long story short.

/ / file: net/ipv4/inet_connection_sock.cstruct sock * inet_csk_accept (struct sock * sk, int flags, int * err) {/ / get struct request_sock_queue * queue = & icsk- > icsk_accept_queue; req = reqsk_queue_remove (queue) from the fully connected queue; newsk = req- > sk; return newsk;} reqsk_queue_remove is a simple operation, which is to retrieve the first element from the linked list of the fully connected queue and return it.

/ / file:include/net/request_sock.hstatic inline struct request_sock * reqsk_queue_remove (struct request_sock_queue * queue) {struct request_sock * req = queue-rskq_accept_head; queue-rskq_accept_head = req-dl_next; if (queue-rskq_accept_head = = NULL) queue-rskq_accept_tail = NULL; return req;} therefore, the focus of accept is to take one of the established full connection queues and return them to the user process.

This paper summarizes that the frequency of three-way handshake is very high in the entry interview for related positions in the back-end. In fact, in the process of three-way handshake, it is not just the sending of a handshake packet and the flow of TCP status. It also includes many key technical points, such as port selection, connection queue creation and processing. Through today's article, we take a deep look at these internal operations in the kernel during the three-way handshake.

The full text is full of tens of thousands of words, in fact, it can be summed up in a picture.

1. When the server listen, the length of the full / semi-connected queue is calculated, and the relevant memory is requested and initialized.

two。 When the client connect, set the local socket status to TCP_SYN_SENT, select an available port, issue the SYN handshake request and start the retransmission timer.

3. When the server responds to the ack, it determines whether the receiving queue is full, and if so, the request may be discarded. Otherwise, issue a synack, apply for request_sock to be added to the semi-connection queue, and start the timer at the same time.

4. When the client responds to synack, it clears the retransmission timer set when connect, sets the current socket status to ESTABLISHED, and issues the ack confirmation of the third handshake after turning on the alive timer.

5. When the server responds to the ack, it deletes the corresponding semi-connected object, creates a new sock, adds it to the full connection queue, and finally sets the new connection status to ESTABLISHED.

6. Accept takes one of the established full connection queues and returns it to the user process.

In addition, it should be noted that if packet loss occurs during the handshake (network problems, or connection queue overflow), the kernel will wait for the timer to expire and try again, and the retry interval is 1s, 2s, 4s in version 3.10. In some older versions, such as 2.6, the first retry time was 3 seconds. The maximum number of retries is controlled by tcp_syn_retries and tcp_synack_retries respectively.

If your online interface normally returns within dozens of milliseconds, but occasionally the occasional response such as 1 s or 3 s takes longer, then you need to check to see if there is a timeout retransmission of the handshake packet.

These are some of the more detailed internal operations in the three-way handshake. If you can tell the underlying logic of the kernel in front of the interviewer, I'm sure the interviewer will be impressed!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.