Shulou(Shulou.com)11/24 Report--

This article comes from the official account of Wechat: developing Internal skills practice (ID:kfngxl). Author: Zhang Yanfei allen

Echo "5000 65000" / proc/sys/net/ipv4/ip_local_port_range

Connect 1VR 192.168.1.101 5000 192.168.1.100 8090

Connect 2VR 192.168.1.101 5001 192.168.1.100 8090

Connect NRV 192.168.1.101... 192.168.1.100 8090

Connect 6W:192.168.1.101 65000 192.168.1.100 8090

/ / modify the file descriptor that can be opened by the whole system as 20Wecho 200000 > / proc/sys/fs/file-max//. Modify the file descriptor that can be opened by all users and each process is 20W#vi / etc/sysctl.conffs.nr_open=210000#sysctl-p#vi / etc/security/limits.conf* soft nofile 200000 * hard nofile 200000 Note: the hard limit in limits cannot exceed nr_open, so change nr_open first. And it is best to change it in sysctl.conf. Avoid restart when the hard limit takes effect, nr_open does not take effect, resulting in startup problems.

"there is a main data structure in socket, sock_common, in which there are two associations."

/ / file: include/net/sock.hstruct sock_common {union {_ _ addrpair skc_addrpair; / / TCP connection IP pair struct {_ _ be32 skc_daddr; _ be32 skc_rcv_saddr;}; union {_ _ portpair skc_portpair; / / TCP connection port pair struct {_ be16 skc_dport; _ u16 skc_num;};} } "where skc_addrpair records the IP pair in the TCP connection, and skc_portpair records the port pair."

"after the network packet arrives at the network card, it goes through DMA, hard interrupt, soft interrupt and so on, and is finally sent to the receiving queue of socket."

"for the TCP protocol, the entry function that the protocol handles is tcp_v4_rcv. Let's take a look at its code."

/ / file: net/ipv4/tcp_ipv4.cint tcp_v4_rcv (struct sk_buff * skb) {th = tcp_hdr (skb); / / get tcp header iph = ip_hdr (skb); / / get ip header sk = _ _ inet_lookup_skb (& tcp_hashinfo, skb, th- > source, th- > dest);}

/ / file: include/net/inet_hashtables.hstatic inline struct sock * _ inet_lookup (struct net * net, struct inet_hashinfo * hashinfo, const _ _ be32 saddr, const _ be16 sport, const _ be32 daddr, const _ be16 dport, const int dif) {u16 hnum = ntohs (dport); struct sock * sk = _ _ inet_lookup_established (net, hashinfo, saddr, sport, daddr, hnum, dif) Return sk?: _ _ inet_lookup_listener (net, hashinfo, saddr, sport, daddr, hnum, dif);} "determine whether there is a socket in the connection state first, which will go to the _ _ inet_lookup_established function"

Struct sock * _ _ inet_lookup_established (struct net * net, struct inet_hashinfo * hashinfo, const _ _ be32 saddr, const _ _ be16 sport, const _ be32 daddr, const u16 hnum, const int dif) {/ / spell the source port and destination port into a 32-bit int integer const _ _ portpair ports = INET_COMBINED_PORTS (sport, hnum) / / the kernel uses hash to speed up the search for socket unsigned int hash = inet_ehashfn (net, daddr, hnum, saddr, sport); unsigned int slot = hash & hashinfo- > ehash_mask; struct inet_ehash_bucket * head = & hashinfo- > ehash [slot]; begin: / / traverse the linked list one by one until sk_nulls_for_each_rcu (sk, node, & head- > chain) {if (sk- > sk_hash! = hash) continue is found If (likely (INET_MATCH (sk, net, acookie, saddr, daddr, ports, dif)) {if (unlikely (! atomic_inc_not_zero (& sk- > sk_refcnt) goto begintw; if (unlikely (! INET_MATCH (sk, net, acookie, saddr, daddr, ports, dif)) {sock_put (sk); goto begin;} goto out;}

/ include/net/inet_hashtables.h#define INET_MATCH (_ _ sk, _ _ net, _ _ cookie, _ _ saddr, _ _ daddr, _ _ ports _ _ dif)\ ((inet_sk (_ _ sk)-inet_portpair = = (_ _ ports)) & &\ (inet_sk (_ _ sk)-inet_daddr = = (_ _ saddr)) & &\ (inet_sk (_ _ sk)-inet_rcv_saddr = = (_ _ daddr)) & &\ (! (_ _ sk)-sk_bound_dev_if\ ((_ _ sk)-sk_bound_) Dev_if = (_ _ dif)) & &\ net_eq (sock_net (_ _ sk)) (_ _ net)) "compare _ _ saddr, _ _ daddr, _ _ ports in network packet tcp header and inet_portpair, inet_daddr, inet_rcv_saddr in socket in Linux in INET_MATCH. If it's a match, socket will find it. Of course, in addition to ip and port, INET_MATCH also compares some other things, so TCP also has quintuple, seven tuple and so on. "

# cat / etc/redhat-releaseRed Hat Enterprise Linux Server release 6.2 (Santiago) # ss-ant | grep ESTAB | wc-l100001blank cat / proc/meminfoMemTotal: 3925408 kBMemFree: 97748 kBBuffers: 35412 kBCached: 119600 kBSlab: 3241528 kB

It is concluded that each connection established by the client consumes a port, so many students feel nervous when they see that the number of connections on the client machine exceeds 3W or 5W, and they always feel that something is going wrong with the machine.

The first edition of this article was also written a long time ago, but Brother Fei took a long time to be satisfied. In this article, we show some of the kernel code of TCP socket. From the source code:

A TCP connection is a pair of socket on the client and server. They all record their ip pairs and port pairs (also known as quads) on their respective kernel objects, and use this to find each other when communicating.

The sender of the TCP connection will copy this information to the IP Header when sending the network packet. The network packet carries this token across the Internet to the destination server. The destination server kernel will match the token (quad) carried in the IP packet header to find the correct socket (connection).

In this process, we can see that the port of the client is only one yuan in this quad. Even if the two connections use the same port number, as long as the client ip is different, or the server is different, it will not affect the kernel to find the corresponding connection correctly, but will not cross the line!

So there are two ways to increase the maximum concurrency of TCP on the client side. The first way is to configure multiple ip for the client. The second way is to connect multiple different server.

However, it is best not to mix these two methods. Because when using multiple IP, the client needs bind. Once bind is done, the kernel will not choose the used port when establishing a connection. The bind function will change the kernel's policy of selecting ports ~ ~

Finally, our hands-on experiments have proved that the client can also break through the order of concurrency of millions. I believe that if you have read this article, you no longer have to be afraid of the number 65535.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.