Shulou(Shulou.com)06/03 Report--

This article mainly shows you the "sample analysis of communication based on tcp protocol in python", which is easy to understand and clear. I hope it can help you solve your doubts. Let me lead you to study and learn the article "sample analysis of communication based on tcp protocol in python".

Read the catalogue

Tcp protocol: streaming protocol (communicating and transmitting in the form of data stream), security protocol (sending and receiving messages need to receive acknowledgements before they can be sent and received, which is a two-way channel communication)

Tcp protocol belongs to the transport layer in the OSI seven-layer protocol. It carries on the data transceiver of the user layer and downstarts the network layer, the data link layer and the physical layer. It can be said that a lot of secure data transmission and communication are based on tcp protocol.

In order to make tcp communication more convenient, we need to introduce a socket module (the module encapsulated by network layer, data link layer and physical layer). We only need to call the relevant interfaces in the module to realize the tedious operation under the transport layer.

Simple tcp protocol communication template: (requires a server and a client)

Server: from socket import * # determines the server-side transport protocol ↓ server = socket (AF_INET, SOCK_STREAM) # where SOCK_STREAM represents streaming protocol TCP, and if SOCK_DGRAM represents UDP protocol # fixed server IP and PORT Enable the client to access the server ↓ server.bind (('127.0.0.1, 8080)) # (' 127.0.0.1, 8080) through IP and port, where IP and PORT must be passed in tuple form Local access Local IP defaults to '127.0.0.1' # set the number of semi-connection pools (usually 5) server.listen (5) # semi-connection pooling: the container for the number of client connection requests. A maximum of 5 client connection requests will enter the queued state before the sending and receiving of the current connected client information is completed. # after the last communication is completed, you can connect to start the communication. # the two-way channel has been established successfully, and the next step of data communication can be carried out. ↓ conn, client_addr = server.accept () # send and receive messages data = conn.recv (1024) # receive a maximum of 1024 bytes each time, and the received data is of binary Bytes type conn.send (data.upper ()) # the received data is processed and the new data is returned Feedback to the client (send data to the client), the data type must also be Bytes type # A round of information has been sent and received Close the established two-way channel conn.close () client: from socket import * # determines the client transport protocol ↓ (the server can communicate effectively just like the client service protocol) client = socket (AF_INET, SOCK_STREAM) # where SOCK_STREAM stands for streaming protocol TCP If it is SOCK_DGRAM, it means UDP protocol # starts to connect server IP and PORT, and establishes a two-way link client.connect (('127.0.0.1, 8080)) # connects through server IP and PORT. After this step, the connection has been established, and then the data communication begins: client.send (' hello) Server'.encode ('utf-8')) # transcodes the sent information into Bytes type data data = client.recv (1024) # the maximum size of each received data is 1024 bytes (1kb) print (data.decode (' utf-8')) # convert type b data to string format # after one transmission client.close () # closes the client connection startup server (server starts Monitor the connection request of the client) start the client (the client sends a connection request to the server) establish a two-way link to complete the client to send information to the server hello The server server receives the hello,server, converts it to uppercase, and returns it to the client (when the server has finished a round of communication) the client receives the feedback from the server and prints out the HELLO,SERVER (when the client has finished a round of communication)

The above is the most basic communication process based on tcp protocol, the client sends, the server receives, the server processes the data and then sends, and the client receives the feedback data sent by the server.

The communication sticky packet problem of TCP protocol:

However, because tcp protocol is a streaming protocol, streaming protocol will have a characteristic: data transmission is like a trickle. When we receive data, the default maximum received data size is 1024 bytes. When the data sent is less than 1024 bytes, of course, there will be no problem. All at once, but when the data sent is more than 1024 bytes. We do not know the size of the data sent. When we can only default to 1024 bytes, it is impossible to collect all the data at once. We can only collect 1024 bytes this time. What about data other than 1024 bytes? Because the transmission of the data is a streaming protocol, the uncollected data will wait outside the door in turn, waiting for you to receive the data again the next time you receive the data, so that if the size of the data transmitted each time is not confirmed, and when the data is received, it will lead to an endless collection of data each time, and the endless data will be queued in the cache for the next time. The endless collection of data is like zhan nian. This is called the communication sticky packet problem of tcp streaming protocol.

A more vivid process of this problem can be seen in the following figure:

Knowing the general process of this sticking bag, we can find a way to prescribe the right medicine to the case:

Analysis of the solution to the sticky package problem:

In the final analysis, the sticky packet problem is caused by incomplete data reception, so the most direct way to solve this problem is to collect the data thoroughly every time.

In order to achieve this goal, we need to know in advance the file size of the data I want to receive before receiving the data. knowing the file size, we can aim at the target and accurately collect the data without leaving behind.

Solution: first send a header file containing the size and length of the file to be sent > then send the original file

Introduction of module struct

Look at the code:

Server: import socketimport structserver = socket.socket () server.bind (('127.0.0.1, 8080)) server.listen (5) while True: conn, client_addr = server.accept () print (' client connected') while True: try: head = conn.recv (4) size = struct.unpack ('iTunes, head) [0] data = conn.recv (size) print (' received client message:' Data.decode ('utf-8') except ConnectionResetError: print (' client disconnected') conn.close () break client: import socketimport structwhile True: try: client = socket.socket () client.connect (('127.0.0.1') 8080) print ('connected to the server') while True: try: msg = 'abcdefghijklmnopqrstuvwxyz1234567890'.encode (' utf-8') head = struct.pack ('i') Len (msg)) client.send (head) client.send (msg) except ConnectionResetError: print ('server disconnected') client.close () break except ConnectionRefusedError: print ('unable to connect to server')

The above methods are only for the purpose of experimenting to solve the sticky package problem. The real application scenario can be when uploading or downloading a large file. At this time, you must know the actual size of the received file in advance and receive each data accurately. At this time, you need to obtain the size of the file to be received before receiving the data, and then prescribe the right medicine to achieve accurate reception, but the implementation method does not have to use the struct module. Struct module is only an official way to solve the sticky package problem, and you can also have your own ideas, such as directly sending the size of the file to be sent in a string format, and then sending this file for only one purpose, knowing the size of the file I received, and accurately receiving the file.

The following is an example of a client downloading a file from the server for your reference: (assuming the download file is at the same level as the server file)

Download server: import socketimport timeimport structimport json# calculates the MD5 value and size import os of the files under the current folder, hashlibdef get_info (file_name): file_info = {} base_dir = os.path.dirname (_ _ file__) file_dir = os.path.join (base_dir, file_name) if os.path.exists (file_dir): # md5 calculates the file data in memory, when we calculate a large file You can use the update method for step-by-step calculation. # add some file data for calculation each time to reduce the memory footprint. With open (file_dir 'rb') as f: le = 0d5 = hashlib.md5 () for line inf: le + = len (line) d5.update (line) file_info [' lenth'] = le # add the file length to the header dictionary file_md5 = d5.hexdigest () file_info ['md5'] = file_md5 # add the file md5 to the header dictionary file_ Size = os.path.getsize (file_dir) / float (1024 * 1024) file_info ['size (MB)'] = round (file_size) 2) # add the file size to the header dictionary return file_info else: return file_infoserver = socket.socket () server.bind (('127.0.0.1, 8080)) server.listen (5) while True: conn, client_addr = server.accept () print ('% s >: client (% s) connected'% (time.strftime ('% Y-%m-%d% HGV% MRV% S')) Client_addr) while True: try: download_filename = conn.recv (1024). Decode ('utf-8') download_file_info_dic = get_info (download_filename) j_head = json.dumps (download_file_info_dic) # convert the file information dictionary to json string format head = struct.pack (' i') Len (j_head) conn.send (head) conn.send (j_head.encode ('utf-8')) if not download_file_info_dic: continue with open (download_filename) 'rb') as f: while True: data=f.read (1024) conn.send (data) # for line in f: # conn.send (line) except ConnectionResetError: print ('% s >: client (% s) disconnected'% (time.strftime ('% Y-%m-%d% HRV% MVO% S')) Client_addr) conn.close () break download client: import socketimport timeimport structimport json# progress bar shows def progress (percent,width=30): text= ('\ r [% -% ds]'% width)% ('x'*int (percent*width)) text=text+'%3s%%' text=text% (round (percent*100)) print (text) End='') while True: try: client = socket.socket () client.connect (('127.0.0.1') 8080) print ('% s >: connected to the server'% time.strftime ('% Y-%m-%d% HY-%m-%d% MVO% S')) while True: try: file_name = input ('Please enter the download file name:') client.send (file_name.encode ('utf-8')) head = client.recv (4) # receive header J_dic_lenth = struct.unpack ('i' Head) [0] # decompress the header Get the length of the file information dictionary in json format j_head = client.recv (j_dic_lenth) # receive the information dictionary in json format file_info_dic = json.loads (j_head) # deserialize json dictionary Get the file information dictionary if not file_info_dic: print ('file does not exist') continue file_lenth = file_info_dic.get ('lenth') file_size = file_info_dic.get (' size (MB)') file_md5 = file_info_dic.get ('md5') rec_len = 0 with open (' cpoy_'+file_name) 'wb') as f: while rec_len

< file_lenth: data = client.recv(1024) f.write(data) rec_len += len(data) per=rec_len/file_lenth progress(per) print() # print('下载比例：%6s %%'%) if not rec_len: print('文件不存在') else: print('文件[%s]下载成功: 大小：%s MB|md5值：[%s]' % (file_name, file_size, file_md5)) except ConnectionResetError: print('%s >

: server terminated'% time.strftime ('% Y-%m-%d% Hclient.close% MGV% S') client.close () break except ConnectionRefusedError: print ('% s >: unable to connect to the server'% time.strftime ('% Y-%m-%d% Hclient.close% MRV% S')

File upload is the same, except that the client sends the file to the server and the server receives it.

Next, let's learn the effect of using socketserver module to realize multi-client concurrent communication under TCP protocol.

Server: import socketserverimport timeclass MyTcpHandler (socketserver.BaseRequestHandler): # here means that the server has listened to a client connection request, handed the communication to a handle method to implement, and then listen to the customer connection request def handle (self): # establish a two-way channel Communicate with print ('% s | client% s connected'% (time.strftime ('% Y-%m-%d% HGV% MVD% S'), self.client_address)) while True: try: data = self.request.recv (1024) msg ='I have received your request [% s], thank you for your attention!' % data.decode ('utf-8') self.request.send (msg.encode (' utf-8') except ConnectionResetError: print ('% s | client% s has been disconnected'% (time.strftime ('% Y-%m-%d% HV% MV% S'), self.client_address)) breakif _ _ name__ = ='_ _ main__': server = socketserver.ThreadingTCPServer (('127.0.0.1') 8080), MyTcpHandler) # bind server IP and PORT And generate the concurrent method object print ('waiting for connection request.') Server.serve_forever () # the server always opens the client: from socket import * import timeserver_addr = ('127.0.0.1, 8080) count = 1while True: if count > 10: time.sleep (1) print ('% s | connection% s timeout'% (time.strftime ('% Y-%m-%d% Hrig% MRV% S'), server_addr) break try: client = socket (AF_INET) SOCK_STREAM) client.connect (('127.0.0.1, 8080)) count = 1 print ('% s | server% s connected successfully'% (time.strftime ('% Y-%m-%d% Hcount% MRV% S')) Server_addr) while True: try: client.send ('Bei' .encode ('utf-8')) data = client.recv (1024) print (data.decode (' utf-8')) time.sleep (0.5) except ConnectionResetError: print ('% s | server% s has been interrupted'% (time.strftime ('% Y-%m-%d% HVO% MVO% S')) Server_addr)) client.close () break except ConnectionRefusedError: print ('cannot connect to the server') count + = 1

At the same time, client 2 and client 3 are added to modify the sending data slightly to realize multi-client concurrent communication server.

Remote shell command line commands are realized through subprocess module.

Server import socketserverimport structimport subprocessclass MyTcpHandler (socketserver.BaseRequestHandler): def handle (self): while True: print ('client connected'% self.client_address) try: cmd = self.request.recv (1024). Decode ('utf-8') res = subprocess.Popen (cmd, shell=True, stdout=subprocess.PIPE) Stderr=subprocess.PIPE) stdout = res.stdout.read () stderr= res.stderr.read () head = struct.pack ('i' Len (stdout + stderr) self.request.send (head) self.request.send (stdout) self.request.send (stderr) except ConnectionResetError: print ('client disconnected'% self.client_address) self.request.close () breakif _ _ name__ ='_ _ main__': server = socketserver.ThreadingTCPServer (('127.0.0.1), 8080) MyTcpHandler) server.serve_forever () client from socket import * import structwhile True: try: client = socket (AF_INET,SOCK_STREAM) client.connect (('127.0.0.1, 8080)) while True: try: cmd = input (' >:). Strip (). Encode ('utf-8') client.send (cmd) head = client.recv (4) size = struct.unpack (' i' Head) [0] cur_size = 0 result = baked 'while cur_size < size: data = client.recv (1024) cur_size + = len (data) result + = data print (result.decode (' gbk')) # windows system default encoding is gbk Decoding must also use gbk except ConnectionResetError: print ('server has been interrupted') client.close () break except ConnectionRefusedError: print ('unable to connect to server')

Through the client input command, execute the shell command on the server side, and execute the subprocess module through the server side to achieve the remote shell command operation. This process mainly needs to consider two difficulties. ① solves the problem of sending and sticking packets of the result data generated by the command. ② notes that the result of the shell command that returns the result is encoded by gbk and needs to be decoded by gbk after receiving.

The above is all the contents of the article "sample Analysis of Communication based on tcp Protocol in python". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.