Shulou(Shulou.com)06/02 Report--

This article focuses on "how to use Python multiplexing selector module", interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how to use Python multiplexing selector module.

1. IO multiplexing

O multiplexing technology uses a middleman who can monitor multiple IO blocking at the same time to monitor these different IO objects. If any one or more of these monitored IO objects have a message return, it will trigger the middleman to return these textual IO objects to get their messages.

The advantage of using IO multiplexing is that a process can also handle multiple IO blocking at the same time in a single thread. Compared with the traditional multi-thread / multi-process model, the cost of Icano multiplexing system is lower, and the system does not need to create new processes or threads, nor does it need to maintain the operation of these processes and threads, which reduces the maintenance workload of the system and saves system resources.

Python provides selector module to implement IO multiplexing. At the same time, the types of options for the middleman are different on different operating systems, such as epoll, kqueue, devpoll, poll,select, etc.; the implementation of kqueue (BSD,mac support), devpoll (solaris support) and epoll is basically the same, epoll is implemented in the Linux2.5+ kernel, Windows system only implements select.

1.1. Comparison of epoll,poll and select

Select and poll use polling to detect whether all monitored IO have data return, and need to constantly traverse each IO object, which is a time-consuming operation and inefficient. The advantage of poll over select is that select limits the maximum number of monitoring IO to 1024, which is obviously not enough for servers that require a large number of network IO connections; poll has no limit on this number. But this also faces a problem, when using polling to monitor these IO, the higher the number of IO, the more time is consumed per poll. The less efficient it is, which is a problem that cannot be solved by polling.

Epoll was born to solve this problem. First of all, he does not have the maximum number of monitored IO, and does not use polling to detect these IO, but uses the event notification mechanism and callback to get these IO objects returned with messages. Only the "active" IO will actively call the callback function. This IO will be processed directly without polling.

2. The basic selector module uses import selectorsimport socket# to create a socketIO object. After listening, it will be able to accept the request message: sock = socket.socket () sock.bind (("127.0.0.1", 80)) sock.listen () slt = selectors.DefaultSelector () # add this socketIO object to the socketIO object using the system default selector,Windows for select,linux and epoll# Monitoring slt.register in select (fileobj=sock, events=selectors.EVENT_READ, data=None) # Loop processing message while True: # select method: poll the selector. When at least one IO object has a message returned, it will return the IO object with the message ready_events = slt.select (timeout=None) print (ready_events) # prepared IO objects break

Ready_events is a list (representing all data-acceptable IO objects registered in this select), and each tuple in the list is:

SelectorKey object:

Fileobj: registered socket object

Fd: file descriptor

Data: the parameter we pass in during registration, which can be any value, is bound to an attribute for later use.

Mask value

EVENT_READ: indicates readable; its value is actually 1

EVENT_WRITE: means writable; its value is actually 2

Or a combination of both.

For example:

[(SelectorKey (fileobj=, fd=456, events=1, data=None)

1)]

To process the request, you only need to use the socket corresponding method, the socket is used to receive the request connection, and the accept method can be used to process the request.

After accepting the request, a new client will be generated, and we will put it into the selector to monitor it. When there is a message, if it is a connection request, the handle_request () function handles it, and if it is a message from the client, the handle_client_msg () function handles it.

There are two types of socket in select, so we need to determine which socket is returned after activation, and then call different functions to make different requests. If there are many kinds of socket in this select, it will not be possible to judge in this way. The solution is to bind the handler to the corresponding selectkey object, using the data parameter.

Def handle_request (sock:socket.socket, mask): # handle new connection conn, addr = sock.accept () conn.setblocking (False) # set non-blocking slt.register (conn, selector.EVENT_READ, data=handle_client_msg) def handle_client_msg (sock:socket.socket Mask) # processing messages data= sock.recv () print (data.decode ()) sock = socket.socket () sock.bind (("127.0.0.1", 80)) sock.listen () slt = selectors.DefaultSelector () slt.register (fileobj=sock, events=selectors.EVENT_READ, data=handle_request) while True: ready_events = slt.select (timeout=None) for event, mask in ready_events: event.data (event.fileobj Mask) # different socket have different data functions Use your own bound data function call, and then take your own socket as a parameter. You can handle different types of socket.

The above uses data to solve the above problem very well, but it should be noted that the functions bound to the data attribute (or callable object) will eventually be called in the way of event.data (event.fileobj), and these functions should take the same arguments.

At this point, I believe you have a deeper understanding of "how to use Python multiplexing selector module". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.