Shulou(Shulou.com)06/02 Report--

This article is about how to understand Python thread pool, the editor thinks it is very practical, so I share it with you to learn. I hope you can get something after reading this article.

To sum up my own experience with Python thread pool, Python is an absolute choice for those who have not learned programming languages or have some knowledge of multiple development languages, and beginner programmers are advised to start programming from Python.

Import Queue, threading, sys from threading import Thread import time,urllib # working thread class Worker (Thread): worker_count = 0 def _ _ init__ (self, workQueue, resultQueue, timeout = 0, * * kwds): Thread.__init__ (self * * kwds) self.id = Worker.worker_count Worker.worker_count + = 1 self.setDaemon (True) self.workQueue = workQueue self.resultQueue = resultQueue self.timeout = timeout self.start () def run (self):''the get-some-work Do-some-work main loop of worker threads''while True: try: callable, args, kwds = self.workQueue.get (timeout=self.timeout) res = callable (* args, * * kwds) print "worker [% 2d]:% s"% (self.id Str (res) self.resultQueue.put (res) except Queue.Empty: break except: print 'worker [% 2d]'% self.id, sys.exc_info () [: 2] class WorkerManager: def _ init__ (self, num_of_workers=10) Timeout = 1): self.workQueue = Queue.Queue () self.resultQueue = Queue.Queue () self.workers = [] self.timeout = timeout self._recruitThreads (num_of_workers) def _ recruitThreads (self, num_of_workers): for i in range (num_of_workers): worker = Worker (self.workQueue, self.resultQueue Self.timeout) self.workers.append (worker) def wait_for_complete (self): #... then Wait for each of them to terminate: while len (self.workers): worker = self.workers.pop () worker.join () if worker.isAlive () and not self.workQueue.empty (): self.workers.append (worker) print "All jobs are are completed." Def add_job (self, callable, * args, * * kwds): self.workQueue.put ((callable, args, kwds)) def get_result (self, * args, * * kwds): return self.resultQueue.get (* args, * * kwds)

The Worker class is a pool of Python threads that constantly fetches tasks that need to be performed from the workQueue queue, executes them, and writes the results to resultQueue. The workQueue and resultQueue here are both off-the-shelf security, and the operations of each thread are mutually exclusive internally. When the get task from workQueue times out, the thread ends.

WorkerManager is responsible for initializing the Python thread pool, providing an interface for queuing tasks and getting results, and waiting for all tasks to complete. A typical test example is as follows, it uses 10 threads to download the content of a fixed page, the actual application should be to perform different tasks.

Def test_job (id, sleep = 0.001): try: urllib.urlopen ('[url] https://www.gmail.com/[/url]').read() except: print'[% 4d]'% id Sys.exc_info () [: 2] return id def test (): import socket socket.setdefaulttimeout (10) print 'start testing' wm = WorkerManager (10) for i in range (500): wm.add_job (test_job, I, item0.001) wm.wait_for_complete () print' end testing' above is how to understand Python thread pool The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.