Shulou(Shulou.com)06/01 Report--

Linux system thread pool example analysis, in view of this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible method.

Before the implementation principle program starts, create a certain number of threads, put them in an idle queue, and initialize the thread pool.

These threads are in a blocking state, taking up only a little bit of memory and not cpu.

When the task arrives, an idle thread is taken from the thread pool and the task is passed into the thread to run.

When all threads are working on tasks, the thread pool automatically creates a certain number of new threads to handle more tasks.

The thread that finishes the task does not quit, but continues to wait in the online pool for the next task.

However, when most of the threads are blocked, the thread pool will automatically destroy some of the threads and reclaim the system resources.

Component thread manager

Used to create and manage thread pools.

Worker thread

The thread in the thread pool that actually executes the task. When initializing threads, a fixed number of threads are pre-created in the pool. These initialized threads are generally idle, do not take up CPU, and take up less memory space.

Task interface

The interface that each task must implement, when there is an executable task in the task queue of the thread pool, it is transferred to execute by the idle worker thread, and the task is abstracted to form an interface, so that the thread pool has nothing to do with the specific task.

Task queue

Used to store unprocessed tasks, providing a buffering mechanism

There are several ways to realize this structure, the common one is queue, which mainly uses the first-in-first-out principle, and the other is a data structure such as linked list, which can allocate memory space dynamically and is more flexible in application.

The simple implementation program consists of three files, namely thread_pool.h, thread_pool.c and test.c.

Thread_pool.h is as follows:

# include struct job {void * (* callback_function) (void * arg); void * arg; struct job * next;}; struct threadpool {int thread_num; int queue_max_num; struct job * head; struct job * tail; pthread_t * pthreads; pthread_mutex_t mutex; pthread_cond_t queue_empty; pthread_cond_t queue_not_empty; pthread_cond_t queue_not_full Int queue_cur_num; int queue_close; int pool_close;}; struct threadpool * threadpool_init (int thread_num, int queue_max_num); int threadpool_add_job (struct threadpool * pool, void * (* callback_function) (void * arg), void * arg); int threadpool_destroy (struct threadpool * pool); void * threadpool_function (void * arg); 123456789101112131415161718192021222324252627282930thread_pool.c as follows:

# include # include "thread_pool.h" struct threadpool * threadpool_init (int thread_num, int queue_max_num) {struct threadpool * pool = NULL; do {pool = (struct threadpool *) calloc (1, sizeof (struct threadpool)); if (! pool) {printf ("calloc error:% m\ n"); break } pool- > thread_num = thread_num; pool- > queue_max_num = queue_max_num; pool- > queue_cur_num = 0; pool- > head = NULL; pool- > tail = NULL; if (pthread_mutex_init (& (pool- > mutex), NULL)) {printf ("init mutex error:% m\ n"); break } if (pthread_cond_init (& (pool- > queue_empty), NULL) {printf ("init queue_empty error:% m\ n"); break;} if (pthread_cond_init (& (pool- > queue_not_empty), NULL)) {printf ("init queue_not_empty error:% m\ n"); break } if (pthread_cond_init (& (pool- > queue_not_full), NULL) {printf ("init queue_not_full error:% m\ n"); break;} pool- > pthreads = calloc (1, sizeof (pthread_t) * thread_num); if (! pool- > pthreads) {printf ("calloc pthreads error:% m\ n") Break;} pool- > queue_close = 0; pool- > pool_close = 0; int i; for (I = 0; I thread_num; iTunes +) {pthread_create (& (pool- > pthreads [I]), NULL, threadpool_function, (void *) pool);} return pool;} while (0); return NULL } int threadpool_add_job (struct threadpool * pool, void * (* callback_function) (void * arg), void * arg) {assert (pool! = NULL); assert (callback_function! = NULL); assert (arg! = NULL); pthread_mutex_lock (& (pool- > mutex)) While ((pool- > queue_cur_num = = pool- > queue_max_num) & & (pool- > queue_close | | pool- > pool_close)) {pthread_cond_wait (& (pool- > queue_not_full), & (pool- > mutex));} if (pool- > queue_close | | pool- > pool_close) {pthread_mutex_unlock (& (pool- > mutex)); return-1 } struct job* pjob = (struct job*) calloc (1, sizeof (struct job)); if (! pjob) {pthread_mutex_unlock (& (pool- > mutex)); return-1;} pjob- > callback_function = callback_function; pjob- > arg = arg; pjob- > next = NULL; if (pool- > head = = NULL) {pool- > head = pool- > tail = pjob Pthread_cond_broadcast (& (pool- > queue_not_empty));} else {pool- > tail- > next = pjob; pool- > tail = pjob;} pool- > queue_cur_num++; pthread_mutex_unlock (& (pool- > mutex)); return 0;} void * threadpool_function (void * arg) {struct threadpool * pool = (struct threadpool *) arg; struct job * pjob = NULL While (1) {pthread_mutex_lock (& (pool- > mutex)); while ((pool- > queue_cur_num = = 0) & &! pool- > pool_close) {pthread_cond_wait (& (pool- > queue_not_empty), & (pool- > mutex));} if (pool- > pool_close) {pthread_mutex_unlock (& (pool- > mutex)) Pthread_exit (NULL);} pool- > queue_cur_num--; pjob = pool- > head; if (pool- > queue_cur_num = = 0) {pool- > head = pool- > tail = NULL;} else {pool- > head = pjob- > next } if (pool- > queue_cur_num = = 0) {pthread_cond_signal (& (pool- > queue_empty));} if (pool- > queue_cur_num = = pool- > queue_max_num-1) {pthread_cond_broadcast (& (pool- > queue_not_full));} pthread_mutex_unlock (& (pool- > mutex)) (* (pjob- > callback_function)) (pjob- > arg); free (pjob); pjob = NULL;}} int threadpool_destroy (struct threadpool * pool) {assert (pool! = NULL); pthread_mutex_lock (& (pool- > mutex)); if (pool- > queue_close | | pool- > pool_close) {pthread_mutex_unlock (& (pool- > mutex)); return-1 } pool- > queue_close = 1; while (pool- > queue_cur_num! = 0) {pthread_cond_wait (& (pool- > queue_empty), & (pool- > mutex));} pool- > pool_close = 1; pthread_mutex_unlock (& (pool- > mutex)); pthread_cond_broadcast (& (pool- > queue_not_empty)); pthread_cond_broadcast (& (pool- > queue_not_full)) Int i; for (I = 0; I thread_num; iTunes +) {pthread_join (pool- > pthreads [I], NULL);} pthread_mutex_destroy (& (pool- > mutex)); pthread_cond_destroy (& (pool- > queue_empty)); pthread_cond_destroy (& (pool- > queue_not_empty)); pthread_cond_destroy (& (pool- > queue_not_full)); free (pool- > pthreads) Struct job * p; while (pool- > head! = NULL) {p = pool- > head; pool- > head = p-> next; free (p);} free (pool); return 0;} 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166test.c is used for testing, as follows:

# include # include "thread_pool.h" void* work (void* arg) {char* p = (char*) arg; printf ("threadpool callback fuction:% s.\ n", p); sleep (1);} int main (void) {struct threadpool * pool = threadpool_init (10,20); threadpool_add_job (pool, work, "1"); threadpool_add_job (pool, work, "2") Threadpool_add_job (pool, work, "3"); threadpool_add_job (pool, work, "4"); threadpool_add_job (pool, work, "5"); threadpool_add_job (pool, work, "6"); threadpool_add_job (pool, work, "7"); threadpool_add_job (pool, work, "8"); threadpool_add_job (pool, work, "9") Threadpool_add_job (pool, work, "10"); threadpool_add_job (pool, work, "11"); threadpool_add_job (pool, work, "12"); threadpool_add_job (pool, work, "13"); threadpool_add_job (pool, work, "14"); threadpool_add_job (pool, work, "15"); threadpool_add_job (pool, work, "16") Threadpool_add_job (pool, work, "17"); threadpool_add_job (pool, work, "18"); threadpool_add_job (pool, work, "19"); threadpool_add_job (pool, work, "20"); threadpool_add_job (pool, work, "21"); threadpool_add_job (pool, work, "22"); threadpool_add_job (pool, work, "23") Threadpool_add_job (pool, work, "24"); threadpool_add_job (pool, work, "25"); threadpool_add_job (pool, work, "26"); threadpool_add_job (pool, work, "27"); threadpool_add_job (pool, work, "28"); threadpool_add_job (pool, work, "29"); threadpool_add_job (pool, work, "30") Threadpool_add_job (pool, work, "31"); threadpool_add_job (pool, work, "32"); threadpool_add_job (pool, work, "33"); threadpool_add_job (pool, work, "34"); threadpool_add_job (pool, work, "35"); threadpool_add_job (pool, work, "36"); threadpool_add_job (pool, work, "37") Threadpool_add_job (pool, work, "38"); threadpool_add_job (pool, work, "39"); threadpool_add_job (pool, work, "40"); sleep (5); threadpool_destroy (pool); return 0;} 1234567891011121314151718192022325262728293031323435363839404143444648495015253545556575859 execute the following compilation command:

The running result of the program is as follows:

->%. / a.out threadpool callback fuction: 1. Threadpool callback fuction: 7. Threadpool callback fuction: 8. Threadpool callback fuction: 2. Threadpool callback fuction: 3. Threadpool callback fuction: 4. Threadpool callback fuction: 5. Threadpool callback fuction: 6. Threadpool callback fuction: 9. Threadpool callback fuction: 10. Threadpool callback fuction: 12. Threadpool callback fuction: 16. Threadpool callback fuction: 11. Threadpool callback fuction: 18. Threadpool callback fuction: 14. Threadpool callback fuction: 15. Threadpool callback fuction: 17. Threadpool callback fuction: 13. Threadpool callback fuction : 19. Threadpool callback fuction: 20. Threadpool callback fuction: 21. Threadpool callback fuction: 23. Threadpool callback fuction: 24. Threadpool callback fuction: 22. Threadpool callback fuction: 26. Threadpool callback fuction: 27. Threadpool callback fuction: 28. Threadpool callback fuction: 25. Threadpool callback fuction: 29. Threadpool callback fuction: 30. Threadpool callback fuction: 31. Threadpool callback fuction: 32. Threadpool callback fuction: 33. Threadpool callback fuction: 34. Threadpool callback fuction: 35. Threadpool callback fuction: 36. Threadpool callback fuction: 37. Threadpool callback fuction: 38. Threadpool callback fuction: 39. Threadpool callback fuction: 40. This is the answer to the sample analysis question on the thread pool of the Linux system. I hope the above content can be of some help to you. If you still have a lot of doubts to be solved, you can follow the industry information channel to learn more about it.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.