Shulou(Shulou.com)06/01 Report--

This article mainly introduces Linux how to achieve C thread pool, has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let the editor take you to understand it.

Multithreaded programming, create a thread, specify to complete a task, and wait for the thread to exit. Although it can meet programming requirements, when we need to create a large number of threads, we may consume a lot of CPU in the process of creating and destroying threads. Add a lot of expenses. For example: the copy of the folder, the response of WEB server.

Thread pools are used to solve a problem like this, reducing the overhead of frequently creating and destroying threads.

Thread pool technology: generally use pre-created thread technology, that is, to create a certain number of threads in advance. After these threads are created in advance, the "task queue" assumes that there are no tasks, then let these threads sleep, once there is a task, wake up the thread to execute the task, and when the task is finished, there is no need to destroy the thread. Until when you want to exit or shut down, then you call the destroy thread pool function to destroy the thread.

After completing the task, the thread does not destroy it, but automatically executes the next task. Moreover, when there are many tasks, you can have function interfaces to increase the number of threads, and when there are fewer tasks, you can have function interfaces to destroy some threads.

If the time it takes to create and destroy a thread is negligible compared to the time it takes to execute the task, then there is no need to use a thread pool in this case.

"Task queue" is a shared resource "mutually exclusive access"

A brief introduction to the use of Thread Pool under Linux C

A thread pool is essentially a data structure that requires a structure to describe it:

The implementation of the struct pthread_pool / / thread pool generally has the following member / / mutex, which is used to protect the "task queue" pthread_mutex_t lock; / / mutex / / thread condition variable indicating whether the "task queue" has a task pthread_cond_t cond; / / condition variable bool shutdown. / / indicates whether to exit the program bool: type false / true// task queue (linked list), pointing to the first task to point to / / all threads get tasks from the task list "shared resources" struct task * task_list; / / there are multiple threads in the thread pool, each thread has tid, and an array is needed to store tidpthread_t * tids / / malloc () / / the number of threads in service in the thread pool, the current number of threads unsigned int active_threads; / / the maximum number of tasks in the thread pool task queue unsigned int max_waiting_tasks; / / how many tasks are currently on the thread pool task queue unsigned int cur_waiting_tasks; / /.} / / the task node above the task queue (linked list), as long as it can describe a task, / / the thread will constantly fetch the task struct task / / task node {/ / 1. The task represented by the task node. The "function pointer" points to the function (cp_file) void* (* do_task) (void* arg) to be executed by the task; / / 2. Pointer to the parameter of the task pointing to the function (file descriptor) void * arg; / / 3. A pointer to the task node type, pointing to the next task struct task * next;}

The thread pool framework code is as follows, and the function is self-filled:

Function interfaces needed to operate thread pool: pthread_pool.c, pthread_pool.h

Think of the "thread pool" as an outsourcing company, and all you need to do is manipulate the function interfaces provided by the thread pool.

Pthread_pool.c

# include "pthread_pool.h" / * init_pool: thread pool initialization function. There are thread_num initial threads @ pool in the specified thread pool: pointer Point to the thread pool you want to initialize @ threa_num: number of threads started in the thread pool you want to initialize return value: success 0 failed-1*/int init_pool (pthread_pool * pool, unsigned int threa_num) {/ / initialize the structure of the thread pool / / initialize thread mutex pthread_mutex_init (& pool- > lock, NULL) / / initialize thread condition variables pthread_cond_init (& pool- > cond, NULL); pool- > shutdown = false; / / do not exit pool- > task_list = (struct task*) malloc (sizeof (struct task)); pool- > tids = (pthread_t *) malloc (sizeof (pthread_t) * MAX_ACTIVE_THREADS); if (pool- > task_list = = NULL | | pool- > tids = = NULL) {perror ("malloc memery error"); return-1;} pool- > task_list- > next = NULL / / how many threads are initialized at the beginning of the thread pool to serve pool- > active_threads = threa_num;// indicates the maximum number of tasks in the thread pool pool- > max_waiting_tasks = the current number of tasks in the task queue in the MAX_WAITING_TASKS;// thread pool pool- > cur_waiting_tasks = 0 tidint / create thread_num threads, and let the thread execute the task deployment function, / / record the task I = 0 for all threads For (I = 0; I tids) [I], NULL, routine, (void*) pool); if (ret! = 0) {perror ("create thread error"); return-1;} printf ("[% lu]: [% s] = = > tids [% d]: [% lu]", pthread_self (), _ FUNCTION__, I, pool- > tids [I]);} return 0;} / * routine: task provisioning function. All threads start to execute this function, which constantly removes the task node from the task queue of the thread pool to execute. The task node contains "function pointer" h "function parameter" * / void * routine (void * arg) {/ / arg indicates that your thread pool pointer while () {/ / acquires thread mutex, lock / / when the thread pool does not end, constantly remove the node / / from the thread pool task queue to execute. / / release thread mutex, unlock / / release task node}} / * destroy_pool: destroy the thread pool and ensure that all tasks have been completed * / int destroy_pool (pthread_pool * pool) {/ / release all space to wait for task execution (join) before termination. / / Wake up all threads / / use the join function to reclaim each thread resource. } / * add_task: add tasks to the task queue, save the tasks pointed to by do_task (function pointer) and the parameters pointed to by arg to a task node, and add them to the pool task queue. @ pool: thread pool to which you want to add tasks @ do_task: tasks you need to add (cp_file) @ arg: parameters of the task you want to perform (file descriptor) * / int add_task (pthread_pool * pool,void* (* do_task) (void* arg) Void*arg) {/ / encapsulate the second parameter and the third parameter as struct task / / then add it to the pool- > task task queue / / Note that the task queue is a shared resource / / if you want to wake up the waiting thread after the task. } / / if there are too many tasks, add pthread_createint add_threads (pthread_pool * pool, unsigned int num) to the thread pool {/ / newly created num threads, let each thread execute the thread provisioning function / / add each newly created thread tid to pool- > tids} / / if there are few tasks, reduce the number of threads in the thread pool pthread_cancel joinint remove_threads (pthread_pool * pool, unsigned int num) {/ / cancel num threads with pthread_cancel / / use the pthread_join function to recover resources. }

Pthread_pool.h

# ifndef _ _ PTHREAD_POOL_H__#define _ _ PTHREAD_POOL_H__// indicates the maximum number of threads in the thread pool # define MAX_ACTIVE_THREADS 20 / indicates the maximum number of tasks in the thread pool # define MAX_WAITING_TASKS 1024 / the task node above the task queue (linked list), as long as you can describe a task / / Thread will continuously fetch task struct task / / task node {/ / 1. The task represented by the task node. The "function pointer" points to the function (cp_file) void* (* do_task) (void* arg) to be executed by the task; / / 2. Pointer to the parameter of the task pointing to the function (file descriptor) void * arg; / / 3. The pointer of the task node type to the next task struct task * next;}; the implementation of struct pthread_pool / / thread pool {/ / generally has the following member / / mutex, which is used to protect the "task queue" pthread_mutex_t lock; / / mutex / / thread condition variable indicates whether the "task queue" has a task pthread_cond_t cond; / / condition variable bool shutdown. / / indicates whether to exit the program bool: type false / true// task queue (linked list), pointing to the first task to point to / / all threads get tasks from the task list "shared resources" struct task * task_list; / / there are multiple threads in the thread pool, each thread has tid, and an array is needed to store tidpthread_t * tids / / malloc () / / the number of threads in service in the thread pool, the current number of threads unsigned int active_threads; / / the maximum number of tasks in the thread pool task queue unsigned int max_waiting_tasks; / / how many tasks are currently on the thread pool task queue unsigned int cur_waiting_tasks; / /.} / * init_pool: thread pool initialization function. There are thread_num initial threads @ pool in the thread pool specified by initialization: pointer to the thread pool you want to initialize @ threa_num: the number of threads started in the thread pool you want to initialize. Return value: success 0 failed-1*/int init_pool (pthread_pool * pool, unsigned int threa_num); / * routine: task allocation function. All threads start to execute this function, which constantly removes the task node from the task queue of the thread pool to execute. The task node contains "function pointer" h "function parameter" * / void * routine (void * arg); / * destroy_pool: destroy the thread pool and ensure that all tasks have been completed * / int destroy_pool (pthread_pool * pool) before termination; / * add_task: add tasks to the task queue, save the tasks pointed to by do_task (function pointer) and the parameters pointed by arg to a task node, and add them to the pool task queue. @ pool: thread pool to which you want to add task @ do_task: task you need to add (cp_file) @ arg: parameter (file descriptor) * / int add_task (pthread_pool * pool,void* (* do_task) (void*arg), void*arg) of the task you want to execute; / / if there are many tasks, add thread pthread_createint add_threads (pthread_pool * pool, unsigned int num) to the thread pool / if there are few tasks, reduce the number of threads in the thread pool pthread_cancel joinint remove_threads (pthread_pool * pool, unsigned int num); # endif thank you for reading this article carefully. I hope the article "Linux how to implement C thread pool" shared by the editor will be helpful to you. At the same time, I hope you will support and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.