Shulou(Shulou.com)05/31 Report--

Most people do not understand the knowledge points of this article, "what are the two ways for Go to achieve thread pool?", so the editor summarizes the following content, detailed content, clear steps, and has a certain reference value. I hope you can get something after reading this article. Let's take a look at this article "what are the two ways to achieve thread pool in Go?"

Introduction to worker pool

Worker pool is actually thread pool thread pool. For go, goroutine is used directly instead of threads, but thread pools are still interpreted in terms of threads.

In the thread pool model, there are two queues and one pool: task queue, completed task queue, and thread pool. The completed task queue may or may not exist, depending on the actual needs.

Whenever a task comes in, it will be put in the task queue. As soon as the thread has finished executing a task, the task is placed in the completed task queue, and sometimes the processing results of the task are also placed in the completed queue.

Worker pool contains a bunch of threads (worker, for go, each worker is a goroutine), waiting to be fed, waiting to assign tasks to them, or going to the task queue to fetch tasks themselves. After getting the task, update the task queue, then execute the task, and put the completed task into the completed queue.

The following figure is from wiki:

There are two ways to implement work pools in Go: traditional mutexes and channel.

The working pool of the traditional mutex mechanism

Suppose the task in Go is defined in the form:

Type Task struct {...}

Every time a task comes in, it is placed in the task queue.

Using the traditional mutex method, the definition structure of the task queue is roughly as follows:

Type Queue struct {M sync.Mutex Tasks [] Task}

Then add Lock () and Unlock () to the function that performs the task. For example:

Func Worker (queue * Queue) {between for {/ / Lock () and Unlock () is critical section queue.M.Lock () / / fetch task task: = queue.Tasks [0] / / update task queue queue.Tasks = queue.Tasks [1:] Queue.M.Unlock () / / execute the task process (task)} in this goroutine

Suppose 100 goroutine are activated in the thread pool to execute Worker (). Lock () and Unlock () guarantee that only one goroutine can get the task and update the task list at the same time. Fetching the task and updating the task queue are the code in critical section, they are atomic. The goroutine can then perform its own tasks. At the same time, other goroutine can compete for mutex, and as long as they get the mutex, they can get the task and update the task list. When a goroutine finishes executing process (task), it will once again participate in the scramble for mutexes because of the for loop.

Just a few major code snippets are given above, and there is a lot of extra code to implement a complete thread pool.

All of the above operations are thread-safe through mutexes. But the problem is that the locking / unlocking mechanism is heavyweight, when the number of worker (that is, goroutine) is enough, the implementation of the locking mechanism will appear a bottleneck.

Realize the work pool through buffered channel

In Go, you can also implement a work pool with buffered channel.

The sample code is long, so here we first split and explain each part, and finally give the complete code snippet.

In the following example, the job of each worker is to calculate the sum of the digits of each value. For example, given a value of 234, the worker calculates 2, 3, 4, 9. The values given here to worker are randomly generated in the range of [0999).

This example has several core functions that need to be explained first, and it also implements the general functions of thread pools through channel:

Create a task buffered channel and store the generated tasks in task buffered channel through the allocate () function

Create a goroutine pool, where each goroutine listens to the task buffered channel and fetches the task

After goroutine executes the task, writes the result to result buffered channel

Take the calculation results from result buffered channel and output them.

First, create the Task and Result structures and create their channels:

Type Task struct {ID int randnum int} type Result struct {task Task result int} var tasks = make (chan Task, 10) var results = make (chan Result, 10)

Here, each Task has its own ID and the random number that the task will be calculated by worker. Each Result contains the calculation result result of the worker and the corresponding task of the result, so that the task information and the calculation result can be extracted from the Result.

In addition, both channels are buffered channel with a capacity of 10. Each worker listens to the tasks channel, takes out the tasks in it for calculation, and then puts the result of the calculation and the task itself into the results channel.

Then there is the function process (), which calculates the sum of digits, which will be one of the tasks of worker.

Func process (num int) int {sum: = 0 for num! = 0 {digit: = num% 10 sum + = digit num / = 10} time.Sleep (2 * time.Second) return sum}

The calculation process is actually very simple, but then sleep for 2 seconds, which is used to pretend that it takes a little time to perform a calculation task.

Then there is worker (), which listens on the tasks channel and fetches the task to calculate, and puts the result into the results channel.

Func worker (wg * WaitGroup) {defer wg.Done () for task: = range tasks {result: = Result {task, process (task.randnum)} results

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.