Shulou(Shulou.com)05/31 Report--

This article mainly explains "how to achieve concurrency control in JavaScript". Interested friends may wish to have a look at it. The method introduced in this paper is simple, fast and practical. Now let the editor take you to learn "how to achieve concurrency control in JavaScript"!

I. Preface

During the development process, you sometimes encounter the need to control the number of tasks executed concurrently.

For example, a crawler can reduce the frequency of requests by limiting the number of concurrent tasks, so as to avoid the problem of blocking requests too frequently.

Next, this article describes how to implement a concurrency controller.

2. Examples

Const task = timeout = > new Promise ((resolve) = > setTimeout (() = > {resolve (timeout);}, timeout)) const taskList = [1000, 3000, 200, 1300, 800, 2000]; async function startNoConcurrentControl () {console.time (NO_CONCURRENT_CONTROL_LOG); await Promise.all (taskList.map (item = > task (item)); console.timeEnd (NO_CONCURRENT_CONTROL_LOG);} startNoConcurrentControl ()

The above sample code uses the Promise.all method to simulate the scenario in which six tasks are executed concurrently, and the total time taken to execute all the tasks is 3000 milliseconds.

The following example will be used to verify the correctness of the implementation method.

III. Realization

Since the number of tasks executed concurrently is limited, a data structure is needed to manage the ongoing tasks.

The * * "first-in, first-out" feature of the queue ensures the order in which tasks are executed concurrently. In JavaScript, you can simulate the queue through "array" * *:

Class Queue {constructor () {this._queue = [];} push (value) {return this._queue.push (value);} shift () {return this._queue.shift ();} isEmpty () {return this._queue.length = 0;}}

For each task, you need to manage its execution functions and parameters:

Class DelayedTask {constructor (resolve, fn, args) {this.resolve = resolve; this.fn = fn; this.args = args;}}

Next, implement the core TaskPool class, which is mainly used to control the execution of tasks:

Class TaskPool {constructor (size) {this.size = size; this.queue = new Queue ();} addTask (fn, args) {return new Promise ((resolve) = > {this.queue.push (new DelayedTask (resolve, fn, args)); if (this.size) {this.size-- Const {resolve: taskResole, fn, args} = this.queue.shift (); taskResole (this.runTask (fn, args));}} pullTask () {if (this.queue.isEmpty ()) {return;} if (this.size = 0) {return } this.size++; const {resolve, fn, args} = this.queue.shift (); resolve (this.runTask (fn, args));} runTask (fn, args) {const result = Promise.resolve (fn (.ARGs)); result.then () = > {this.size--; this.pullTask () }) .catch () = > {this.size--; this.pullTask ();}) return result;}}

TaskPool includes three key methods:

AddTask: put the new task in the queue and trigger the task pool status detection. If the current task pool is not fully loaded, the task is taken out of the queue and placed in the task pool for execution.

RunTask: the current task is executed. After the task execution is completed, the task pool status is updated, and the mechanism for actively pulling new tasks is triggered.

PullTask: if the current queue is not empty and the task pool is not fully loaded, the tasks in the queue are actively removed for execution.

Next, control the number of concurrency in the previous example to 2:

Const cc = new ConcurrentControl (2); async function startConcurrentControl () {console.time (CONCURRENT_CONTROL_LOG); await Promise.all (taskList.map (item = > cc.addTask (task, [item])) console.timeEnd (CONCURRENT_CONTROL_LOG);} startConcurrentControl ()

The execution process is as follows:

The total time taken to execute the task is 5000 milliseconds.

IV. Optimal parameter transfer of higher-order functions

Await Promise.all (taskList.map (item = > cc.addTask (task, [item])

The method of manually passing the parameters of each task is very tedious. Here, you can use * * "high-order functions to achieve automatic transparent transmission of parameters" * *:

AddTask (fn) {return (... args) = > {return new Promise ((resolve) = > {this.queue.push (new DelayedTask (resolve, fn, args)); if (this.size) {this.size--; const {resolve: taskResole, fn: taskFn, args: taskArgs} = this.queue.shift (); taskResole (this.runTask (taskFn, taskArgs)) })}}

The modified code is much simpler:

Await Promise.all (taskList.map (cc.addTask (task)

Fifth, optimize the operation of getting out of the team

Arrays are generally stored based on a block of * * "continuous memory" * *. When calling the shift method of the array, the header element is deleted first (time complexity O (1)), and then the undeleted element needs to be moved one bit to the left (time complexity O (n)), so the time complexity of shift operation is O (n).

Because of the characteristics of JavaScript language, V8 gives a solution of space and time tradeoff when implementing JSArray. In different scenarios, JSArray will switch between FixedArray and HashTable modes.

In hashTable mode, shift operation saves the time complexity of moving to the left, and its time complexity can be reduced to O (1). Even so, shift is still a time-consuming operation.

In scenarios with a large number of array elements and frequent shift operations, it can be optimized in the way of "reverse + pop".

Const Benchmark = require ('benchmark'); const suite = new Benchmark.Suite; suite.add (' shift', function () {let count = 10; const arr = generateArray (count); while (count--) {arr.shift ();}}) .add ('reverse + pop', function () {let count = 10; const arr = generateArray (count)) Arr.reverse (); while (count--) {arr.pop ();}}) .on ('cycle', function (event) {console.log (String (event.target));}) .on (' complete', function () {console.log ('Fastest is' + this.filter ('fastest'). Map (' name')) Console.log ('\ n')}) .run ({async: true})

From the benchmark data run by benchmark.js, it is easy to see which approach is more efficient:

Looking back at the previous implementation of the Queue class, since there is only one array to store tasks, the direct use of reverse + pop will inevitably affect the order in which tasks are executed.

Here we need to introduce the design of double arrays, one array is responsible for queue operation, and one array is responsible for dequeue operation.

Class HighPerformanceQueue {constructor () {this.q1 = []; / for push data this.q2 = []; / / for shift data} push (value) {return this.q1.push (value);} shift () {let Q2 = this.q2; if (q2.length = 0) {const Q1 = this.q1 If (q1.length = 0) {return;} this.q1 = Q2; / / Thank you @ shaonialife for correcting Q2 = this.q2 = q1.reverse ();} return q2.pop () } isEmpty () {if (this.q1.length = 0 & & this.q2.length = 0) {return true;} return false;}}

Finally, the effect of optimization is verified by benchmark test:

At this point, I believe you have a deeper understanding of "JavaScript how to achieve concurrency control", might as well come to the actual operation of it! Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.