Shulou(Shulou.com)06/01 Report--

A detailed explanation of the queue principle of Redis implementation

Scenario description:

It is used to process time-consuming requests, such as sending messages in batches. If sending is triggered directly on the web page, the program will time out.

In high concurrency scenarios, when requests increase instantly at some point, requests can be written to the queue, and the background is processing these requests.

Panic buying scenario, first-in, first-out mode

Command:

Rpush + blpop or lpush + brpop

Rpush: push data to the right of the list

Blpop: client blocks until queue has value output

Simple queue:

Simple.php$stmt = $pdo- > prepare ('select id, cid, name from zc_goods limit 200000'); $stmt- > execute (); while ($row = $stmt- > fetch (PDO::FETCH_ASSOC)) {$redis- > rPush (' goods:task', json_encode ($row));} $redis- > close ()

Get 200 million items and push the json data into the goods:task queue

QueueBlpop.php// outgoing while (true) {/ / blocking setting timeout is 3 seconds $task = $redis- > blPop (array ('goods:task'), 3); if ($task) {$redis- > rPush (' goods:success:task', $task [1]); $task = json_decode ($task [1], true); echo $task ['id']. ':. $task ['cid']. ':. 'handle success'; echo PHP_EOL;} else {echo' nothing'. PHP_EOL; sleep (5);}}

Set the blpop blocking time to 3 seconds. When there is data out of the queue, saving it to goods:success:task indicates that the execution is successful. When there is no data in the queue, the program sleeps for 10 seconds to re-check whether the goods:task has data out of the queue.

Execute commands in cli mode:

Php simple.phpphp queueBlpop.php

Priority queue

Train of thought:

When blpop has multiple keys, blpop traverses the keys from left to right, and the client returns as soon as a key pops up the element. For example:

Blpop key1 key2 key3 key4

Traversing from key1 to key4, if which key has a value, this value pops up, and if multiple key have values at the same time, the key on the left will be popped first.

Priority.php// sets priority queue $high = 'goods:high:task';$mid =' goods:mid:task';$low = 'goods:low:task';$stmt = $pdo- > prepare (' select id, cid, name from zc_goods limit 200000'); $stmt- > execute (); while ($row = $stmt- > fetch (PDO::FETCH_ASSOC)) {/ / cid less than 100 is placed in low-level queue if ($row ['cid']

RPush ($low, json_encode ($row)); / / cid is placed in intermediate queue} elseif ($row ['cid'] > 100 & & $row [' cid'])

RPush ($mid, json_encode ($row));} / / cid is placed in the advanced queue else {$redis- > rPush ($high, json_encode ($row));}} $redis- > close (); priorityBlop.php// priority queue $high = 'goods:high:task';$mid =' goods:mid:task';$low = 'goods:low:task' / / dequeue while (true) {/ / High priority queue on the left $task = $redis- > blPop (array ($high, $mid, $low), 3); if ($task) {$task = json_decode ($task [1], true); echo $task ['id']. ':. $task ['cid']. ':. 'handle success'; echo PHP_EOL;} else {echo' nothing'. PHP_EOL; sleep (5);}}

The queues with high priority are placed on the left side of the blpop command, sorted in turn, and the blpop command pops up the values of high, mid, and low queues in turn.

Execute commands in cli mode:

Php priority.phpphp priorityBlpop.php

Delay queue

Train of thought:

You can use an ordered collection to save deferred tasks, member to save task contents, and score to save (current time + delay time). Use time as score. The program only needs to compare the score of the first task of the ordered set with the current time. If the current time is smaller than score, it means that all the tasks of the ordered set have not yet reached the execution time.

Delay.php$stmt = $pdo- > prepare ('select id, cid, name from zc_goods limit 200000'); $stmt- > execute (); while ($row = $stmt- > fetch (PDO::FETCH_ASSOC)) {$redis- > zAdd (' goods:delay:task', time () + rand (1,300), json_encode ($row));}

Import 200000 tasks into the ordered set goods:delay:task, and all tasks are delayed to execute within 1 to 300 seconds after that.

DelayHandle.php

While (true) {/ / because it is an ordered set, as long as the delay time of the first record is judged, for example, the first item has not reached the execution time / / other tasks of the relative description set have not reached the execution time

$rs = $redis- > zRange ('goods:delay:task', 0,0, true)

/ / the collection has no tasks, and the sleep time is set to 5 seconds.

If (empty ($rs)) {echo'no tasks, sleep 5 seconds'. PHP_EOL;sleep (5); continue;} $taskJson = key ($rs); $delay = $rs [$taskJson]; $task = json_decode ($taskJson, true); $now = time (); / / until time to execute deferred task if ($delay zRem ('goods:delay:task', $taskJson); $redis- > rPush (' goods:task', $taskJson); echo $task ['id']. ' Run'. PHP_EOL

/ / release the lock

ReleaseLock ($task ['id'], $identifier);} else {

/ / delay task is not up to execution time

$sleep = $delay-$now

/ / the maximum value is set to 2 seconds to ensure that if a new task (delay time of 1 second) enters the collection, it can be processed in time.

$sleep = $sleep > 2.2: $sleep; echo 'wait'. $sleep. ' Seconds'. PHP_EOL; sleep ($sleep);}}

This file handles delayed tasks in an ordered set, and if the delayed task reaches execution time, it moves the delayed task to the task queue.

QueueBlpop.php// outgoing while (true) {/ / blocking setting timeout is 3 seconds $task = $redis- > blPop (array ('goods:task'), 3); if ($task) {$redis- > rPush (' goods:success:task', $task [1]); $task = json_decode ($task [1], true); echo $task ['id']. ':. $task ['cid']. ':. 'handle success'; echo PHP_EOL;} else {echo' nothing'. PHP_EOL;sleep (5);}}

Working with tasks in the task queue

Execute the command in cli mode:

Php delay.phpphp delayHanlde.phpphp queueBlpop.php

If you have any questions, please leave a message or go to the community to exchange and discuss, thank you for reading, hope to help you, thank you for your support!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.