Shulou(Shulou.com)05/31 Report--

Most people don't understand the knowledge points of this "redis atomic operation example analysis" article, so Xiaobian summarizes the following contents for everyone. The contents are detailed, the steps are clear, and they have certain reference value. I hope everyone can gain something after reading this article. Let's take a look at this "redis atomic operation example analysis" article together.

redis atomic operation

When we use Redis, we inevitably encounter concurrent access problems, such as if multiple users place orders at the same time, the inventory of goods cached in Redis will be updated concurrently. Once there is concurrent write operation, the data will be modified. If we do not control the concurrent write request well, it may cause the data to be corrected and affect the normal use of the business (for example, the inventory data is wrong, resulting in abnormal orders).

In order to guarantee the correctness of concurrent access, Redis provides two methods, locking and atomic operation.

Locking is a common method. Before reading data, the client needs to obtain the lock first, otherwise it cannot operate. When a client acquires a lock, it holds the lock until the client finishes updating the data before releasing the lock.

It looks like a good solution, but there are actually two problems here: one is that if there are many locking operations, it will reduce the concurrent access performance of the system; the second is that when Redis clients want to lock, they need to use distributed locks, and distributed locks are complex to implement, requiring additional storage systems to provide unlocking operations, which I will introduce to you in the next class.

Atomic operations are another way to provide concurrent access control. Atomic operation refers to the operation that the execution process maintains atomicity, and the atomic operation does not need to be locked when it is executed, thus realizing lock-free operation. In this way, concurrency control can be guaranteed, and the impact on system concurrency performance can be reduced.

What do you need to control in concurrent access?

Concurrent access control refers to controlling the process of multiple clients accessing the same data to ensure that any operation sent by a client is mutually exclusive when executed on a Redis instance. For example, when the access operation of client A is executed, the operation of client B cannot be executed, and it needs to wait until the operation of client A is completed before it can be executed.

The operations corresponding to concurrent access control are mainly data modification operations. When the client needs to modify the data, the basic process is divided into two steps:

The client reads the data locally and modifies it locally.

After the client modifies the data, it writes back to Redis.

We call this process Read-Modify-Write (RMW). When multiple clients perform RMW operations on the same piece of data, we need to make the code involved in the RMW operations execute atomically. RMW operation codes that access the same data are called critical section codes.

However, there are some potential problems when multiple clients are executing critical section code concurrently, and I'll explain this with an example of a multi-client update to an inventory of goods.

Let's look at the critical area code first. Suppose the client wants to subtract 1 from the inventory of goods. The pseudocode looks like this:

current = GET(id)current--SET(id, current)

As you can see, the client first reads the current inventory value of the commodity from Redis according to the commodity id (corresponding to Read), and then the client subtracts 1 from the inventory value (corresponding to Modify), and then writes the inventory value back to Redis (corresponding to Write). When multiple clients execute this code, it is a critical section code.

If we have no control over the execution of critical section code, data update errors occur. In the example above, suppose that there are two clients A and B, and the critical section code is executed at the same time. An error will occur. You can see the following diagram.

It can be seen that client A reads inventory value 10 at t1 and deducts 1. At t2, client A has not written back the deducted inventory value 9 to Redis. At this time, client B reads inventory value 10 and deducts 1. The inventory value recorded by B is also 9. At t3, A writes back inventory 9 to Redis, and at t4, B writes back inventory 9.

If the correct logic is followed, clients A and B each make one deduction from the inventory value, and the inventory value should be 8. Therefore, the inventory value here is obviously updated incorrectly.

The reason for this phenomenon is that the client in the critical section code reads data, updates data, and writes back data involves three operations, and these three operations are not mutually exclusive when executed. Multiple clients modify based on the same initial value, rather than modifying based on the modified value of the previous client.

In order to ensure the correctness of concurrent data modification, we can use locks to turn parallel operations into serial operations, and serial operations have mutual exclusion. Once a client holds a lock, other clients can only wait until the lock is released before taking it and modifying it.

The pseudocode below shows the use of locks to control the execution of critical section code. You can take a look.

LOCK()current = GET(id)current--SET(id, current)UNLOCK()

Although locking ensures mutual exclusion, locking also results in reduced concurrency performance.

As shown in the figure below, when client A locks to perform an operation, clients B and C need to wait. After A releases the lock, assuming B gets the lock, then C still needs to wait, so only A can access the shared data during t1 period, and only B can access the shared data during t2 period. The concurrency performance of the system will of course decline.

Similar to locking, atomic operations can also achieve concurrency control, but atomic operations have less impact on system concurrency performance. Next, we will understand atomic operations in Redis.

Two Atomic Operation Methods of Redis

In order to achieve the mutually exclusive execution of critical section code required by concurrency control, Redis atomic operations use two methods:

Multiple operations are implemented into one operation in Redis, that is, a single command operation;

Write multiple operations into a single Lua script and execute a single Lua script atomically.

Let's first look at the single-command operation of Redis itself.

Redis uses a single thread to process client request operations serially, so when Redis executes a command operation, other commands cannot be executed, which is equivalent to command operations being mutually exclusive. Of course, Redis snapshot generation, AOF rewrite these operations, can be used to use background threads or child processes to execute, that is, and the main thread operations in parallel. However, these operations only read data and do not modify it, so we do not need concurrency control for them.

You may also notice that although Redis's single command operation can be executed atomically, in practice, data modification may include multiple operations, including at least three operations: reading data, increasing or decreasing data, and writing back data. This is obviously not a single command operation. What should I do?

Don't worry, Redis provides INCR/DECR commands that turn these three operations into one atomic operation. INCR/DECR commands can increment/decrement data, and they themselves are single command operations, Redis is inherently mutually exclusive when executing them.

For example, in the inventory deduction example above, the client can use the following code to directly complete the inventory value of the item id minus 1 operation. Even if you have multiple clients executing the following code, you don't have to worry about inventory deduction errors.

DECR id

So, if the RMW operation we perform is to increase or decrease the value of the data, the atomic operations INCR and DECR provided by Redis can directly help us with concurrency control.

However, if the operation we want to perform is not simply adding or subtracting data, but has more complex decision logic or other operations, then Redis's single command operation cannot guarantee the exclusive execution of multiple operations. So, at this point, we need to use the second method, which is Lua script.

Redis executes the entire Lua script as a whole without interruption by other commands, thus ensuring the atomicity of the Lua script operations. If we have multiple operations to perform but can't do them with command operations like INCR/DECR, we can write them into a Lua script.

We can then use the EVAL command of Redis to execute the script. As a result, these operations are mutually exclusive when executed.

Here is another example of how Lua is used.

When the number of users accessing a business application increases, we sometimes need to limit the number of visits of a certain client within a certain time range, such as the purchase limit of popular goods, the number of likes per minute in Social networks, etc.

So how do you limit it? We can save the client IP as a key and the number of client visits as a value in Redis. After each client visit, we use INCR to increase the number of visits.

However, in this scenario, client-side throttling actually includes restrictions on the number of accesses and the time range, for example, the number of accesses per minute cannot exceed 20. Therefore, we can set the expiration time for the corresponding key-value pair when the client accesses it for the first time, for example, set it to expire after 60s. At the same time, every time the client visits, we read the current number of visits of the client, and if the number exceeds the threshold, we report an error and restrict the client from accessing again. You can look at the code below, which implements a limit of 20 client accesses per minute.

//Get the number of visits corresponding to ip current = GET(ip)//If the number of visits exceeds 20 times, an error IF current != will be reported NULL AND current > 20 THEN ERROR "exceed 20 accesses per second"ELSE //If the number of visits is less than 20, increment the visit count by one value = INCR(ip) //If this is the first visit, set the expiration time of the key-value pair to 60s later IF value == 1 THEN EXPIRE(ip,60) END //perform other operations DO THINGSEND

As you can see, in this example we have used INCR to atomically increment the count. However, the logic of client-side current limiting is not only counting, but also including the number of visits and expiration time settings.

For these operations, we also need to guarantee their atomicity. Otherwise, if the client uses multithreaded access, the initial value of the number of accesses is 0, and the first thread executes INCR (ip), and the second thread immediately executes INCR(ip). At this time, the number of accesses corresponding to ip is increased to 2, and we can no longer set the expiration time for this ip. This will result in that after the number of client visits corresponding to this IP reaches 20 times, it will no longer be accessible. Even after 60s, you can no longer continue to visit, obviously not in line with business requirements.

Therefore, the actions in this example cannot be implemented with Redis single command, in which case we can use Lua script to ensure concurrency control. We can add 1 to the access count, determine if the access count is 1, and set the expiration time in a Lua script, as follows:

local currentcurrent = redis.call("incr",KEYS[1])if tonumber(current) == 1 then redis.call("expire",KEYS[1],60)end

Assuming we write a script named lua.script, we can then execute it using the Redis client with the eval option. Parameters required by the script are passed through keys and args in the following command.

redis-cli --eval lua.script keys , args

In this way, the three operations of adding 1 to the number of accesses, determining whether the number of accesses is 1, and setting the expiration time can be performed atomically. Even if the client has multiple threads executing the script at the same time, Redis will execute the script code serially in turn, avoiding data errors caused by concurrent operations.

The above is the content of this article about "redis atomic operation example analysis". I believe everyone has a certain understanding. I hope the content shared by Xiaobian will be helpful to everyone. If you want to know more relevant knowledge content, please pay attention to the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.