Shulou(Shulou.com)05/31 Report--

This article is to share with you about the usefulness of Lua scripts in Redis. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

Redis lua script related commands

The content of this section is the basic command, which can be skipped after a cursory reading, and then come back to query it when you use it.

Redis has added commands related to lua script, EVAL, EVALSHA, SCRIPT EXISTS, SCRIPT FLUSH, SCRIPT KILL, SCRIPT LOAD since 2.6.0, and debug function and command SCRIPT DEBUG of lua script since 3.2.0. Here is a brief introduction to the command.

EVAL executes a lua script, and each time you need to pass the complete lua script to the redis server.

SCRIPT LOAD caches a lua script into redis and returns a tag string, which is not executed.

EVALSHA executes a script, but the input parameter is the tag returned in "2", saving network bandwidth

. SCRIPT EXISTS determines whether the tag string returned by "2" exists in the server.

SCRIPT FLUSH clears all cached scripts on the server.

SCRIPT KILL kills the running script.

SCRIPT DEBUG sets debug mode, which can be set for synchronization, async, and shutdown. Synchronization blocks all requests.

In a production environment, it is recommended to use EVALSHA, which is more efficient than each sending script body of EVAL and wasting bandwidth. Note here that SCRIPT KILL kills when the script is running, and if the script has performed a write operation, it will fail because it violates the atomicity of the redis lua script. Debugging should be done as soon as possible after the test environment is completed before releasing to the production environment. Do not use synchronous mode for debugging in the production environment. The reasons will be discussed in more detail below.

Writing and debugging of lua script in Redis

Redis lua script is an extension of its existing commands, a single command can not be completed, need multiple commands, but to ensure that atomic actions can be realized by scripts. The logic in the script is generally simple, do not add too complex things, because redis is single-threaded, when the script execution, other commands, scripts need to wait until the current script execution is completed. Therefore, it is not necessary to fully understand the syntax of lua, but it is enough to understand the basic use. Here, the syntax of lua is not introduced too much and will be interspersed with the script examples.

An example of a second kill rush to buy

Suppose there is a flash sale activity with an inventory of 100. each user can only snap up uid once. The design rush purchase process is as follows:

First through the uid to determine whether has been robbed, has been robbed to return 0 to the end.

Judge whether the surplus inventory of goods is greater than 0, enter "3" if yes, and return 0 if not.

Add the user uid to the purchased user set.

The number of items is reduced by 1 and return to success 1 ends.

Local goodsSurpluslocal flag-- determines whether the user has robbed local buyMembersKey = tostring (KEYS [1]) local memberUid = tonumber (ARGV [1]) local goodsSurplusKey = tostring (KEYS [2]) local hasBuy = redis.call ("sIsMember", buyMembersKey, memberUid)-- has snapped up Return 0if hasBuy ~ = 0 then return 0endmurf-ready to snap up goodsSurplus = redis.call ("GET", goodsSurplusKey) if goodsSurplus = = false then return 0endmure-there are no remaining snap-up items goodsSurplus = tonumber (goodsSurplus) if goodsSurplus 4 local buyMembersKey = tostring (KEYS [1]) lua debugger > n * Stopped at 5, stop reason = step over- > 5 local memberUid = tonumber (ARGV [1]) lua debugger > n * Stopped at 6, stop reason = step over- > 6 local goodsSurplusKey = tostring (KEYS [2]) lua debugger > s* Stopped at 7 Stop reason = step over- > 7 local hasBuy = redis.call ("sIsMember", buyMembersKey, memberUid)

Continue executes code from the current line to the end or to a breakpoint.

Show relevance

[l] list 、 [l] list [line] 、 [l] list [line] [ctx] 、 [w] hole

To show the code near the current line, [line] is to reassign the central line, and [ctx] is to specify a few lines of code around the center of the display. [W] hole is to show all lines of code

Print related

[p] rint 、 [p] rint

Print all current local variables, which are the specified variables, as follows:

Lua debugger > print goodsSurplus = nil flag = nil buyMembersKey = "hadBuyUids" memberUid = 58247lua debugger > print buyMembersKey "hadBuyUids"

Breakpoint correlation

[B] reak, [b] reak, [b] reak -, [b] reak 0

Show breakpoints, add breakpoints like specified lines, delete breakpoints on specified lines, delete all breakpoints

Other commands

[r] edis 、 [m] axlen [len] 、 [a] bort 、 [e] eval 、 [t] race

Execute the redis command in debugging

Set the maximum length of display content. 0 means there is no limit.

Exit debug mode, and changes will be retained in synchronous mode (parameter-ldb-sync-mode is set).

Execute a line of lua code.

Show the execution stack.

Describe the [m] axlen [len] command in detail, as follows:

Local myTable = {} local count = 0while count

< 1000 do myTable[count] = count count = count + 1endreturn 1 在最后一行打印断点，执行print可以看到，输出了一长串内容，我们执行maxlen 10之后，再次执行print可以看到打印的内容变少了，设置为maxlen 0之后，再次执行可以看到所有的内容全部展示了。 详细说下[t]race命令，代码如下： local function func1(num) num = num + 1 return numendlocal function func2(num) num = func1(num) num = num + 1 return numendfunc2(123) 执行b 2在 func1 中打断点，然后执行c，断点地方停顿，再次执行t，可以到如下信息： lua debugger>

TIn func1:- > # 3 return numFrom func2: 7 num = func1 (num) From top level: 12 func2

Request current limit

At this point, we have a basic understanding of the redis lua script, basic syntax, debugging, and then implement a request current limiter. The process and code are as follows:

-- local busIdentify = tostring (KEYS [1]) local ip = tostring (KEYS [2]) local expireSeconds = tonumber (ARGV [1]) local limitTimes = tonumber (ARGV [2]) local identify = busIdentify.. "_".. Iplocal times = redis.call ("GET", identify)-- [[get the recorded time to continue to determine whether the limit is exceeded or return 0 plus 1]-- if times ~ = false then times = tonumber (times) if times > = limitTimes then return 0 else redis.call ("INCR", identify) return 1 endend-- does not exist Set to 1 and set expiration time local flag = redis.call ("SETEX", identify, expireSeconds, 1) return 1

Save the above lua script to / path/to/limit.lua and execute redis-cli-- eval / path/to/limit.lua limit_vgroup 192.168.1.19,103, which indicates the limit_vgroup service. 192.168.1.1 this ip restricts access to three times every 10 seconds.

All right, at this point, a request current limit function is completed. After three consecutive executions, the above program will return 0, execute in 10 seconds, and return 1 again, thus achieving the purpose of current limit.

Some students may say that this request current limit function is still worth optimizing. If there are two consecutive counting cycles, the last three requests in the first cycle, and then the second cycle is coming soon, we can ask again. How to optimize this place? let's move on.

Request current limit optimization

The above counter method is simple and rough, but there is a problem with the critical point. In order to solve this problem, a concept similar to sliding window is introduced to make the period of statistical times continuous, which can well solve the problem of critical point. The principle of sliding window is shown in the following figure:

Establish a redis list structure whose length is equivalent to the number of visits. For each request, determine whether the length of the list structure exceeds the limit. If not, add it directly to the head of the queue to return success. Otherwise, determine whether the data at the end of the queue has exceeded the limit time, did not exceed the direct return failure, exceeded the deletion of the element at the end of the queue, and inserted the request time into the head of the queue and returned successfully.

Local busIdentify = tostring (KEYS [1]) local ip = tostring (KEYS [2]) local expireSeconds = tonumber (ARGV [1]) local limitTimes = tonumber (ARGV [2])-pass in additional parameters, request timestamp local timestamp = tonumber (ARGV [3]) local lastTimestamplocal identify = busIdentify.. "_".. Iplocal times = redis.call ("LLEN", identify) if times

< limitTimes then redis.call("RPUSH", identify, timestamp) return 1endlastTimestamp = redis.call("LRANGE", identify, 0, 0)lastTimestamp = tonumber(lastTimestamp[1])if lastTimestamp + expireSeconds >

= timestamp then return 0endredis.call ("LPOP", identify) redis.call ("RPUSH", identify, timestamp) return 1

Save the above lua script to / path/to/limit_fun.lua and execute redis-cli-- eval / path/to/limit_fun.lua limit_vgroup 192.168.1.19, 10 3 1548660999.

At first, I wanted to put the timestamp calculation redis.call ("TIME") into the redis lua script. Later, I found that redis will report an error when it is used. This is because redis copies the lua script to the slave and persistence by default. If the script is a non-pure function (pure function), there may be inconsistencies when executing in the slave library or during downtime recovery. Here we can compare the replication mode based on SQL statement in mysql. Redis in version 3.2 added the redis.replicate_commands function to solve this problem, in the first line of the script to execute this function, redis will modify the data command collection, and then wrapped in MULTI/EXEC, this method is called script effects replication, similar to the row-based replication mode in mysql, the value of the impure function is calculated for persistence and master-slave replication. Here we refer the change parameter to the caller, and the caller passes in the timestamp to solve this problem.

In addition, starting from version 5, redis supports script effects replication by default, so there is no need to call the open function on the first line. If it is time-consuming calculation, of course, this is good. Synchronization and recovery only need to be calculated once and then there is no need to calculate, but if the data generated by a cycle, more bandwidth may be wasted during synchronization. It is more direct without scripts, but this should be less common.

At this point, the script optimization is complete, but I think of another question, our environment is a stand-alone environment, if it is a distributed environment, how to execute the script, how to deal with it, in the next section, we will discuss this problem.

Lua processing in Cluster Environment

In the redis cluster, the keys are assigned to different slots and then to the corresponding machines. When the keys for the operation are one, there is no problem, but if there are multiple keys for the operation, how does the cluster know which machine the operation falls to? Such as simple mget commands, mget test1 test2 test3, and we pass in multiple parameters when we execute the script above, so let's move on with this problem.

First start a redis cluster with docker, docker pull grokzen/redis-cluster, pull the image, and then execute docker run-p 7000VL 7000-p 7001docker pull grokzen/redis-cluster 7002-p 7003RV 7003-p 7004RV 7004-p 7005Rd 7005-name redis-cluster-script-e "IP=0.0.0.0" grokzen/redis-cluster starts the container, which starts a redis cluster with 3 masters and 3 slaves.

We enter the cluster from any node, such as redis-cli-c-p 7003. After entering, we execute cluster nodes to see the cluster information. We link to the slave library and execute set lua fun. Some students may ask whether the slave library can also be written. No problem, the cluster will calculate which slot the lua key belongs to, and then direct it to the corresponding master library.

When you execute mset lua fascinating redis powerful, you can see that the cluster returned an error message, telling us that the requested key did not fall on the same slot.

(error) CROSSSLOT Keys in request don't hash to the same slot

Again, in the lua script above, we add the cluster port number and execute redis-cli-p 7000-eval / tmp/limit_fun.lua limit_vgroup 192.168.1.19,10 3 1548660999, which returns the same error as above.

In order to solve this problem, redis officials have provided us with the method of hash tag. What do you mean? we take a segment of the key to calculate the hash, and the calculation falls into that slot, so that the same key with different functions can fall into the same slot. Hash tag is a string enclosed by {} this pair of parentheses, for example, above, we change it to mset lua {yes} fascinating redis {yes} powerful, and it can be executed successfully. The operation of mset here falls to the machine of port 7002.

Similarly, we can do hash tag processing on the key name of the incoming script. Here, we should note that not only the key name should have the same hash tag, but also the actual key in it must have the same hash tag, otherwise an error Lua script attempted to access a non local key in a cluster node will be reported. What do you mean? take our example above, as shown below during execution, you can see that the first two keys are added with hash tag-yes. This is fine, because the script only uses a spliced key-- limit_vgroup {yes} _ 192.168.1.19 {yes}.

Redis-cli-c-p 7000-eval / tmp/limit_fun.lua limit_vgroup {yes} 192.168.1.19 {yes}, 10 3 1548660999

If we add redis.call ("GET", "yesyes") to the script (don't let this key fall on the same solt as our spliced key), you can see and report the above error, so when executing the script, as long as you pass in the parameter key and the key that executes the redis command in the script has the same hash tag.

In addition, here is a hash tag rule:

The key contains {characters; Jianzhong contains {characters and is to the right of {characters; and there is at least one character between {,}, and the character between them is used as the hash tag of the key.

Therefore, the hash tag of the key limit_vgroup {yes} _ 192.168.1.19 {yes} is yes. The hash tag of the foo {} {bar} key is itself. The hash tag of the foo {{bar}} key is {bar.

Using golang connections using redis

Here we use the golang example to show that the lua script is cached to each node in the cluster through ForEachMaster, and the returned sha value is saved, and the code is executed later through evalsha.

Package mainimport ("github.com/go-redis/redis"fmt") func createScript () * redis.Script {script: = redis.NewScript (`local busIdentify = tostring (KEYS [1]) local ip = tostring (KEYS [2]) local expireSeconds = tonumber (ARGV [1]) local limitTimes = tonumber (ARGV [2])-pass in additional parameters, request timestamp local timestamp = tonumber (ARGV [3]) local lastTimestamp local identify = busIdentify.. "_".. Ip local times = redis.call ("LLEN", identify) if times

< limitTimes then redis.call("RPUSH", identify, timestamp) return 1 end lastTimestamp = redis.call("LRANGE", identify, 0, 0) lastTimestamp = tonumber(lastTimestamp[1]) if lastTimestamp + expireSeconds >

= timestamp then return 0 end redis.call ("LPOP", identify) redis.call ("RPUSH", identify, timestamp) return 1 `) return script} func scriptCacheToCluster (c * redis.ClusterClient) string {script: = createScript () var ret string c.ForEachMaster (func (m * redis.Client) error {if result, err: = script.Load (m). Result () Err! = nil {panic ("caching script to master node failed")} else {ret = result} return nil} return ret} func main () {redisdb: = redis.NewClusterClient (& redis.ClusterOptions {Addrs: [] string {": 7000", ": 7001", ": 7002", ": 7003", ": 7004", ": 7005",} }) / / Cache the script to all nodes Execute once to get the result sha: = scriptCacheToCluster (redisdb) / / execute the cache script ret: = redisdb.EvalSha (sha, [] string {"limit_vgroup {yes}", "192.168.1.19 {yes}",}, 10,3 err 1548660999) if result, err: = ret.Result () Err! = nil {fmt.Println ("exception occurs, return value:", err.Error ())} else {fmt.Println ("return value:", result)} / / example error case, ret1: = redisdb.EvalSha (sha + "error", [] string {"limit_vgroup {yes}", "192.168.1.19 {yes}",}, 10,3M 1548660999) if result, err: = ret1.Result () Err! = nil {fmt.Println ("exception occurs, return value:", err.Error ())} else {fmt.Println ("return value:", result)}}

Execute the above code and the return value is as follows:

Return value: 0

If an exception occurs, the return value: NOSCRIPT No matching script. Please use EVAL.

Thank you for reading! This is the end of this article on "what is the use of Lua scripts in Redis?". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, you can share it for more people to see!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.