Shulou(Shulou.com)06/01 Report--

This article mainly introduces the redis performance and monitoring scheme in detail, the sample code in this article is very detailed, which has a certain reference value, interested friends can refer to it.

1. Redis slowlog analysis

SLOWLOG subcommand [argument]

The following are the slow query configuration parameters for redis.conf:

Slowlog-log-slower-than 10000 # if the query time exceeds 10ms, a maximum of 128slow queries will be recorded in slowlog-max-len.

The above parameters can be dynamically queried or set in redis: use the config get and config set commands

Read slow query: you can get specified slow queries

127.0.0.1 slowlog get 6320 > redis 11) 1) (integer) 394689 # slowlog unique number 2) (integer) 1480851711 # time of this slowlog event 3) (integer) 10639 # time consuming 4) 1) redis command corresponding to "HGET" # slowlog event 2) "hash:submit_sent_150004" 3) "150004296481227343645165312"

Reset slowlog statistics:

SLOWLOG RESET

Redis slow commands Analysis:

1. Redis uses a single thread to process the client's request, and as a result, when the request is slow, all other clients will wait for the request to be served. If you need to perform a lot of slow commands, it is recommended that you put slow queries on redis slave for execution.

2. About keys commands: a very common source of delay generated by executing slow commands is the use of KEYS commands in a production environment. As described in the Redis documentation, KEYS should be used for debugging purposes only.

Redis2.8 later introduced new commands to query large collections. Check the SCAN,SSCAN,HSCAN and ZSCAN commands for more information.

SCAN iterates over the keyset in the currently selected Redis database.

SSCAN iterates over elements of the sets collection type.

HSCAN iterates through the fields of the hash type and its associated values.

ZSCAN iterates the elements of the sort set type and their associated scores.

Because these commands allow incremental iterations and return only a small number of elements per call, they can be used in production without the disadvantages of commands such as KEYS or SMEMBERS, which may block the server for a long time (or even a few seconds) when calling keys or elements of a large collection.

2. How to use SCAN,SSCAN,HSCAN and ZSCAN commands

SCAN is a cursor-based iterator. This means that the server returns an updated cursor each time the command is invoked, and the user needs to use it as a cursor parameter in the next call.

When the cursor is set to 0, the iteration starts and terminates when the cursor returned by the server is 0. The following is an example of a SCAN iteration:

127.0.0.1 list:submit_sent_2016 6319 > scan 01) "65536" 2) 1) "list:submit_sent_2016-12-02-13 list:submit_sent_2016-12-02-13 50" 130806 "2)" list:submit_sent_2016-12-01-15 14) 01mm 130479 "3)" list:submit_sent_2016-12-01-13 list:submit_sent_2016-01-13 list:submit_sent_2016-01-13 list:submit_sent_2016-12-01-13 list:submit_sent_2016-12-01-13 list:submit_sent_2016-01-13 list:submit_sent_2016-12-01-13 list:submit_sent_2016-12-01-13 list:submit_sent_2016-12-02-10 rig 46th 186064 "5) : submit_sent_2016-12-01-16 list:submit_sent_2016-12-01-13 list:submit_sent_2016-12-02-08) 29mm 189316 "10)" 189316 "10): Submit_sent_2016-12-01-13 submit_sent_2016-12-01-13 list:submit_sent_2016-12-01-13 list:submit_sent_2016-12-01-18 14) 1715 0243 "4)" list:submit_sent_2016-12) -01-11 list:submit_sent_2016 22 "137606" 5) "list:submit_sent_2016-12-01-21 15" 183748 "6)" list:submit_sent_2016-12-02-11 14) 47 "155212" 7) "list:submit_sent_2016-12-01-11 list:submit_sent_2016-12-01-11 14) 01mm 137065" 8) "list:submit_sent_2016-12-02-0818 1202" 9) "list:submit_sent_2016-12" -02-12 list:submit_sent_2016 16 "136096" 10) "12-01-18" 159893 "

Start an iteration with a cursor value of 0 and call SCAN until the returned cursor is 0 again, which is called a full iteration.

Count option for scan: specify the number of outputs

127.0.0.1 list:submit_sent_2016 6319 > scan 0 count 201) "884736" 2) 1) "list:submit_sent_2016-12-02-13 list:submit_sent_2016 50" 130806 "2)" list:submit_sent_2016-12-01-15 list:submit_sent_2016 01mm 130479 "3)" list:submit_sent_2016-12-01-1321 "178888" 4) "list:submit_sent_2016-12-02-10 46" 186064 "5) "list:submit_sent_2016-12-01-16 purl 48mm 135546" 6) "list:submit_sent_2016-12-02-14purr 27mm 185158" 7) "list:submit_sent_2016-12-02-09VOV 57mm 186532" 8) "list:submit_sent_2016-12-01-13purr 24" 183217" 9) "list:submit_sent_2016-12-02-08vir 29" 189316" 10) List:submit_sent_2016-12-01-13 list:submit_sent_2016-12-01-13 list:submit_sent_2016-12-01-17 list:submit_sent_2016-12-01-18 list:submit_sent_2016-12-01-18 14) list:submit_sent_2016-12-01-18 14) list:submit_sent_2016-12-01-11 22 "137606" 15) "list:submit_sent _ 2016-12-01-21 list:submit_sent_2016 15156096 "20)" list:submit_sent_2016-12-01-18186096 "20)" list:submit_sent_2016-12-01-18186096 "20)" list:submit_sent_2016-12-01-18186096 "20)"

Match option for scan: similar to the keys command matching by pattern, only the MATCH parameter is appended to the end of the SCAN command

127.0.0.1 scan 0 match * submit_sent*1) "65536" 2) 1) "list:submit_sent_2016-12-02-13 list:submit_sent_2016 50" 130806 "2)" list:submit_sent_2016-12-01-15 submit_sent*1 01mm 130479 "3)" list:submit_sent_2016-12-01-1321 "178888" 4) "list:submit_sent_2016-12-02-10 46" 186064 " 5) "list:submit_sent_2016-12-01-16 list:submit_sent_2016-01-16 list:submit_sent_2016-12-01-16 list:submit_sent_2016-12-01-13 list:submit_sent_2016-12-01-13 list:submit_sent_2016-12-02-0829" 189316 " 10) "list:submit_sent_2016-12-01-13 match 46mm 177645" 127.0.0.1 match 6319 > scan 0 count 15 match * submit_sent* # find matching data and return 15) 1) "2031616" 2) 1) "list:submit_sent_2016-12-02-13 submit_sent* 50" 130806 "2)" list:submit_sent_2016-12-01-15 submit_sent* 01mm 130479 "3)" list : submit_sent_2016-12-01-13 list:submit_sent_2016-12-01-13) 27mm 188888 "4)" list:submit_sent_2016-12-02-10 purge 46th 186064 "5)" 135546 "6)" list:submit_sent_2016-12-02-14 purl 27" 185158 "7)" list:submit_sent_2016-12-02-0957 "186532" 8). : submit_sent_2016-12-01-13 list:submit_sent_2016-12-01-17 list:submit_sent_2016-12-01-17) list:submit_sent_2016-12-02-18 28 "138052" 13) "list:submit_sent _ 2016-12-01-18 purl 17mm 150243 "14)" list:submit_sent_2016-12-01-11 Vera 22 "137606" 15) "list:submit_sent_2016-12-01-21 purge 15" 183748 "

The method for sscan to query the sets collection:

Redis 127.0.0.1 foo foobar feelsgood (integer) 6redis 127.0.0.1 6redis 127.0.0.1 6redis 6379 > sscan myset 0 match Fleet 1) "0" 2) 1) "foo" 2) "feelsgood" 3) "foobar" redis 127.0.0.1

The method for hscan to query the hash collection:

Redis 127.0.0.1 hmset hash name Jack age 33OKredis 6379 > hscan hash 01) "0" 2) 1) "name" 2) "Jack" 3) "age" 4) "33"

When a Linux kernel enables transparent giant pages, Redis has a large latency cost after using fork calls to persist data on disk.

You can use this method to turn off this feature of the system kernel:

Echo never > / sys/kernel/mm/transparent_hugepage/enabled

Note: you need to restart redis to take effect.

3. Check whether redis is affected by the use of swap by the system:

Find the redis process id:redis-cli-p 6319 info | grep process_idprocess_id:32139 to view the memory usage information of the redis process: cd / proc/32139 to view the statistics of the process using swap partitions, preferably with no use or only a small amount of 4kB: cat smaps | grep 'Swap:' prints out both memory mapping and swap usage information: see if those large memory consumption lead to large swap usage cat smaps | egrep' ^ (Swap:Size)'

4. Use redis watchdog positioning delay: experimental function, please make sure that the redis data has been backed up.

Redis software watchdog this feature can only be enabled dynamically, using the following command: CONFIG SET watchdog-period 500Note: redis will start to frequently monitor its own delay problems and output them to log files. Turn off watchdog:CONFIG SET watchdog-period 0 Note: turning on the watchdog function will have an impact on the performance of redis services.

5. On the delay monitoring framework of redis

Redis latency monitoring framework

How to enable the redis monitoring framework:

CONFIG SET latency-monitor-threshold 100

By default, the threshold is set to 0, which disables redis monitoring. In fact, enabling this monitoring feature adds little cost to redis. However, for a redis that is running well, it is not necessary to turn on the monitoring function.

How to use the LATENCY command

View the latest delay events:

127.0.0.1 event name 6319 > latency latest1) 1) "command" # event name 2) (integer) 1480865648 # occurrence time 3) (integer) 207 # time, millisecond 4) (integer) 239 # maximum delay recording of such an event since redis was started or the last latency reset

View historical information about deferred events:

LATENCY HISTORY event-name

For a given event, the command returns up to 160 elements. The application may want to get the raw data to perform monitoring, display graphics, and so on.

127.0.1 latency history command 1) 1) (integer) 1480865710 2) (integer) 2072) 1) (integer) 1480865711 2) (integer) 217 3) 1) (integer) 1480865712 2) (integer) 1984) 1) (integer) 1480865713 2) (integer) 265) 1) (integer) 1480865714 2) (integer) 224

Statistics return to zero:

LATENCY RESET [event-name... Event-name]

Display delayed events as a text chart:

LATENCY GRAPH event-name127.0.0.1:6379 > latency graph commandcommand-high 500ms, low 101ms (all time high 500ms)-# _ _ | | _ | 11186542sssss |

Note: each column represents a late event; below shows how long the event occurred before the current time, such as 2m or 38s; the minimum delay event in the statistical event is recorded as a short underscore and the maximum delay event is represented as a # high above. The diagram can reflect a changing trend of delay events.

LATENCY DOCTOR, intelligent analysis and suggestion of delay event statistics:

127.0.1 do you Dave?1 6379 > latency doctorDave, I have observed latency spikes in this Redis instance.You don't mind talking about it, do you Dave?1. Command: 5 latency spikes (average 300ms, mean deviation 120ms, period 73.40 sec). Worst all time event 500ms.I have a few advices for you:- Your current Slow Log configuration only logs events that are slower than your configured latency monitor threshold. Please use 'CONFIG SET slowlog-log-slower-than 1000mm.-Check your Slow Log to understand what are the commands you are running which are too slow to execute. Please check http://redis.io/commands/slowlog for more information.- Deleting, expiring or evicting (becaus127.0.0.1:6320 > latency doctorI have a few advices for you:- I detected anon zero amount of anonymous huge pages used by your process. This creates very serious latency events in different conditions, especially when Redis is persisting on disk. To disable THP support use the command 'echo never > / sys/kernel/mm/transparent_hugepage/enabled', make sure to also add it into / etc/rc.local so that the command will be executed again after a reboot. Note that even if you have already disabled THP, you still need to restart the Redis process to get rid of the huge pages already created.

On the redis performance and monitoring program to share here, of course, not only the above and everyone analysis methods, but the editor can ensure that its accuracy is absolutely no problem. I hope that the above content can have a certain reference value for everyone, and can be put into practice. If you like this article, you might as well share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.