Shulou(Shulou.com)06/01 Report--

Why is there blocking in Redis? Perhaps many people do not understand, in order to let you understand better, the editor summed up the following content, I hope you can gain something according to this article.

Find blockage

The online application service first senses that it can add exception statistics to the application side and report to the police through email, SMS and Wechat.

With the help of log system, statistical exception and trigger alarm logic

With the help of the Redis monitoring system to find the blocking problem, trigger the alarm. CacheCloud system is recommended.

Internal cause

Unreasonable use of API or data structures

For high concurrency scenarios, avoid executing commands that are more complex than O (n) O (n) on large objects.

Found slow query: slowlog get {n}

Large object found: redis-cli-h {ip}-p {port} bigkeys

CPU saturation

CPU saturation means that redis runs single-core CPU to 100%.

Top command to view redis process CPU utilization

Redis-cli-h {ip}-p {port}-stat gets the current redis usage and determines whether concurrency has reached its limit

Info commandstats analysis commands spend unreasonable time and may cause excessive memory optimization.

Persistent blocking

1. Fork blocking

When a RDB or AOF rewrite occurs, the redis main thread calls fork to produce a child process to complete the persistent file rewrite

Use the info stats command to get the lastest_fork_usec metric, indicating that the last fork operation of redis was time-consuming

2. AOF disk flushing block

When AOF is enabled, the file is usually flushed once a second. When the hard disk pressure is too high, fsync needs to wait for the write to be completed.

View aof_delayed_fsync metrics in redis logs or info persistence statistics

Can use iotop difference which process may consume too much hard disk resources

3. HugePage write operation blocking

For operating systems with Transparent HugePages enabled, the copy memory page unit caused by each write command is changed from 4KB to 2MB

It slows down the execution time of write operations, resulting in a large number of slow queries for write operations

External cause

CPU competition

1. Process competition: redis is a typical CPU-intensive application. Use the top, sar commands to locate the point in time and processes consumed by CPU

2. Bind CPU: the common optimization is to bind the redis process to the CPU, with lower CPU context switching overhead. If the CPU child process is bound by CPU, there will be fierce CPU competition between the parent and child processes, which will greatly affect the redis stability.

Memory swapping

If the operating system swaps the memory used by redis to the hard disk, it will lead to a sharp decline in redis performance after swapping.

Identify the check method for redis memory swapping:

1. Query the redis process number

Redis-cli info server | grep process_id

2. Query memory exchange information according to process number

Cat / proc/ {process_id} / smaps | grep Swap

If the exchange volume is 0KB or individual 4KB, it is normal.

Prevent memory swapping:

1. Make sure the machine has enough available memory.

2. Ensure that all redis examples set the maximum available memory (maxmemory) to prevent uncontrollable growth of redis memory in extreme cases.

3. Reduce the priority of system using swap, such as echo 10 > / proc/sys/vm/swappiness.

Network problem

1. Connection rejection

Network flash disconnection: generally in the case of network cutoff or bandwidth depletion

Redis connection rejection:

New connections are denied when the number of connections is greater than maxclients. Rejected_connections index of info stats

The client accesses redis by NIO persistent connection or connection pooling as far as possible.

When redis is used in scenarios with a large number of distributed node access and short life cycle (such as Map/Reduce), it is recommended that you set tcp-keepalive and timeout parameters to allow redis to actively check and close invalid connections.

Connection overflow:

Process limit: the process can open the maximum number of files control, ulimit-n, usually 1024, redis with a large number of connections needs to increase this value.

Backlog queue overflow: the system uses backlog queues for specific port tcp connections. Redis defaults to 511and system backlog defaults to 128.On-line, you can use cron to perform netstat-s timed execution | grep overflowed statistics

2. Network delay

Measure the network delay between machines

Redis-cli-h {ip}-p {port}-latencyredis-cli-h {ip}-p {port}-latency-history completes a row of statistics in 15 seconds by default, and-I controls the sampling time redis-cli-h {ip}-p {port}-latency-dist statistics, which is sampled once every second.

3. Network card soft interrupt

Only one CPU can be used for a single Nic queue. Under high concurrency, the Nic data interaction is concentrated in the same CPU, which leads to the inability to make full use of multi-core CPU.

It generally appears in the scene of high network traffic throughput.

So much for sharing the reasons for the blocking of Redis. I hope the above content can be helpful to you and learn more. If you think the article is good, you can share it for more people to see. Zhang!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.