Shulou(Shulou.com)05/31 Report--

Editor to share with you what you must pay attention to when using Redis, I believe that most people do not know much about it, so share this article for your reference, I hope you will learn a lot after reading this article, let's go to know it!

1. The usage specification of Redis

1.1.The main points of key specification

When designing the key of Redis, we should pay attention to the following points:

The business name is prefixed with a key, separated by a colon to prevent key collision overrides. For example, live:rank:1

To ensure that the semantics of the key is clear, the length of the key is as long as 30 characters.

Key prohibits the inclusion of special characters, such as spaces, line feeds, single and double quotation marks, and other escape characters.

Redis key as far as possible to set up ttl to ensure that unused Key can be cleaned or eliminated in time.

1.2.The main points of value specification

The value value of Redis can not be set at will.

First, it will be problematic to store a large amount of bigKey, which will lead to slow queries, excessive memory growth, and so on.

If it is a String type, the size of a single value is controlled within 10k.

In the case of hash, list, set, or zset types, the number of elements generally does not exceed 5000.

Second, choose the appropriate data type. Many friends only use the String type of Redis, which is set and get. In fact, Redis provides a wealth of data structure types, and some business scenarios are more suitable for other data results such as hash, zset, and so on.

Counterexample:

Set user:666:name jayset user:666:age 18

Positive example

Hmset user:666 name jay age 18

1.3. Set the expiration time for Key and pay attention to the key of different businesses at the same time, so as to spread the expiration time as much as possible.

Because the data of Redis is stored in memory, and memory resources are very valuable.

We generally use Redis as a cache rather than a database, so the life cycle of key should not be too long.

Therefore, your key, it is generally recommended to use expire to set the expiration time.

If a large number of key expires at a certain point in time, there may be stutters or even cache avalanches in Redis at that time, so the expiration time of key for different businesses should be scattered. Sometimes, the same business, you can also add a random value on the time, so that the expiration time is scattered.

1.4. It is recommended to use batch operations to improve efficiency.

When we write SQL every day, we all know that batch operation will be more efficient. It is more efficient to update 50 entries at a time than to cycle 50 times and update one at a time. In fact, the same is true of Redis operation commands.

The execution of a command by Redis client can be divided into four processes: 1. Send command-> 2. Command queue-> 3. Command execution-> 4. Returns the result. 1 and 4 are called RRT (round trip time for command execution). Redis provides batch operation commands, such as mget, mset, etc., which can effectively save RRT. However, most commands do not support batch operations, such as hgetall, and there is no mhgetall. Pipeline can solve this problem.

What is Pipeline? It can assemble a group of Redis commands, transmit them to Redis through RTT, and then return the execution results of these Redis commands to the client sequentially.

Let's first take a look at the model that executed n commands without using Pipeline:

The command was executed n times with Pipeline, and the whole process requires 1 RTT. The model is as follows:

2. Those orders that Redis has a hole in.

2.1. Be careful with O (n) complexity commands, such as hgetall, smember,lrange, etc.

Because Redis executes commands in a single thread. The time complexity of hgetall, smember and other commands is O (n). When n continues to increase, it will cause Redis CPU to keep soaring and block the execution of other commands.

Hgetall, smember,lrange and other commands are not necessarily impossible to use, need to comprehensively evaluate the amount of data, clear the value of n, and then decide. For example, hgetall, if there are more hash elements n, you can give priority to using hscan.

2.2 use Redis's monitor command with caution

The Redis Monitor command is used to print out the commands received by the Redis server in real time. If we want to know what command operations the client has done to the redis server, we can use the Monitor command to view it, but it is generally used for debugging, so try not to use it in production! Because the monitor command may cause redis's memory to continue to soar.

Monitor's model is purple, it will output all the commands executed in the Redis server, generally speaking, the QPS of the Redis server is very high, that is, if the monitor command is executed, the Redis server in the Monitor client output buffer will have a lot of "inventory", which takes up a lot of Redis memory.

2.3.The keys instruction cannot be used in the production environment

The Redis Keys command is used to find all key that match a given pattern pattern. If you want to check the number of key of a certain type of Redis, many partners think of using the keys command, as follows:

Keys key prefix *

However, the keys of redis is ergodic matching, and the complexity is O (n). The more data in the database, the slower it is. We know that redis is single-threaded, and if there is a lot of data, the keys instruction will cause the keys thread to block and the online service will stop until the instruction has been executed. Therefore, generally in a production environment, do not use the keys instruction. The official document also states:

Warning: consider KEYS as a command that should only be used in production environments with extreme care. It may ruin performance when it is executed against large databases. This command is intended for debugging and special operations, such as changing your keyspace layout. Don't use KEYS in your regular application code. If you're looking for a way to find keys in a subset of your keyspace, consider using sets.

In fact, you can use the scan directive, which provides pattern matching like the keys command. Its complexity is also O (n), but it steps through the cursor and does not block the redisthread; but it has a certain probability of repetition and needs to be deduplicated on the client side.

Scan supports incremental iterative commands, and incremental iterative commands also have disadvantages: for example, using the SMEMBERS command can return all the elements currently contained in the set key, but for incremental iterative commands such as SCAN, because the key may be modified during the incremental iteration of the key, the incremental iterative command can only provide limited guarantees for the elements returned.

2.4 prohibit the use of flushall and flushdb

The Flushall command is used to empty the data for the entire Redis server (delete all key for all databases).

The Flushdb command is used to clear all key in the current database.

These two commands are atomic and will not be terminated. Once the execution starts, the execution will not fail.

2.5 pay attention to using the del command

What commands do you usually use to delete key? Is it direct del? If you delete a key, you can use the del command directly. But have you ever thought about the time complexity of del? Let's discuss it on a case-by-case basis:

If you delete a key of type String, the time complexity is O (1), and you can del directly.

If you delete a List/Hash/Set/ZSet type, its complexity is O (n), where n represents the number of elements.

Therefore, if you delete a key of type List/Hash/Set/ZSet, the more elements, the slower it will be. When n is very large, you should pay special attention to blocking the main thread. So, if we don't use del, how should we delete it?

If it is a List type, you can execute lpop or rpop until all elements are deleted.

If it is a Hash/Set/ZSet type, you can first execute the hscan/sscan/scan query, and then execute hdel/srem/zrem to delete each element in turn.

Avoid using commands that are too complex, such as SORT and SINTER.

Executing commands with higher complexity consumes more CPU resources and blocks the main thread. So you should avoid executing aggregate commands such as SORT, SINTER, SINTERSTORE, ZUNIONSTORE, ZINTERSTORE and so on. It is generally recommended to put it on the client side to execute.

3. The operation of avoiding the pit in the actual combat of the project

3.1 points for attention in the use of distributed locks

A distributed lock is actually an implementation of a lock that controls different processes in a distributed system to access shared resources together. Distributed locks are required in business scenarios such as issuing orders and grabbing red packets. We often use Redis as a distributed lock, with the following considerations:

3.1.1 two commands SETNX + EXPIRE are written separately (typical error implementation example)

If (jedis.setnx (key_resource_id,lock_value) = = 1) {/ / lock expire (key_resource_id,100); / / set expiration time try {do something / / business request} catch () {} finally {jedis.del (key_resource_id); / / release lock}}

If the process crash may restart maintenance after performing the setnx lock and is about to execute the expire to set the expiration time, then the lock will be immortal, and other threads will never get the lock, so the general distributed lock can not be implemented in this way.

3.1.2 SETNX + value value is the expiration time (as some partners do, there are holes)

Long expires = System.currentTimeMillis () + expireTime; / / system time + set expiration time String expiresStr = String.valueOf (expires); / / if the current lock does not exist, return if (jedis.setnx (key_resource_id, expiresStr) = = 1) {return true;} / / if the lock already exists, get the expiration time of the lock String currentValueStr = jedis.get (key_resource_id) / / if the expiration time obtained is less than the current time of the system, if (currentValueStr! = null & & Long.parseLong (currentValueStr) has expired

< System.currentTimeMillis()) { // 锁已过期，获取上一个锁的过期时间，并设置现在锁的过期时间（不了解redis的getSet命令的小伙伴，可以去官网看下哈） String oldValueStr = jedis.getSet(key_resource_id, expiresStr); if (oldValueStr != null && oldValueStr.equals(currentValueStr)) { // 考虑多线程并发的情况，只有一个线程的设置值和当前值相同，它才可以加锁 return true; }} //其他情况，均返回加锁失败return false;} 这种方案的缺点： 过期时间是客户端自己生成的，分布式环境下，每个客户端的时间必须同步 没有保存持有者的唯一标识，可能被别的客户端释放/解锁。 锁过期的时候，并发多个客户端同时请求过来，都执行了jedis.getSet()，最终只能有一个客户端加锁成功，但是该客户端锁的过期时间，可能被别的客户端覆盖。 3.1.3： SET的扩展命令（SET EX PX NX）（注意可能存在的问题） if（jedis.set(key_resource_id, lock_value, "NX", "EX", 100s) == 1）{ //加锁 try { do something //业务处理 }catch(){　　}　　finally { jedis.del(key_resource_id); //释放锁 }} 这个方案还是可能存在问题： 锁过期释放了，业务还没执行完。 锁被别的线程误删。 3.1.4 SET EX PX NX + 校验唯一随机值,再删除（解决了误删问题，还是存在锁过期，业务没执行完的问题） if（jedis.set(key_resource_id, uni_request_id, "NX", "EX", 100s) == 1）{ //加锁 try { do something //业务处理 }catch(){　　}　　finally { //判断是不是当前线程加的锁,是才释放 if (uni_request_id.equals(jedis.get(key_resource_id))) { jedis.del(lockKey); //释放锁 } }} 在这里，判断是不是当前线程加的锁和释放锁不是一个原子操作。如果调用jedis.del()释放锁的时候，可能这把锁已经不属于当前客户端，会解除他人加的锁。 一般也是用lua脚本代替。lua脚本如下： if redis.call('get',KEYS[1]) == ARGV[1] then return redis.call('del',KEYS[1]) else return 0end; 3.1.5 Redisson框架 + Redlock算法 解决锁过期释放，业务没执行完问题+单机问题 Redisson 使用了一个Watch dog解决了锁过期释放，业务没执行完问题，Redisson原理图如下: 以上的分布式锁，还存在单机问题： 如果线程一在Redis的master节点上拿到了锁，但是加锁的key还没同步到slave节点。恰好这时，master节点发生故障，一个slave节点就会升级为master节点。线程二就可以获取同个key的锁啦，但线程一也已经拿到锁了，锁的安全性就没了。 针对单机问题，可以使用Redlock算法。有兴趣的朋友可以看下我这篇文章哈，七种方案！探讨Redis分布式锁的正确使用姿势 3.2 缓存一致性注意点 如果是读请求，先读缓存，后读数据库 如果写请求，先更新数据库，再写缓存 每次更新数据后，需要清除缓存 缓存一般都需要设置一定的过期失效 一致性要求高的话，可以使用biglog+MQ保证。 有兴趣的朋友，可以看下我这篇文章哈：并发环境下，先操作数据库还是先操作缓存？ 3.3 合理评估Redis容量，避免由于频繁set覆盖，导致之前设置的过期时间无效。 我们知道，Redis的所有数据结构类型，都是可以设置过期时间的。假设一个字符串，已经设置了过期时间，你再去重新设置它，就会导致之前的过期时间无效。 Redis setKey源码如下： void setKey(redisDb *db,robj *key,robj *val) { if(lookupKeyWrite(db,key)==NULL) { dbAdd(db,key,val); }else{ dbOverwrite(db,key,val); } incrRefCount(val); removeExpire(db,key); //去掉过期时间 signalModifiedKey(db,key);} 实际业务开发中，同时我们要合理评估Redis的容量，避免频繁set覆盖，导致设置了过期时间的key失效。新手小白容易犯这个错误。 3.4 缓存穿透问题 先来看一个常见的缓存使用方式：读请求来了，先查下缓存，缓存有值命中，就直接返回；缓存没命中，就去查数据库，然后把数据库的值更新到缓存，再返回。 缓存穿透：指查询一个一定不存在的数据，由于缓存是不命中时需要从数据库查询，查不到数据则不写入缓存，这将导致这个不存在的数据每次请求都要到数据库去查询，进而给数据库带来压力。 通俗点说，读请求访问时，缓存和数据库都没有某个值，这样就会导致每次对这个值的查询请求都会穿透到数据库，这就是缓存穿透。 缓存穿透一般都是这几种情况产生的： 业务不合理的设计，比如大多数用户都没开守护，但是你的每个请求都去缓存，查询某个userid查询有没有守护。 业务/运维/开发失误的操作，比如缓存和数据库的数据都被误删除了。 黑客非法请求攻击，比如黑客故意捏造大量非法请求，以读取不存在的业务数据。 如何避免缓存穿透呢？ 一般有三种方法。 如果是非法请求，我们在API入口，对参数进行校验，过滤非法值。 如果查询数据库为空，我们可以给缓存设置个空值，或者默认值。但是如有有写请求进来的话，需要更新缓存哈，以保证缓存一致性，同时，最后给缓存设置适当的过期时间。（业务上比较常用，简单有效） 使用布隆过滤器快速判断数据是否存在。即一个查询请求过来时，先通过布隆过滤器判断值是否存在，存在才继续往下查。 布隆过滤器原理：它由初始值为0的位图数组和N个哈希函数组成。一个对一个key进行N个hash算法获取N个值，在比特数组中将这N个值散列后设定为1，然后查的时候如果特定的这几个位置都为1，那么布隆过滤器判断该key存在。 3.5 缓存雪奔问题 缓存雪奔： 指缓存中数据大批量到过期时间，而查询数据量巨大，请求都直接访问数据库，引起数据库压力过大甚至down机。 缓存雪奔一般是由于大量数据同时过期造成的，对于这个原因，可通过均匀设置过期时间解决，即让过期时间相对离散一点。如采用一个较大固定值+一个较小的随机值，5小时+0到1800秒酱紫。 Redis 故障宕机也可能引起缓存雪奔。这就需要构造Redis高可用集群啦。 3.6 缓存击穿问题 缓存击穿： 指热点key在某个时间点过期的时候，而恰好在这个时间点对这个Key有大量的并发请求过来，从而大量的请求打到db。 缓存击穿看着有点像，其实它两区别是，缓存雪奔是指数据库压力过大甚至down机，缓存击穿只是大量并发请求到了DB数据库层面。可以认为击穿是缓存雪奔的一个子集吧。有些文章认为它俩区别，是区别在于击穿针对某一热点key缓存，雪奔则是很多key。 解决方案就有两种： 1.使用互斥锁方案。缓存失效时，不是立即去加载db数据，而是先使用某些带成功返回的原子操作命令，如(Redis的setnx）去操作，成功的时候，再去加载db数据库数据和设置缓存。否则就去重试获取缓存。 2. "永不过期"，是指没有设置过期时间，但是热点数据快要过期时，异步线程去更新和设置过期时间。 3.7、缓存热key问题 在Redis中，我们把访问频率高的key，称为热点key。如果某一热点key的请求到服务器主机时，由于请求量特别大，可能会导致主机资源不足，甚至宕机，从而影响正常的服务。 而热点Key是怎么产生的呢？主要原因有两个： 用户消费的数据远大于生产的数据，如秒杀、热点新闻等读多写少的场景。 请求分片集中，超过单Redi服务器的性能，比如固定名称key，Hash落入同一台服务器，瞬间访问量极大，超过机器瓶颈，产生热点Key问题。 那么在日常开发中，如何识别到热点key呢？ 凭经验判断哪些是热Key； 客户端统计上报； 服务代理层上报 如何解决热key问题？ Redis集群扩容：增加分片副本，均衡读流量； 对热key进行hash散列，比如将一个key备份为key1,key2……keyN，同样的数据N个备份，N个备份分布到不同分片，访问时可随机访问N个备份中的一个，进一步分担读流量； 使用二级缓存，即JVM本地缓存,减少Redis的读请求。 4、Redis配置运维 4.1 使用长连接而不是短连接，并且合理配置客户端的连接池 如果使用短连接，每次都需要过 TCP 三次握手、四次挥手，会增加耗时。然而长连接的话，它建立一次连接，redis的命令就能一直使用，酱紫可以减少建立redis连接时间。 连接池可以实现在客户端建立多个连接并且不释放，需要使用连接的时候，不用每次都创建连接，节省了耗时。但是需要合理设置参数，长时间不操作 Redis时，也需及时释放连接资源。 4.2 只使用 db0 Redis-standalone架构禁止使用非db0.原因有两个 一个连接，Redis执行命令select 0和select 1切换，会损耗新能。 Redis Cluster 只支持 db0，要迁移的话，成本高 4.3 设置maxmemory + 恰当的淘汰策略。 为了防止内存积压膨胀。比如有些时候，业务量大起来了，redis的key被大量使用，内存直接不够了，运维小哥哥也忘记加大内存了。难道redis直接这样挂掉？所以需要根据实际业务，选好maxmemory-policy(最大内存淘汰策略)，设置好过期时间。一共有8种内存淘汰策略： volatile-lru：当内存不足以容纳新写入数据时，从设置了过期时间的key中使用LRU（最近最少使用）算法进行淘汰； allkeys-lru：当内存不足以容纳新写入数据时，从所有key中使用LRU（最近最少使用）算法进行淘汰。 volatile-lfu：4.0版本新增，当内存不足以容纳新写入数据时，在过期的key中，使用LFU算法进行删除key。 allkeys-lfu：4.0版本新增，当内存不足以容纳新写入数据时，从所有key中使用LFU算法进行淘汰； volatile-random：当内存不足以容纳新写入数据时，从设置了过期时间的key中，随机淘汰数据；。 allkeys-random：当内存不足以容纳新写入数据时，从所有key中随机淘汰数据。 volatile-ttl：当内存不足以容纳新写入数据时，在设置了过期时间的key中，根据过期时间进行淘汰，越早过期的优先被淘汰； noeviction：默认策略，当内存不足以容纳新写入数据时，新写入操作会报错。 4.4 开启 lazy-free 机制 Redis4.0+版本支持lazy-free机制，如果你的Redis还是有bigKey这种玩意存在，建议把lazy-free开启。当开启它后，Redis 如果删除一个 bigkey 时，释放内存的耗时操作，会放到后台线程去执行，减少对主线程的阻塞影响。

The above are all the contents of this article "what are the main points you must pay attention to when using Redis?" Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.