Shulou(Shulou.com)05/31 Report--

This article mainly talks about "what are the development specifications of Redis". Interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Now let the editor take you to learn "what are the development specifications of Redis"?

I. key value design

1. Key name design

Readability and manageability

Prefixed with business name (or database name) (to prevent key conflicts), separated by colons, such as business name: table name: id

Ugc:video:1

Conciseness

Under the premise of ensuring semantics, control the length of key. When there are more key, the memory footprint should not be ignored, for example:

User: {uid}: friends:messages: {mid} is reduced to u: {uid}: fr:m: {mid}.

Do not include special characters

Counterexample: includes spaces, line breaks, single and double quotation marks, and other escape characters

2. Value design

Reject bigkey

Prevent network card traffic and slow query. The type of string should be controlled within 10KB, and the number of hash, list, set and zset elements should not exceed 5000.

Counterexample: a list with 2 million elements.

For non-string bigkey, do not use del deletion, but use hscan, sscan, and zscan to delete progressively. At the same time, you should pay attention to prevent the automatic deletion of bigkey expiration time (for example, if a zset setting of 2 million expires within 1 hour, it will trigger del operation and cause blocking, and the operation will not fail to appear in slow query (latency can be found). Select the appropriate data type for search method and delete method.

For example: entity type (to reasonably control and use data structure memory coding to optimize configuration, such as ziplist, but also pay attention to the balance between memory savings and performance) counterexample:

Set user:1:name tom set user:1:age = 19 set user:1:favor football

Positive example:

Hmset user:1 name tom age = 19 favor football

Control the lifecycle of key

Redis is not a trash can. It is recommended to use expire to set the expiration time (conditions allow you to break up the expiration time to prevent centralized expiration). Data that does not expire should focus on idletime.

Second, command use

1. The O (N) command focuses on the number of N

For example, hgetall, lrange, smembers, zrange, sinter, etc., are not unusable, but you need to be clear about the value of N. Ergodic requirements can be replaced by hscan, sscan, and zscan.

2. Disable the command

Prohibit the online use of keys, flushall, flushdb, etc., disable commands through redis's rename mechanism, or use scan to deal with them step by step.

3. Rational use of select

The multi-database of redis is weak, which is distinguished by numbers, and many clients have poor support. At the same time, there will be interference if multi-database is actually used for multi-business or single-thread processing.

4. Use batch operations to improve efficiency

Native commands: for example, mget, mset.

Non-native commands: you can use pipeline to improve efficiency.

However, it is important to control the number of elements in a batch operation (for example, less than 500, which is actually related to the number of element bytes).

Note the difference between the two:

Native is atomic operation, pipeline is non-atomic operation.

Pipeline can package different commands, but can't do it natively.

Pipeline needs to be supported by both client and server.

5. Excessive use of Redis transaction function is not recommended.

The transaction function of Redis is weak (rollback is not supported), and the cluster version (self-developed and official) requires that the key of a transaction operation must be on a slot (can be solved using hashtag function)

6. Redis cluster version has special requirements on using Lua.

1. All key should be passed by the KEYS array. The redis command called in redis.call/pcall, the location of key must be KEYS array, otherwise it directly returns error, "- ERR bad lua script for redis cluster, all the keys that the script uses should be passed using the KEYS arrayrn" 2, all key must be on 1 slot, otherwise it directly returns error, "- ERR eval/evalsha command keys must in same slotrn".

7. Monitor command

When using the monitor command if necessary, be careful not to use it for a long time.

III. Client use

1. Avoid using one Redis instance for multiple applications

Irrelevant business split, public data to do services.

2. use connection pooling

Can effectively control the connection, while improving efficiency, standard use:

Execute the command as follows:

Jedis jedis = null; try {jedis = jedisPool.getResource (); / / the specific command jedis.executeCommand ()} catch (Exception e) {logger.error ("op key {} error:" + e.getMessage (), key, e);} finally {/ / Note that the connection is not closed here, but Jedis is returned to the resource pool in JedisPool mode. If (jedis! = null) jedis.close ();}

3. Fuse function

For high concurrency, it is recommended that the client add a circuit breaker (such as netflix hystrix)

4. Reasonable encryption

Set a reasonable password and use SSL encrypted access if necessary (supported by Aliyun Redis)

5. Elimination strategy

Select maxmemory-policy (maximum memory elimination strategy) and set the expiration time according to your own business type.

The default policy is volatile-lru, that is, after the maximum memory is exceeded, lru algorithm is used to eliminate key in expired keys to ensure that unexpired data will not be deleted, but OOM problems may occur.

Other strategies are as follows:

Allkeys-lru: delete the key according to the LRU algorithm, regardless of whether the data has set the timeout property or not, until enough space is freed.

Allkeys-random: randomly delete all keys until there is enough space.

Volatile-random: randomly delete expired keys until enough space is made available.

Volatile-ttl: deletes the data that is about to expire recently based on the ttl property of the key object. If not, fall back to the noeviction policy.

Noeviction: no data is stripped out, all writes are rejected and the client error message "(error) OOM command not allowed when used memory" is returned, where Redis only responds to read operations.

IV. Related tools

1. Data synchronization

Data synchronization between redis can be done using: redis-port

2. Big key search

Redis big key search tool

3. Key search for hot spots

The internal implementation uses monitor, so redis-faina Ali Cloud Redis, which is recommended to use facebook for a short time, has solved hot key issues at the kernel level.

5. Delete bigkey

You can use pipeline acceleration to do the following. Redis 4.0 already supports asynchronous deletion of key. Welcome to use it.

1. Hash deletion: hscan + hdel

Public void delBigHash (String host, int port, String password, String bigHashKey) {Jedis jedis = new Jedis (host, port); if (password! = null & &! ".equals (password)) {jedis.auth (password);} ScanParams scanParams = new ScanParams (). Count (100); String cursor =" 0 "; do {ScanResult scanResult = jedis.hscan (bigHashKey, cursor, scanParams); List entryList = scanResult.getResult () If (entryList! = null & &! entryList.isEmpty ()) {for (Entry entry: entryList) {jedis.hdel (bigHashKey, entry.getKey ());}} cursor = scanResult.getStringCursor ();} while (! "0" .equals (cursor)); / / delete bigkey jedis.del (bigHashKey);}

2. Delete List: ltrim

Public void delBigList (String host, int port, String password, String bigListKey) {Jedis jedis = new Jedis (host, port); if (password! = null & &! ".equals (password)) {jedis.auth (password);} long llen = jedis.llen (bigListKey); int counter = 0; int left = 100; while (counter < llen) {/ / truncate jedis.ltrim (bigListKey, left, llen) from the left at a time Counter + = left;} / / finally delete key jedis.del (bigListKey);}

3. Delete Set: sscan + srem

Public void delBigSet (String host, int port, String password, String bigSetKey) {Jedis jedis = new Jedis (host, port); if (password! = null & &! ".equals (password)) {jedis.auth (password);} ScanParams scanParams = new ScanParams (). Count (100); String cursor =" 0 "; do {ScanResult scanResult = jedis.sscan (bigSetKey, cursor, scanParams); List memberList = scanResult.getResult () If (memberList! = null & &! memberList.isEmpty ()) {for (String member: memberList) {jedis.srem (bigSetKey, member);}} cursor = scanResult.getStringCursor ();} while (! "0" .equals (cursor)); / / delete bigkey jedis.del (bigSetKey);}

4. SortedSet deletion: zscan + zrem

Public void delBigZset (String host, int port, String password, String bigZsetKey) {Jedis jedis = new Jedis (host, port); if (password! = null & &! ".equals (password)) {jedis.auth (password);} ScanParams scanParams = new ScanParams (). Count (100); String cursor =" 0 "; do {ScanResult scanResult = jedis.zscan (bigZsetKey, cursor, scanParams); List tupleList = scanResult.getResult () If (tupleList! = null & &! tupleList.isEmpty ()) {for (Tuple tuple: tupleList) {jedis.zrem (bigZsetKey, tuple.getElement ());}} cursor = scanResult.getStringCursor ();} while (! "0" .equals (cursor)); / / delete bigkey jedis.del (bigZsetKey) } at this point, I believe you have a deeper understanding of "what are the development specifications of Redis?" you might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.