Shulou(Shulou.com)06/02 Report--

This article mainly explains "how to delete the 70 million Key of Reids". Interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Now let the editor take you to learn "how to delete the 70 million Key of Reids"!

Difficulty analysis

Shared Redis service cluster

Since the data of this business line is about 3G in Redis, it is not necessary to build a separate Redis service cluster. In the spirit of saving as much as possible, we decided to share a cluster with other projects (this cluster configuration: 16 nodes, 128 GB of memory, which is luxurious). The cluster configuration is as follows:

In the case of such a shared cluster, it is impossible to simply and violently release. Therefore, you can only choose how to delete the Key.

Key naming is not standard

To delete Key, you must first accurately locate which Key needs to be deleted. If you do not delete Key, it will affect the normal operation of other services! If the Key itself sets the expiration time, but some data needs to be persistent. However, the damn project manager has been speeding up the progress of the project, resulting in developers in many parts of the development process is not in place, such as Redis Key scattered in every corner of the project code; such as naming is not very standard. I don't know how to review code! Oh, there must be no time to review, the damn project manager.

I will randomly truncate the Key name in the payment service:

How's it going? Do you think the code written by our developers is very low~? don't laugh, in practice, there is something more low than this! I hope you don't meet it, or it's really painful.

Solution idea

After the above analysis, we briefly summarize as follows:

What we really care about is the Key with no expiration time set.

Do not delete Key by mistake, otherwise your performance will be lost next month.

Due to the naming and use of Key and its non-standard, it is very difficult to locate Key.

It seems that scanning for matching Key with the scan command won't work. We can only search through cyber manhunt.

Fortunately, Idea has a good search method, and spring-boot-starter-data-redis is used in this project. So I locate all the codes that operate on redis by searching RedisTemplate and StringRedisTemplate, the specific steps are as follows:

1. Count the prefixes of Key through these codes and input them into the text.

2. Use the python script to load the Key in the text and add the "*" wildcard after it.

3. Scan out these key through python script and scan command

4. In order to facilitate inspection, we do not directly use the del command to delete key. Before deleting key, we first get its serialized length by debug object key, and then perform the deletion and return the serialized length. In this way, we can count the serialization length of all key to get the amount of space we release. The key code is as follows:

Def get_key (rdbConn,start): try: keys_list = rdbConn.scan (start,count=2000) return keys_list exceptException,e: print e''Redis DEBUG OBJECT command got key info' 'def get_key_info (rdbConn KeyName): try: rpiple = rdbConn.pipeline () rpiple.type (keyName) rpiple.debug_object (keyName) rpiple.ttl (keyName) key_info_list = rpiple.execute () return key_info_list exceptException,e: print "INFO:" E def redis_key_static (key_info_list): keyType = key_info_list [0] keySize = key_info_list [1] ['serializedlength'] keyTtl = key_info_list [2] key_size_static (keyType,keySize,keyTtl)

In the above way, we can count how much memory has been released.

Because this cluster is so close to 70 million key:

So, when it was dawn the next day, I looked at it sleepily and finally deleted it, time 0713. The morning rush is coming.

Know shame and then be brave

I have never experienced the experience of clearing resources as a result of business offline. This incident really makes me feel that the subtleties are the truth of true kung fu. If we had followed the development specifications to use and design redis key in the first place, we wouldn't have wasted so much time. In order to make the naming and use of key more standardized, and to avoid this situation in the future, after waking up in the afternoon, I added a configuration and customized key serialization to the redis common component library, as follows:

ConfigurationProperties (prefix = "spring.redis.prefix") publicclassRedisKeyPrefixProperties {privateBoolean enable = Boolean.TRUE; privateString key; publicBoolean getEnable () {return enable;} publicvoid setEnable (Boolean enable) {this.enable = enable;} publicString getKey () {return key;} publicvoid setKey (String key) {this.key = key }} / * @ desc adds the prefix * @ author create by liming sun on 2020-07-21 14:09:51 * / publicclassPrefixStringKeySerializerextendsStringRedisSerializer {privateCharset charset = StandardCharsets.UTF_8; privateRedisKeyPrefixProperties prefix; publicPrefixStringKeySerializer (RedisKeyPrefixProperties prefix) {super (); this.prefix = prefix } @ Override publicString deserialize (@ Nullablebyte [] bytes) {String saveKey = newString (bytes, charset); if (prefix.getEnable ()! = null&& prefix.getEnable ()) {String prefixKey = spliceKey (prefix.getKey ()); int indexOf = saveKey.indexOf (prefixKey); if (indexOf > 0) {saveKeysaveKey = saveKey.substring (indexOf) }} return (saveKey.getBytes ()) = = null? Null: saveKey);} @ Override publicbyte [] serialize (@ NullableString key) {if (prefix.getEnable ()! = null&& prefix.getEnable ()) {key = spliceKey (prefix.getKey ()) + key;} return (key = = null? Null: key.getBytes (charset);} privateString spliceKey (String prefixKey) {if (StringUtils.isNotBlank (prefixKey) & &! prefixKey.endsWith (":")) {prefixKeyprefixKey = prefixKey + "::;} return prefixKey;}}

Use effect

In order to avoid this kind of work inefficient and have to do again, we specify in the development specification that the use of redis in new projects must be set to this configuration, and the prefix is set to: project number. In addition, the key in a module must be uniformly defined in the RedisKeyConstant class of the second-party library. The configuration is as follows:

Spring: redis: prefix: enable: true key: E00P01@Bean publicRedisTemplate redisTemplate (RedisConnectionFactory redisConnectionFactory) {RedisTemplate redisTemplate = newRedisTemplate (); redisTemplate.setConnectionFactory (redisConnectionFactory); / / key Serializer redisTemplate.setKeySerializer (newPrefixStringKeySerializer ()) that supports key prefix setting; redisTemplate.setValueSerializer (newGenericJackson2JsonRedisSerializer ()); return redisTemplate;}

In the above way, we can at least distinguish the key from the project dimension, avoiding the problem that multiple projects share the same cluster and lead to duplicate key. The key is divided from the project dimension. More convenient for management and operation and maintenance. If the management granularity of key is more fine, we can even refine it to a specific business dimension. We have carried out pressure tests in the test environment, and adding the key prefix has almost no effect on the performance of redis. Acceptable in terms of performance.

At this point, I believe you have a deeper understanding of "how to delete the 70 million Key of Reids". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.