Shulou(Shulou.com)06/02 Report--

With regard to the problem of Redis, I believe that many inexperienced people do not know what to do about it. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.

Yesterday I wrote an article about setting up my own redis cluster and using it in my own project. Looking at the faces written by others this morning, I found that redis was often asked in the interview (the direction of the main Java). So check the official documents and wheels built by others, and summarize some questions you must master in the redis interview and study. Every detail, it is impossible to cover all the content, try to write out the more common ones.

What is Redis?

Redis is an open source key-value database written in C language. Similar to Memcached, it supports relatively more value types, including string (string), list (linked list), set (collection), zset (sorted set-ordered collection), and hash (hash type). These data types support push/pop, add/remove, and take intersection union and difference sets, and richer operations, and these operations are atomic. On this basis, redis supports a variety of different sorting methods. Like memcached, data is cached in memory for efficiency. The difference is that redis will periodically write updated data to disk or modify operations to additional record files, and on this basis to achieve master-slave (master-slave) synchronization. Currently, Vmware is funding the development and maintenance of redis projects.

The difference and comparison between Redis and Memcached

1. Redis not only supports simple KBH data, but also provides storage of data structures such as list,set,zset,hash. Memcache supports a simple data type, String.

2. Redis supports data backup, that is, data backup in master-slave mode.

3. Redis supports data persistence, which can keep the data in memory on disk, and can be loaded and used again when rebooting, while Memecache stores all the data in memory.

The speed of redis is much faster than that of memcached

5. Memcached is a multithreaded, non-blocking IO multiplexing network model; Redis uses a single-threaded IO multiplexing model.

If you want to know more about it, you can check out this note on Mutu.com (highly recommended): "the Perplexity of two feet-Memcached and Redis": https://www.imooc.com/article/23549

The Choice of Redis and Memcached

Ultimate strategy: everything you do with Redis's String type can be replaced with Memcached in exchange for a better performance boost; in addition, Redis is preferred

What are the benefits of using redis?

(1) it is fast, because the data is stored in memory, and the advantage similar to HashMap,HashMap is that the time complexity of search and operation is O (1).

(2) supports rich data types and string,list,set,sorted set,hash

(3) transactions are supported, and all operations are atomic. the so-called atomicity means that changes to the data are either performed or not performed at all.

(4) rich features: can be used for caching, messages, set expiration time by key, and will be deleted automatically after expiration

Redis common data structure usage scenario

1. String

Common commands: set,get,decr,incr,mget, etc.

The String data structure is a simple key-value type, and value can actually be not only a String, but also a number.

General key-value caching application

Regular count: Weibo, fans, etc.

2.Hash

Common commands: hget,hset,hgetall, etc.

Hash is a mapping table for field and value of type string, and hash is particularly suitable for storing objects. For example, we can Hash data structure to store user information, product information and so on.

For example: the home page of a recent e-commerce website project uses redis's hash data structure to cache, because the home page of a website is visited the most, so usually the home page of a website can be cached through redis cache to improve performance and concurrency. I use jedis client to connect and operate the redis cluster or stand-alone redis I built. Using jedis can easily operate redis. Generally speaking, it is not difficult to build a simple cluster to implement redis as a cache. Those who are interested can read the article I wrote yesterday:

"it is easy to understand the principle, construction and use of redis cluster": https://juejin.im/post/5ad54d76f265da23970759d3

3.List

Common commands: lpush,rpush,lpop,rpop,lrange, etc.

List is a linked list. Redis list has many application scenarios, and it is also one of the most important data structures of Redis. For example, Weibo's follow list, fan list, latest news ranking and other functions can be implemented using Redis's list structure.

Redis list is implemented as a two-way linked list, that is, it can support reverse lookup and traversal, which is more convenient to operate, but brings some additional memory overhead.

4.Set

Common commands:

Sadd,spop,smembers,sunion et al.

The function provided by set is similar to that of list, which is a list function, except that set can arrange weights automatically.

Set is a good choice when you need to store list data and do not want duplicate data, and set provides an important interface to determine whether a member is in an set collection, which list cannot provide.

In the Weibo application, you can store all the followers of a user in a collection and all their fans in a collection. Redis can be very convenient to achieve functions such as common concern, common preferences, second friends and so on.

5.Sorted Set

Common commands: zadd,zrange,zrem,zcard, etc.

Compared with set, sorted set adds a weight parameter score, so that the elements in the collection can be arranged in order according to score.

For example, in the LVB system, real-time ranking information includes live room online user list, various gift ranking lists, on-screen comment messages (which can be understood as message ranking by message dimension) and other information, which is suitable to be stored using the SortedSet structure in Redis.

There are 2000w data in MySQL and only 20w data in Redis. How to ensure that the data in Redis are hot data (what are the data elimination strategies in redis? )

Related knowledge: when the redis in-memory dataset size rises to a certain size, the data phase-out strategy (recovery strategy) will be implemented. Redis provides 6 data elimination strategies:

1. Volatile-lru: select the least recently used data elimination from the dataset with an expiration time set (server. DB [I]. Obsolete).

2. Volatile-ttl: select the expired data from the dataset (server. DB [I] .expires) that will expire.

3. Volatile-random: eliminate data from any selected dataset with an expiration time set (server. DB [I]. Expires).

4. Allkeys-lru: select the least recently used data elimination from the dataset (server. DB [I]. Dict).

5. Allkeys-random: eliminate data from any selection from the dataset (server. DB [I]. Dict)

6. No-enviction (expulsion): prohibition of expulsion data

How to solve the concurrency competition problem of Redis?

Redis is a single-process single-thread mode, using queue mode to change concurrent access into serial access. Redis itself does not have the concept of lock, and Redis does not compete for multiple client connections, but problems such as connection timeout, data conversion errors, blocking, and client closure occur when Jedis clients access Redis concurrently, all of which are caused by client connection confusion. There are two solutions to this:

1. From the client point of view, in order to ensure that each client communicates with Redis in a normal and orderly manner, the connection is pooled, and the internal lock synchronized is used to read and write Redis on the client.

two。 From the server point of view, the lock is realized by using setnx.

Note: for the first, the application needs to handle the synchronization of resources on its own, and the methods you can use are more common, you can use synchronized or lock;, the second setnx command that requires Redis, but you need to pay attention to some problems.

How does the Redis recycling process work? What algorithm is used for Redis recycling?

Redis memory recovery: LRU algorithm (well written, recommended): https://www.cnblogs.com/WJ5888/p/4371647.html

Redis massive data insertion

Explanation given by the official document: http://www.redis.cn/topics/mass-insert.html

Advantages, disadvantages and types of Redis partitions

Explanation provided by the official documentation: http://www.redis.net.cn/tutorial/3524.html

How to expand the persistent data and cache of Redis?

Persistence and caching mechanism of redis: https://blog.csdn.net/tr1912/article/details/70197085?foxhandler=RssReadRenderProcessHandler

Capacity expansion can be achieved through the redis cluster. I used my own redis cluster when I was working on the project.

Then I wrote an article about redis cluster: "it is easy to understand the principle, construction and use of redis cluster": https://juejin.im/post/5ad54d76f265da23970759d3

Common performance problems and solutions for Redis:

Master is best not to do any persistence work, such as RDB memory snapshots and AOF log files

If the data is important, a Slave enables AOF to back up data, and the policy is set to synchronize once per second.

For the speed of master-slave replication and the stability of connection, Master and Slave are better located in the same local area network.

Try to avoid adding slave libraries to the stressed master libraries.

Redis and message queuing

Author: Weng Wei

Link: https://www.zhihu.com/question/20795043/answer/345073457

Do not use redis for message queuing, which is not the design goal of redis. But too many people use redis to do the message queue, the author of redis can not take it. In addition, based on the core code of redis, another message queue disque is implemented: antirez/disque: https://github.com/antirez/disque deployment, protocol and other aspects are very similar to redis, and support clustering, delayed messages and so on.

I am in contact with the website process more or use redis to do cache, such as second kill system, home page cache and so on.

Haowen Mark

The following articles are highly recommended.

"the way to go deep into Redis: principle analysis, scene use and video interpretation": https://zhuanlan.zhihu.com/p/28073983:

Mainly introduced: Redis cluster open source solution, Redis protocol introduction and persistent Aof file parsing, Redis short connection performance optimization and so on, the article is too much practical information, large capacity, it is recommended to have a look at plenty of time. In addition, the article also provides a video explanation, which can be said to be very attentive.

"typical scenario of Aliyun Redis mixed storage: how to easily build a video studio system": https://yq.aliyun.com/articles/582487?utm_content=m_46529:

This paper mainly introduces the video studio system, and how to use Aliyun Redis hybrid storage examples to easily and quickly build video studio services with large data volume and low latency. It also introduces the use scenario of the data structure of redis that we have improved before.

"some pitfalls Meituan stepped on on Redis-some questions encountered by 5.redis cluster": http://carlosfu.iteye.com/blog/2254573: mainly introduces two common problems of redis clusters, and then shares some good articles about redis clusters.

After reading the above, have you mastered the methods of Redis problems? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.