Shulou(Shulou.com)05/31 Report--

In this article, the editor introduces in detail "what are the most frequently asked questions in Redis", with detailed content, clear steps and proper handling of details. I hope that this article "what are the most frequently asked questions in Redis" can help you solve your doubts.

1. What is redis?

Redis is a memory-based high-performance key-value database.

Characteristics of 2.Reids

Redis is essentially an in-memory database of Key-Value type, much like memcached, the whole database is loaded and operated in memory, and the database data is saved to the hard disk by asynchronous operation periodically. Because of its pure memory operation, Redis has excellent performance, which can handle more than 100000 read and write operations per second, making it the fastest Key-Value DB known.

Redis's outstanding place is not only performance, Redis's greatest charm is to support the preservation of a variety of data structures, in addition, the maximum limit of a single value is 1GB, unlike memcached can only save 1MB data, so Redis can be used to achieve many useful functions, such as using his List to do FIFO two-way linked list, to achieve a lightweight high-performance message queue service, with his Set can do high-performance tag system and so on. In addition, Redis can also set the expire time for the stored Key-Value, so it can also be used as an enhanced version of memcached.

The main disadvantage of Redis is that the database capacity is limited by physical memory and can not be used for high-performance read and write of massive data, so the suitable scenarios of Redis are mainly limited to high-performance operations and operations with a small amount of data.

3. What are the benefits of using redis?

(1) it is fast, because the data is stored in memory, and the advantage similar to HashMap,HashMap is that the time complexity of search and operation is O (1).

(2) supports rich data types and string,list,set,sorted set,hash

1) String commands: set/get/decr/incr/mget, etc.; Application scenarios: String is the most commonly used data type, and ordinary key/value storage can be classified into this category. Implementation: String is stored in redis as a string by default, which is referenced by redisObject. When you encounter incr, decr and other operations, it will be converted to numeric calculation. In this case, the encoding field of redisObject is int. 2) Hash common commands: hget/hset/hgetall and other application scenarios: we need to store a user information object data, including user ID, user name, age and birthday. Through the user ID, we want to obtain the user's name or age or birthday; implementation: the Hash of Redis is actually an internally stored Value as a HashMap, and provides direct access to this Map member. As shown in the figure, Key is the user ID and value is a Map. The key of this Map is the property name of the member, and value is the property value. In this way, the modification and access of the data can be directly through the Key of the internal Map (the key of the internal Map is called field in Redis), that is, the corresponding attribute data can be manipulated through key (user ID) + field (attribute tag). Currently, there are two ways to implement HashMap: when the number of HashMap members is relatively small, Redis will use a compact storage method similar to an one-dimensional array in order to save memory, instead of using the real HashMap structure, then the encoding of the corresponding value redisObject is zipmap, and when the number of members increases, it will be automatically converted to the real HashMap, and the encoding will be ht. Hash 3) List common commands: lpush/rpush/lpop/rpop/lrange, etc.; application scenarios: there are many application scenarios of Redis list, and it is also one of the most important data structures of Redis, such as twitter's watch list, fan list, etc. can be implemented using Redis's list structure. Implementation: the implementation of Redis list is a two-way linked list, that is, it can support reverse search and traversal, which is more convenient to operate, but it brings some additional memory overhead. Many implementations within Redis, including sending buffer queues, also use this data structure. 4) Set common commands: sadd/spop/smembers/sunion, etc. Application scenario: the external function provided by Redis set is similar to that of list, except that set can arrange weights automatically. When you need to store a list of data and do not want to have duplicate data, set is a good choice, and set provides an important interface to determine whether a member is in a set collection, which list cannot provide. Implementation: the internal implementation of set is a HashMap whose value is always null. In fact, it quickly arranges the weight by calculating hash, which is why set can determine whether a member is in the collection or not. 5) Sorted Set common commands: zadd/zrange/zrem/zcard, etc.; Application scenarios: the usage scenario of Redis sorted set is similar to that of set, except that set is not automatically ordered, while sorted set can sort members by providing an additional parameter of score, and it is inserted in order, that is, automatic sorting. When you need an ordered and non-repeating list of collections, you can choose the sorted set data structure. For example, twitter's public timeline can be stored as a score with the publication time, so that the acquisition is automatically sorted according to time. Implementation: the internal use of Redis sorted set HashMap and jump table (SkipList) to ensure data storage and order, HashMap is put in the member to score mapping, while the jump table is stored in all the members, sorting according to the score stored in HashMap, the use of jump table structure can achieve higher search efficiency, and relatively simple in implementation.

(3) transactions are supported, and all operations are atomic. the so-called atomicity means that changes to the data are either performed or not performed at all.

(4) rich features: can be used for caching, messages, set expiration time by key, and will be deleted automatically after expiration

What are the advantages of 4.redis over memcached?

(1) all values of memcached are simple strings, and redis, as its replacement, supports richer data types.

(2) redis is much faster than memcached. (3) redis can persist its data.

What are the differences between 5.Memcache and Redis?

(1) Storage method Memecache stores all the data in memory, and will hang up when the power is off, and the data cannot exceed the memory size. Some of the Redis is stored on the hard disk, which ensures the persistence of the data.

(2) data support type Memcache is relatively simple to support data types. Redis has complex data types.

(3) using the underlying model, the underlying implementation between them and the application protocol for communicating with the client are different. Redis directly builds its own VM mechanism, because the general system calls system functions, it will waste a certain amount of time to move and request.

What scenarios does 6.redis apply to?

Redis is most suitable for all scenarios of data in-momory, such as:

(1), session cache (Session Cache)

One of the most common scenarios where Redis is used is session caching (session cache). The advantage of caching sessions with Redis over other stores (such as Memcached) is that Redis provides persistence

(2), full-page cache (FPC)

In addition to the basic session token, Redis provides a very simple FPC platform. Back to consistency, even if the Redis instance is restarted, users will not see a drop in page loading speed because of disk persistence, which is a great improvement, similar to PHP native FPC.

(3), queue

One of the advantages of Reids in the field of memory storage engines is that it provides list and set operations, which makes Redis a good message queuing platform to use. The operation used by Redis as a queue is similar to the push/pop operation of list by a native program language such as Python.

If you quickly search for "Redis queues" in Google, you will immediately find a large number of open source projects designed to use Redis to create very good back-end tools to meet a variety of queue needs. For example, Celery has a background that uses Redis as broker, which you can check from here.

(4), ranking / counter

Redis does a great job of incrementing or decrementing numbers in memory. Set and Sorted Set also make it easy for us to perform these operations, and Redis just happens to provide these two data structures. So, to get the top 10 users from the sorted set-we call it "user_scores", we just need to do something like this:

Of course, this assumes that you are sorting incrementally according to the scores of your users. If you want to return the user and the user's score, you need to do this:

ZRANGE user_scores 0 10 WITHSCORES

Agora Games is a good example, implemented in Ruby, and its ranking uses Redis to store data, as you can see here.

(5), publish / subscribe

Last (but certainly not least) is Redis's publish / subscribe capabilities. There are indeed a lot of publish / subscribe usage scenarios.

7. Cache invalidation strategy and primary key invalidation mechanism of redis

As a cache system to regularly clean up invalid data, it needs a primary key invalidation and elimination strategy.

In Redis, a key with a lifetime is called volatile. When you create a cache, you set the lifetime for a given key, which may be deleted when the key expires (lifetime is 0).

1. Some operations that affect the survival time

The lifetime can be removed by using the DEL command to delete the entire key, or the original data can be overwritten by the SET and GETSET commands, that is, the lifetime of the current data is different after modifying the value corresponding to the key and overwriting it with the same key and value.

For example, performing INCR commands on a key, LPUSH commands on a list, or HSET commands on a hash table do not change the lifetime of the key itself. On the other hand, if you use RENAME to rename a key, the survival time of the renamed key is the same as before.

Another possibility of the RENAME command is to try to rename one key with lifetime to another another_key with lifetime, where the old another_key (and its lifetime) is deleted, and the old key is renamed another_key, so that the new another_key has the same lifetime as the original key. Using the PERSIST command, you can remove key's lifetime and make key a persistent key again without deleting the key.

2. How to update the survival time

You can EXPIRE a key that already has a time to live, and the new time to live replaces the old time to live. The precision of expiration time has been controlled within 1ms, and the time complexity of primary key failure is O (1).

EXPIRE is used in conjunction with the TTL command, and TTL can view the current lifetime of key. Successful setting returns 1, and 0 when key does not exist or cannot set time to live for key.

Maximum cache configuration

In redis, users are allowed to set the maximum memory size used

Server.maxmemory

The default is 0, and no maximum cache is specified. If new data is added and exceeds the maximum memory, it will crash redis, so be sure to set it. When the redis in-memory dataset size rises to a certain size, the data elimination strategy is implemented.

Redis provides 6 data elimination strategies:

.volatile-lru: select the least recently used data elimination from the dataset with an expiration time set (server. DB [I]. Obsolete).

.volatile-ttl: select the expired data from the dataset (server. DB [I] .expires) that will expire.

.volatile-random: data elimination from any selected dataset with an expiration time set (server.db [I] .expires)

.allkeys-lru: select the least recently used data elimination from the dataset (server. DB [I] .dict).

.allkeys-random: data elimination from any selection of the dataset (server. DB [I] .dict)

.no-enviction (expulsion): prohibition of eviction data

Note the six mechanisms here. Volatile and allkeys specify whether to phase out data from a dataset with an expiration time or from all datasets, and the following lru, ttl, and random are three different phase-out strategies, plus a no-enviction never recycling strategy.

Use policy rules:

1. If the data shows a power-law distribution, that is, some data access frequency is high and some data access frequency is low, then use allkeys-lru

2. If the data is evenly distributed, that is, all data are accessed at the same frequency, use allkeys-random

Three data elimination strategies:

Ttl and random are easier to understand and easier to implement. The main reason is that Lru has recently used the least elimination strategy, and the design will sort the key by failure time, and then select the key that fails first to be eliminated.

8. Why does redis need to put all the data in memory?

In order to achieve the fastest reading and writing speed, Redis reads all the data into memory and writes the data to disk asynchronously. So redis has the characteristics of high speed and data persistence. If you do not put the data in memory, the speed of the disk Imax O will seriously affect the performance of the redis. Today, when memory is getting cheaper and cheaper, redis will become more and more popular.

If the maximum used memory is set, the new value cannot be inserted after the number of records of the data has reached the memory limit.

9.Redis is single process and single thread.

Redis uses queue technology to change concurrent access into serial access, which eliminates the overhead of traditional database serial control.

How to solve the concurrency competition problem of 10.redis?

Redis is a single-process single-thread mode, using queue mode to change concurrent access into serial access. Redis itself does not have the concept of lock, and Redis does not compete for multiple client connections, but problems such as connection timeout, data conversion errors, blocking, and client closing connections occur when Jedis clients access Redis concurrently.

Due to confusion of client connection. There are two solutions to this:

1. From the client point of view, in order to ensure that each client communicates with Redis in a normal and orderly manner, the connection is pooled, and the internal lock synchronized is used to read and write Redis on the client.

two。 From the server point of view, the lock is realized by using setnx.

Note: for the first, the application needs to handle the synchronization of resources on its own, and the methods you can use are more common, you can use synchronized or lock;, the second setnx command that requires Redis, but you need to pay attention to some problems.

11. Common performance problems and solutions of redis:

1) .Master writes memory snapshots, and the save command dispatches the rdbSave function, which will block the work of the main thread. When the snapshot is relatively large, it will have a great impact on performance and will intermittently suspend the service. Therefore, it is best not to write memory snapshots for Master.

2). Master AOF persistence, if you do not rewrite AOF files, this persistence method will have the least impact on performance, but AOF files will continue to grow, and excessive AOF files will affect the recovery speed of Master restart. Master is best not to do any persistence work, including memory snapshots and AOF log files, especially do not enable memory snapshots to be persistent

If the data is critical, a Slave enables AOF to back up data, and the policy is to synchronize once per second.

3) when the .Master calls BGREWRITEAOF to rewrite the AOF file, AOF will take up a lot of CPU and memory resources when rewriting, resulting in high service load and temporary service suspension.

4)。 Due to the performance problem of Redis master-slave replication, for the speed of master-slave replication and the stability of connection, Slave and Master should be in the same local area network.

Understanding of 12.redis things CAS (check-and-set operation to achieve optimistic locking)?

Like many other databases, Redis as a NoSQL database also provides a transaction mechanism. In Redis, the four commands MULTI/EXEC/DISCARD/WATCH are the cornerstones of our transactions. I believe that this concept is not unfamiliar to developers with experience in developing department-based databases, but even so, we will briefly list

In Redis

The implementation characteristics of the transaction:

1)。 All commands in the transaction will be executed sequentially, and during the transaction execution, Redis will not provide any service for the requests of other clients, thus ensuring that all commands in the transaction will be executed by atoms.

2)。 Compared with transactions in a relational database, if a command fails in a Redis transaction, subsequent commands will continue to be executed.

3)。 We can start a transaction with the MULTI command, which can be interpreted as a "BEGIN TRANSACTION" statement by people with experience in serial database development. Commands executed after this statement will be treated as operations within the transaction, and finally we can commit / roll back all operations within the transaction by executing the EXEC/DISCARD command. These two Redis commands can be considered equivalent to COMMIT/ROLLBACK statements in a relational database.

4)。 Before the transaction starts, if there is a communication failure between the client and the server and the network is disconnected, all subsequent statements to be executed will not be executed by the server. However, if the network outage event occurs after the client executes the EXEC command, then all commands in the transaction are executed by the server.

5)。 When using Append-Only mode, Redis writes all writes within the transaction to disk in this call by calling the system function write. However, if a system crash occurs during the writing process, such as downtime caused by a power failure, only part of the data may be written to disk, while the other part of the data may have been lost. The Redis server performs a series of necessary consistency checks when it restarts, and once a similar problem is found, it exits immediately and gives the appropriate error prompt. At this point, we need to make full use of the redis-check-aof tool provided in the Redis toolkit, which can help us locate data inconsistencies and roll back some of the data that has been written. After the repair, we can restart the Redis server again.

13.WATCH commands and optimistic locks based on CAS?

In Redis transactions, the WATCH command can be used to provide CAS (check-and-set) functionality. Suppose we monitor multiple Keys before the transaction executes through the WATCH command. If any Key value changes after the WATCH, the transaction executed by the EXEC command will be abandoned and the Null multi-bulk reply will be returned to notify the caller of the transaction.

Execution failed. For example, we assume again that the incr command is not provided in Redis to accomplish the atomic increment of key values, and if we want to achieve this function, we have to write our own code. The pseudo code is as follows:

Val = GET mykey

Val = val + 1

SET mykey $val

The above code can only ensure that the execution result is correct in the case of a single connection, because if multiple clients are executing the code at the same time, there will be an error scenario that often occurs in multithreaded programs-race condition. For example, clients An and B both read the original value of mykey at the same time, assuming that the value is 10, and then both clients add this value and set it back to the Redis server, which results in a mykey result of 11 instead of 12. To solve a similar problem, we need the help of the WATCH command, as shown in the following code:

WATCH mykey

Val = GET mykey

Val = val + 1

MULTI

SET mykey $val

EXEC

Different from the previous code, the new code monitors the key through the WATCH command before getting the value of mykey, and then surrounds the set command in the transaction, which effectively ensures that each connection before executing EXEC, if the mykey value obtained by the current connection is modified by the client of another connection, then the EXEC command of the current connection will fail. In this way, after judging the return value, the caller can know whether the val has been reset successfully.

14. Have you ever used a Redis distributed lock? what is it?

First use setnx to scramble for the lock, and then use expire to add an expiration time to the lock to prevent the lock from forgetting to release.

At this point, the other person will tell you that you have a good answer, and then ask what happens if the process accidentally crash or restarts maintenance before expire is executed after setnx.

At this point you should give surprising feedback: Oh, yes, this lock will never be released. Then you need to scratch your head, pretend to think for a moment, as if the next result is your own initiative, and then answer: I remember that the set instruction has very complex parameters, which should be able to combine setnx and expire into one instruction at the same time! At this time, the other party will show a smile, the heart began to say: press, this boy is not bad.

15. What if there are 100 million key in Redis, of which 10w key start with a fixed known prefix, what if we find them all?

Use the keys directive to scan out the key list for the specified mode.

The other party then asked: if this redis is providing services to online business, what is the problem with using the keys instruction?

At this point you have to answer one of the key features of redis: the single-threaded nature of redis. The keys instruction will cause the thread to block for a period of time, and the online service will stop until the instruction has been executed before the service can resume. At this time, you can use the scan instruction. The scan instruction can extract the key list of the specified mode without blocking, but there will be a certain probability of repetition. It can be deduplicated on the client side, but the overall time will be longer than using the keys instruction directly.

16. Have you ever used Redis as an asynchronous queue? how do you use it?

Generally, list structure is used as a queue, rpush produces messages, and lpop consumes messages. When there is no message from lpop, sleep appropriately and try again later.

If the other party asks whether it is possible not to use sleep? List also has an instruction called blpop, which blocks until the message arrives when there is no message.

What if the other party asks if it can be produced and consumed more than once? Using the pub/sub topic subscriber mode, you can implement message queuing for 1VR N.

If the other party asks what are the shortcomings of pub/sub? In the case of consumers offline, the produced messages will be lost, and professional message queues such as rabbitmq will have to be used.

If the other party asks redis how to implement the delay queue? I guess now you really want to beat the interviewer to death. If you have a baseball bat in your hand, why do you ask in such detail? But you are restrained, and then reply calmly: use sortedset, take the timestamp as score, the message content as key to call zadd to produce the message, and the consumer uses the zrangebyscore instruction to get the data polling before N seconds for processing.

At this point, the interviewer has secretly given you a thumbs up. But what he doesn't know is that at the moment you have your middle finger up, behind the chair.

17. If there are a large number of key that need to be set to expire at the same time, what do you generally need to pay attention to?

If a large number of key expiration times are set too centrally, redis may experience temporary stutters at that point of expiration. Generally, it is necessary to add a random value to the time so that the expiration time is scattered.

How does 18.Redis persist?

Bgsave persists the image in full, and aof does incremental persistence. Because bgsave will take a long time, not real-time, and lead to a lot of data loss during downtime, aof is needed to cooperate with it. When the redis instance is restarted, the memory is rebuilt using the bgsave persistence file, and then the recent operation instructions are replayed by aof to fully restore the state before the restart.

The other party asked what would happen if the machine suddenly lost power. Depending on the configuration of the aof log sync property, if performance is not required, sync the disk as each instruction is written, and no data will be lost. However, it is not realistic to sync every time under the requirement of high performance. Timing sync is generally used, such as 1s1 times. At this time, a maximum of 1 second of data will be lost.

The other party asked what is the principle of bgsave? Just give me two words, fork and cow. Fork means that redis performs bgsave operations by creating child processes, and cow refers to copy on write. After the child process is created, the parent process shares the data segment, and the parent process continues to provide read and write services. The dirty page data will be gradually separated from the child process.

What are the benefits of 19.Pipeline, and why use pipeline?

The time for multiple IO round trips can be reduced to one, as long as there is no causal correlation between the instructions executed by pipeline. When using redis-benchmark for pressure testing, it can be found that one of the important factors affecting the QPS peak of redis is the number of pipeline batch instructions.

Do you know the synchronization mechanism of 20.Redis?

Redis can use master-slave synchronization and slave synchronization. In the first synchronization, the master node does a bgsave, and records the subsequent modification operations to the memory buffer. After the completion, the rdb files are synchronized to the replication node. After the replication node accepts, the rdb image is loaded into memory. After the loading is completed, the synchronization process is completed by informing the master node to synchronize the operation records modified during the period to the replication node for playback.

21. Have you ever used Redis clustering? what is the principle of clustering?

Redis Sentinal focuses on high availability and automatically promotes slave to master in case of master downtime and continues to provide services.

Redis Cluster focuses on scalability and uses Cluster for sharding storage when a single redis runs out of memory.

After reading this, the article "what are the most frequently asked questions about Redis" has been introduced. If you want to master the knowledge points of this article, you still need to practice and use it yourself. If you want to know more about related articles, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.