Shulou(Shulou.com)05/31 Report--

Editor to share with you what are the data types and characteristics of Redis, I believe most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to know it!

Redis provides us with five data types. Basically, the frequency we use is String, while the other four data types are used slightly less frequently than String. The reason is:

String is easy to use, can easily store complex objects, and uses a lot of scenes.

Because Redis expiretime can only be set on key, such as List, Hash, Set, Zset belong to the collection type and manage a set of item, we cannot set the expiration time on the item of these collections, so using expiretime to deal with the cache failure of the collection becomes a little more complicated. However, it is easier for String to use expire time to manage expiration policies because it contains fewer items. The set we are talking about here is a broad similar set.

At a deeper level, we don't know much about the use and principles of the other four data types. So at this time, we tend to ignore the possibility that using a certain data type in a particular scenario will be much higher than String performance, such as using Hash structure to improve the modification of an item in an entity.

Here we do not intend to list the use of these five data types, because there are many of these data on the Internet. We mainly discuss the functional features of these five data types, find out which real business scenarios they are suitable for dealing with, and how to combine these five data types to find solutions to complex cache problems.

I. data types and characteristics of Redis

Let's take a brief look at String, List, Hash, Set, and Zset:

1) String

String is the string type provided by Redis. Expire time can be set independently for the String type, which is usually used to store long string data, such as an object's json string.

In use, the most ingenious thing about the String type is that it can splice key dynamically. Usually, we can put a set of id in Set, and then dynamically check whether the String still exists. If it does not exist, it means that it has expired or the active delete is due to data modification, we need to do the cache data load again.

Although Set cannot set the expiration time of item, we can associate Set Item with String Key to achieve the same effect.

On the left in the following figure is a Set collection whose key is Set:order:ids, which may be a full collection or a collection obtained by a query condition:

Sometimes complex scenarios require multiple Set collections to support computing, and there may be many such collections in the Redis server. These sets can be called functional data, which are used to assist cache calculation. After performing various set operations, we will get the subset that the current query needs to return, and then we will get the real data of an order.

The String:order: {orderId} string key is not necessarily to serve a scenario, but the data of the * * layer of the entire system, which needs to be obtained in various scenarios. Those Set collections can be thought of as query condition data, which are used to assist in the calculation of query conditions.

Redis provides us with the TYPE command to view the data type of a certain key, such as the String type:

SET string:order:100 order-100 TYPE string:order:100 string

2) List

List is very suitable for improving throughput scenarios, because its unique LPUSH, RPUSH, LPOP, RPOP functions can seamlessly support producer and consumer architecture patterns.

This is ideal for implementing work-stealing algorithms similar to those in the Java Concurrency Fork/Join framework (work theft).

Note: Java Fork/Join framework uses parallelism to improve performance, but it will bring race condition (race condition) problems caused by concurrent take task, so work-stealing algorithm is used to solve the performance loss caused by competition.

A typical payment callback peak scenario is simulated in the following figure:

Where the peak occurs, we usually use the method of adding buffer to speed up the request processing speed, so as to improve the concurrent processing capacity and through put.

After the payment gateway receives the callback, it will be handed over to the distributor without any processing.

The dispenser is a stateless cluster, and each node gets the message channel that the downstream processor registers with the registry by pull handler queue list. Each dispatcher node maintains a local queue list and then pushes messages sequentially to these queue list.

There will be a little problem here, that is, how to do load balance when paying gateway to call the dispenser? If it is not the average load, one queue list may be higher than the other queue list.

The dispenser does not need to do soft load balance, because it does not matter if one queue list is more than other queue list, because the downstream message handler will steal other slow-consuming queue list according to the work-stealing algorithm.

Redis List's LPUSH, RPUSH, LPOP, and RPOP features can indeed improve this scale-out computing power in many scenarios.

3) Hash

The Hash data type is obviously based on the Hash algorithm, and the time complexity of searching items is O (1). In extreme cases, the problem of item Hash conflict may occur. Redis is solved by linked list plus key judgment. The specific data structure within Redis will be introduced later, so we won't expand it here.

The characteristics of Hash data types can usually be used to solve the mapping relationship, while some items need to be updated or deleted. If it is not an item that needs to be maintained, it can generally be solved by using String.

If there is a need to modify a field, there is obviously a lot of overhead to use String, which needs to be read out and deserialized into an object, then operated, and then serialized and written back to Redis, which may have concurrency problems.

Then we can use the entity attribute Hash storage feature provided by Redis Hash, we can think of Hash Value as a Hash Table, and each attribute of the entity gets the final data index of the attribute through Hash.

The following figure uses the Hash data type to record the a/bmetrics of the page:

On the left is the statistics of each area of the home page index, and on the right is the statistics of each region of the marketing marketing.

In the program, we can easily use the atomic feature of Redis to accumulate an item in Hash.

HMSET hash:mall:page:ab:metrics:index topbanner 10 leftbanner 5 rightbanner 8 bottombanner 20 productmore 10 topshopping 8 OK HGETALL hash:mall:page:ab:metrics:index 1) "topbanner" 2) "10" 3) "leftbanner" 4) "5" 5) "rightbanner" 6) "8" 7) "bottombanner" 8) "20" 9) "productmore" 10) "10" 11) "topshopping" 12) "8) "HINCRBY hash:mall:page:ab:metrics:index topbanner 1 (integer) 11

Use Redis Hash Increment for atomic increment operations. The HINCRBY command can increment any given integer, or you can increment floating-point type data through HINCRBYFLOAT.

4) Set

The Set collection data type can support collection operations and cannot store duplicate data.

The characteristics of Set*** are the computing power of sets, inter intersection, union union, diff difference sets, these characteristics can be used to do high-performance cross computing or eliminate data.

Set collections are relatively numerous and free to use in scenarios. To take a simple example, commodity and activity scenarios are more common in application systems. A valid collection of items is cached with a Set, and a collection of active items is cached with a Set. If the goods appear on and off the shelves, you only need to maintain the effective goods Set, each time you get the active goods, you need to filter whether there are products off the shelves, and if so, you need to remove them from the active goods.

Of course, you can delete the cached active items directly when you take them off the shelves, but the activity is load from the marketing system. Even if I delete the active items in cache, there will still be items off the shelves the next time I load them from the marketing system.

Of course, this is just an example, a scenario has different implementation methods.

In the following figure, there are two different sets on the left and right sides:

On the left is the ids collection of available goods in the marketing domain, and on the right is the ids collection of active goods in the marketing domain, and the intersection of the two sets is calculated.

SADD set:marketing:product:available:ids 1000100 1000120 1000130 1000140 1000150 1000160 SMEMBERS set:marketing:product:available:ids 1) "1000100" 2) "1000120" 3) "1000130" 4) "1000140" 5) "1000150" 6) "1000160" SADD set:marketing:activity:product:ids 1000120 1000120 1000130 1000140 1000300 SMEMBERS set:marketing:activity:product:ids 1) "1000100" 2) "1000120" 3) "1000130" 4) "1000140" 5) "1000200" 6) "1000300" SINTER set:marketing:product:available:ids set:marketing:activity:product:ids 1) "1000100" 2) "1000120" 3) "1000130" 4) "1000140"

In some complex scenarios, you can also use the SINTERSTORE command to store the result of the intersection calculation in a target set. This is particularly useful in pipelines using pipeline commands, where wrapping SINTERSTORE commands in a pipeline command string allows you to reuse the calculated result set.

Because Redis is a Signle-Thread single-threaded model, based on this feature, we can use the pipeline pipeline provided by Redis to submit a series of logical commands that will not be disturbed by commands from other clients during processing.

5) Zset

Zset sort collections are similar to Set collections, but Zset provides sorting capabilities. When we introduce the Set collection, we know that the members of the Set collection are unordered, and Zset fills the gap that the collection can be sorted.

The great function of Zset*** is that it can be sorted according to a certain score ratio score, which is urgently needed in many business scenarios. For example, in promotional activities, goods are sorted according to the number of goods sold, and popular scenic spots are sorted according to the number of inflows in tourist attractions. Basically everything people do needs to be sorted according to certain conditions.

In fact, Zset can be used everywhere in our application system. Here we give a simple example. In the group purchase system, we usually need to sort the group list according to the number of participants, and everyone wants to join the group that is about to form a group.

The following figure shows a Zset,score score created based on the group purchase code, which is the sum of the number of participants:

ZADD zset:marketing:groupon:group:codes 5 G_PXYJY9QQFA 8 G_4EXMT6NZJQ 20 G_W7BMF5QC2P 10 G_429DHBTGZX 8 G_KHZGH9U4PP ZREVRANGEBYSCORE zset:marketing:groupon:group:codes 1000 0 1) "G_W7BMF5QC2P" 2) "G_ZMZ69HJUCB" 3) "G_429DHBTGZX" 4) "G_KHZGH9U4PP" 5) "G_4EXMT6NZJQ" 6) "G_PXYJY9QQFA" ZREVRANGEBYSCORE zset:marketing:groupon:group:codes 1000 0 withscores 1) "G_W7BMF5QC2P" 2) "20" 3) "G_ZMZ69HJUCB" 4) "10" 5) "G_429DHBTGZX" 6) "10" 7) "G_KHZGH9U4PP" 8) "8" 9) "G_4EXMT6NZJQ" 10) "8" 11) "G_PXYJY9QQFA" 12) "5"

Zset itself provides many ways to sort collections, and if you need a score score, you can use the withscore sentence to bring out the score for each item.

In some special situations, combinatorial sorting may be needed, and multiple Zset may be used to sort the same entity in different dimensions, by time, by number of people, and so on. At this time, you can combine the convenience brought by Zset, and use pipeline to combine multiple Zset to get a combinatorial sort set.

Second, the case: Hujiang group purchase system greatly promotes the design of hot-top interface cache.

Taking the cache design of the hot-top interface of Hujiang Group purchase system as an example, we summarize the respective characteristics and general usage scenarios of the five data types provided by Redis. But not only can we use these data types separately, we can use these data types together to complete complex cache scenarios.

Let's share an example of using multiple Zset and String to optimize the foreground interface of a group purchase system. Due to space and time constraints, only the information related to this case is introduced here.

Note: hot-top interface refers to hot spots and ranking interfaces, indicating that it has relatively high pageviews and concurrency. Generally speaking, when it is greatly promoted, there will be several APIs with high performance requirements.

Let's first analyze the general information contained in a query interface.

First of all, a query interface must have query condition query conditions, followed by sort sorting information and * * page paging information. This is the basic responsibility of general APIs. Of course, in special scenarios, you also need to support the data session consistency requirements of master/slave replication, and you need to provide trace tags to query data back and forth, so we will not expand here.

We can abstract the information from these dimensions:

Querycondition: query criteria, such as companyid = 100 self _ idling 1010101, and so on.

Sort: sort information, usually a column sort by default, but in complex scenarios, it is possible to allow interface users to customize sorting fields, such as some tenant information columns.

Page: paging information, simply understood as the number of lines to which the data records are sorted.

Since we only use Redis to improve cache capabilities here, it doesn't involve any search capabilities, so we ignore other complex queries here. In fact, we use Elastcsearch in complex places to improve our search ability.

Above we analyze and summarize the basic information of a query interface, and there is also a design principle about high concurrency interface, which is to separate the hot-top interface from the general search interface, because only divide and conquer can choose different technologies according to their characteristics.

If we encapsulate all query scenarios in one interface regardless of responsibilities, it will be very troublesome to optimize the performance of the interface later. Some scenarios cannot or are difficult to be solved with cache, because various scenario logic is coupled in the interface, and the performance will not be high even if it can be implemented barely.

The purpose of laying the groundwork is to reach a basic consensus when introducing the case. Now let's look at the specific logic of the hot-top interface of this group buying system.

Note: during the promotion, the group purchase list needs to be displayed. The number of visits to this API is very large. Group purchase activities need to be sorted in reverse order according to the number of participants, and the specified number of group list needs to be returned by page. Let's assume that the interface is called getTopGroups (getTopGroupsRequestrequest).

1) query condition query condition problem

Let's carefully analyze, first of all, different query conditions from the DB query data is not the same, that is to say, the query group list is not the same, there may be company companies, channel channels and other filtering conditions.

Since there will not be too many groups under a group purchase activity, hundreds at most is the limit, so there will be no more than dozens of group lists for a query condition, and no more than 10 hot query conditions will be analyzed according to the scenario. Therefore, we choose to Hash the query condition into a code to cache the full group list collection of this query condition, but these result sets do not have any sort.

2) sort scheduling problem

If we look at the ranking problem according to the number of participants, we can immediately think of using Zset to deal with the group sorting problem, because there is only one sorting dimension, so one Zset is enough. We use a Zset to cache the number of participants of all groups, which is a full group sort set.

So how do we sort the list of groups from users' query conditions according to the number of participants? It is just possible to use the intersection operation of Zset to directly calculate the Zset subset of the current set.

3) page paging problem

Get the paging collection by using Zrange on the sorted group list Zset. Let's take a look at the complete process, how to deal with query, sorting, paging.

The following figure calculates Hash Code from query condition, and then queries the list of full groups of current conditions through DB:

Zset:marketing:groupon:hottop:available:groupkey represents the number of participants in the full group, which is cached by a Zset. Then the two Zset calculations are intersected, and it can be concluded that the Zset,*** with the number of participants needed for the current query is using Zrevrange to get the paging range.

ZADD zset:marketing:groupon:hottop:condition:2986080 0 G4ZD5732YZQ 0 G5VW3YF42UC 0 GF773FEJ7CC 0 GFW8DUEND8S 0 GKPKKW8XEY9 0 GL324DGWMZM (integer) 6 ZADD zset:marketing:groupon:hottop:available:group 5 GN7KQH36ZWK 10 GS7VB22AWD4 15 GF773FEJ7CC 17 G5VW3YF42UC 18 G4ZD5732YZQ 32 GTYJKCEJBRR 40 GKPKKW8XEY9 45 GL324DGWMZM 50 GFW8DUEND8S 60 GYTKY4ACWLT (integer) 10 ZINTERSTORE zset:marketing:groupon:hottop:condition:interstore 2 zset:marketing:groupon:hottop:condition:2986080 zset:marketing:groupon:hottop:available:group (integer) 6 ZRANGE zset:marketing:groupon:hottop:condition:interstore 0- 1 withscores 1) "GF773FEJ7CC" 2) "15" 3) "G5VW3YF42UC" 4) "17" 5) "G4ZD5732YZQ" 6) "18" 7) "GKPKKW8XEY9" 8) "40" 9) "GL324DGWMZM" 10) "45" 11) "GFW8DUEND8S" 12) "50" ZREVRANGE zset:marketing:groupon:hottop:condition:interstore 24 withscores 1) "GKPKKW8XEY9" 2) "40" 3) "G4ZD5732YZQ "4)" 18 "5)" G5VW3YF42UC "6)" 17 "

Once you have the returned regiment code collection, you can obtain the group details of String type in batches through mget, and the code will not be posted here.

The above is all the contents of the article "what are the data types and characteristics of Redis". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.