Shulou(Shulou.com)06/01 Report--

This article will share with you about the new features supported by Redis5.0. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

New features supported by Redis5.0 explain the article from Huawei Cloud help Center

The Redis5.x version of DCS inherits all the enhancements and new commands of version 4.x, while also compatible with the new features of the open source Redis5.x version.

Stream data structure

Stream is a new data type introduced by Redis 5.0. it is a new persistent message queue that supports multicast.

Click the link below for more details.

Https://www.huaweicloud.com/product/dcs.html

The structure diagram of Redis Stream, shown in figure 7-1, is a persistent data structure that uses a message linked list to string all added messages together.

Stream data structures have the following characteristics:

There can be multiple consumer groups in Stream.

Each consumer group contains a Last_delivered_id that points to the last element (message) currently consumed by the consumer group.

Each consumption group can contain multiple consumer objects, consumers share the Last_delivered_id in the consumption group, and consumers in the same consumption group compete, that is, an element can only be consumed by one of the consumers.

A Pending_ids,Pending_ids record is also maintained within the consumer object that has been sent to the client, but the element id of ACK (consumption confirmation) has not yet been completed.

A comparison between Stream and other Redis data structures is shown in Table 7-4.

Figure 1-1 schematic diagram of Stream data structure

Table 1-1 comparison of existing data structures between Stream and Redis

Stream

List, Pub/Sub, Zset

It is efficient to get elements with O (logN) complexity.

The complexity of getting elements by List is O (N).

Offset is supported, and each message element has a unique id. The id will not be changed because new elements are added or other elements are eliminated.

List does not have the concept of offset. If an element is expelled, the latest element cannot be determined.

Supports persistence of message elements, which can be saved to AOF and RDB.

Pub/Sub does not support persistent messages.

Support consumption grouping

Pub/Sub does not support consumption grouping

Support for ACK (consumption confirmation)

Pub/Sub does not support

There is no obvious relationship between Stream performance and the number of consumers.

Pub/Sub performance is positively related to the number of clients

Allows historical data to be expelled by timeline, supports block, gives radix tree and listpack, and has less memory overhead.

Zset cannot add the same elements repeatedly, does not support eviction and block, and has a high memory overhead.

Message elements cannot be expelled from the middle.

Zet supports deletion of any element

Introduction to Stream related commands

Next, the Stream-related commands are introduced in the order in which they appear in the usage process. Detailed orders are shown in Table 7-5.

First, you use XADD to add flow elements, that is, to create a Stream, and you can specify the maximum number of messages to save when adding flow elements.

Then create a consumer group through XGROUP.

Consumers use XREADGROUP instructions for consumption.

After the client consumes, use the XACK command to confirm that the message has been consumed successfully.

Figure 1-2 introduction of Stream related commands

Table 1-2 detailed commands for Stream

Command

Description

Grammar

XACK

Removes one or more messages from the list of pending items in the consumer group of the stream (PEL for short).

XACK key group ID [ID...]

XADD

Appends the specified stream entry to the stream of the specified key. If key does not exist, as a side effect of running this command, the key is automatically created using entries from the stream.

XADD key ID field string [field string...]

XCLATM

In the context of the consumer group of the stream, this command changes the ownership of the pending message, so the new owner is the consumer specified in the command parameters.

XCLAIM key group consumer min-idle-time ID [ID...] [IDLE ms] [TIME ms-unix-time] [RETRYCOUNT count] [FORCE] [JUSTID]

XDEL

Removes the specified entry from the specified stream and returns the number of entries that were successfully deleted, which may be different from the number of ID passed if the passed ID does not exist.

XDEL key ID [ID...]

XGROUP

This command is used to manage the consumer groups associated with the stream data structure. With XGROUP, you can:

Create a new consumer group associated with the flow.

L destroy a consumer group.

L removes the specified consumer from the consumer group.

Set the final delivery ID of the consumer group to something else.

XGROUP [CREATE key groupname id-or-$] [SETID key id-or-$] [DESTROY key groupname] [DELCONSUMER key groupname consumername]

XINFO

Retrieve different information about the stream and the associated consumer group.

XINFO [CONSUMERS key groupname] key key [HELP]

XLEN

The number of entries in the return stream. If the specified key does not exist, this command returns 0 as if the stream were empty.

XLEN key

XPENDING

Get data from the stream through consumer groups. Check the interface of the list of messages to be processed to observe and understand which clients in the consumer group are active and which messages are waiting for consumption, or to see if there are any idle messages.

XPENDING key group [start end count] [consumer]

XRANGE

The entry in the return stream that meets the given ID range.

XRANGE key start end [COUNT count]

XREAD

Reads data from one or more streams and returns only entries whose ID is greater than the last received ID reported by the caller.

XREAD [COUNT count] [BLOCK milliseconds] STREAMS key [key...] ID [ID...]

XREADGROUP

A special version of the XREAD command that specifies the consumer group to read.

XREADGROUP GROUP group consumer [COUNT count] [BLOCK milliseconds] STREAMS key [key...] ID [ID...]

XREVRANGE

Same as XRANGE, but significantly different in that entries are returned in reverse order and start-end parameters are obtained in reverse order

XREVRANGE key end start [COUNT count]

XTRIM

XTRIM clips the stream to a specified number of items and, if necessary, expels old items (items with smaller ID).

XTRIM key MAXLEN [~] count

Message (flow element) consumption confirmation

Compared with Pub/Sub, Stream not only increases the consumption grouping mode, but also supports message consumption confirmation.

When a message is taken over by a consumer calling a XREADGROUP command or a XCLAIM command, the server is not sure whether it has been processed at least once. Therefore, once the consumer has successfully processed a message, it should call XACK to notify Stream so that the message will not be processed again, and the PEL (pending_ids) entry on the message will be cleared, freeing memory from the Redis server.

In some cases, due to network problems, the client does not call XACK after consumption, and the corresponding element ID will be retained in the PEL. After the client is reconnected, the start message ID of XREADGROUP is recommended to be set to 0-0, which means that all PEL messages and messages since last_id are read. At the same time, consumers need to be able to support repeated message delivery when consuming messages.

Figure 1-3 interpretation of ACK mechanism

Memory usage optimization

Redis5.x has further optimized its memory usage on the basis of the previous version.

L active defragmentation

When the key is frequently modified and the length of the value is constantly changing, Redis allocates new memory space for the key. Because Redis pursues high performance and implements its own memory allocator to manage memory, it does not release the original memory to OS, resulting in memory fragmentation. When the used_memory_rss/used_memory is higher than 1.5, it is generally considered that the proportion of memory fragmentation is too high and the memory utilization is low.

Therefore, reasonable planning and use of cached data and standardizing data writing are helpful to reduce the generation of memory fragmentation.

Redis3.x and below: memory fragmentation can be resolved by periodically restarting the service. It is recommended that the actual cached data do not exceed 50% of the configured available memory.

Redis4.x: active memory defragmentation is supported, and the service performs automatic memory defragmentation during operation. At the same time, Redis4.x supports manual cleanup of memory fragments through the memory purge command.

Redis5.0: enhanced active defragmentation, with Jemalloc version update, faster and smarter, lower latency.

L HyperLogLog algorithm optimization

HyperLogLog is a cardinality counting method, which uses a small amount of memory space to complete the counting statistics of massive data. In Redis5.0, the HyperLogLog algorithm is improved to optimize the memory efficiency of counting statistics.

For example: B-tree counting efficiency is very high, but memory consumption is also relatively high. HyperLogLog can save a lot of storage space. When B-tree needs 1m memory statistics, HyperLogLog only needs 1kb.

L enhanced ability to report memory information statistics

The INFO command returns more detailed information.

Command addition and optimization

Client Management Enhancement

− Redis-cli supports cluster management

In Redis4.x and previous versions, you need to install the redis-trib module to manage the cluster.

Redis5.0 optimizes Redis-cli and integrates all the management functions of the cluster. You can view the help information by using the command redis-cli-- cluster help.

− optimizes client performance in frequent connection and interruption scenarios

This optimization value is highlighted when your application needs to use short connections.

Ordered collections are easier to use

Add two new commands to the ordered collection: ZPOPMIN and ZPOPMAX.

− ZPOPMIN key [count]

Deletes and returns the maximum number of count members with the lowest score in the ordered collection key. If more than one member is returned, it will also be ranked from lowest to highest according to the score (value value comparison).

− ZPOPMAX key [count]

Deletes and returns the maximum number of count members with the highest score in the ordered collection key. If more than one member is returned, it will also be ranked from highest to lowest according to the score (value value comparison).

Help adds more subcommand instructions

Support help to view the Quick usage Guide directly, you no longer need to log in to redis.io every time to find it. For example, enter the stream playbook on the command line: xinfo help

one

two

three

four

five

six

seven

127.0.0.1 purl 6379 > xinfo help

1) XINFO arg arg... Arg. Subcommands are:

2) CONSUMERS-Show consumer groups of group.

3) GROUPS-Show the stream consumer groups.

4) STREAM-Show information about the stream.

5) HELP-Print this help.

127.0.0.1purl 6379 >

Redis-cli command input prompt

After entering the complete command, Redis-cli will display a parameter reminder to help users remember the syntax format of the command.

As shown in the following figure, enter the zadd command, and Redis-cli uses a light color font to display the syntax of zadd.

RDB supports storage of LFU and LRU

Since Redis5.0, storage key eviction policies LRU and LFU have been added to the RDB snapshot file:

L FIFO: first in, first out. The earliest data stored will be eliminated first.

L LRU: least used recently. Data that has not been used for a long time will be eliminated first.

L LFU: used least often. Over a period of time, the data that is least used will be eliminated first.

The RDB file format of Redis5.0 has changed and is backward compatible. Therefore, if you migrate using snapshots, you can migrate from a lower version of Redis to Redis5.0, but not from Redis5.0 to a lower version.

Thank you for reading! This is the end of this article on "what are the new features supported by Redis5.0?". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, you can share it for more people to see!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.