Shulou(Shulou.com)06/03 Report--

This article takes Huawei Cloud DCS for Redis version as an example to introduce the new features of Redis4.0. The article is reproduced from Huawei Cloud help Center.

Compared with the Redis3.x version, the Redis4.x version of DCS has the following major feature enhancements in addition to the features added by open source Redis.

L takes less time to create

The instance has been changed from virtual machine to physical machine capacitive deployment, and it takes only 8 seconds and 10 seconds to create the instance.

L performance improvement

High-performance version of Redis4.x, the operating system uses lightweight LibOS developed by Huawei, and single-node QPS is upgraded from 10w to 30w

L cost savings

For the high-performance version of Redis4.x, the instance specification takes 1G as the step size, and how many gigabytes are used to buy as many gigabytes. It can be flexibly customized and is no longer limited to the n-power requirement of 2.

Compared with the previous version, Huawei Cloud Redis4.x has made a significant improvement in performance and greatly improved the user experience. If you want to know more details about the actual content, click on this link.

Https://www.huaweicloud.com/product/dcs.html

More surprises for you.

Next, let's introduce the specific features of the new features of the open source Redis4.0 version.

The features of the Redis4.x version update mainly involve three aspects:

The addition of new commands, such as MEMORY, SWAPDB.

Lazyfree mechanism, delaying the deletion of large key, reducing the impact of deletion operations on the consumption of system resources.

Memory performance optimization, that is, active defragmentation.

MEMORY command

Before Redis3.x, only a limited number of memory statistics were known through the info memory command. Redis4.0 introduces a new command, memory, to give you a deeper understanding of Redis's memory usage.

127.0.0.1 memory help 6379 [8]

1) MEMORY arg arg... Arg. Subcommands are:

2) DOCTOR-Return memory problems reports.

3) MALLOC-STATS-Return internal statistics report from the memory allocator.

4) PURGE-Attempt to purge dirty pages for reclamation by the allocator.

5) STATS-Return information about the memory usage of the server.

6) USAGE [SAMPLES]-- Return memory in bytes used by and its value. Nested values are sampled up to

Usage

Enter memory usage [key] to return the value actual memory usage estimate of key if the current key exists, or nil if key does not exist.

127.0.1 in-memory cache service compatible with Redis 6379 [8] > set dcs "DCS is an online, distributed, in-memory cache service compatible with Redis, Memcached, and I

N-memory data grid (IMDG).

OK

127.0.0.1 memory usage dcs 6379 [8]

(integer) 171

127.0.0.1Plus 6379 [8] >

Usage counts the memory footprint of value and the memory footprint of key itself, excluding the Expire memory footprint of key.

127.0.0.1 set dcs.huaweiclou 6379 [8] > "Distributed Cache Service"

OK

127.0.0.1 memory usage dcs.huaweiclou 6379 [8]

(integer) 85

127.0.0.1 set dcs.huaweicloud 6379 [8] > "Distributed Cache Service"

OK

127.0.0.1 memory usage dcs.huaweicloud 6379 [8]

(integer) 86 / / memory footprint also changes after the length of the key name changes.

127.0.0.1 expire dcs.huaweicloud 6379 [8] >

(integer) 1

127.0.0.1 memory usage dcs.huaweicloud 6379 [8]

(integer) 86 / / memory footprint does not change after adding expiration time

127.0.0.1Plus 6379 [8] >

For data types such as hash, list, set, sorted set, and so on, the usage command samples statistics and provides estimates of memory usage.

Usage: memory usage keyset samples 1000

Where keyset represents the key,1000 of a collection data type represents the number of samples.

Stats

Returns the memory usage details of the current instance.

Usage: memory stats

127.0.0.1 memory stats 6379 [8]

1) "peak.allocated"

2) (integer) 2412408

3) "total.allocated"

4) (integer) 2084720

5) "startup.allocated"

6) (integer) 824928

7) "replication.backlog"

......

The following is the specific meaning of some of the data return items

Table 1-1 memory stats

Data return item

Description

Peak.allocated

The peak amount of memory allocated by allocator while the Redis instance is running. Used_memory_peak with info memory

Total.allocated

The number of bytes of memory currently allocated by allocator. Used_memory with info memory

Startup.allocated

Number of bytes of memory consumed by Redis startup

Replication.backlog

Redis replication backlog buffer (replication backlog) memory usage byte, set by repl-backlog-size parameter. Default is 1m.

Clients.slaves

Number of bytes of memory consumed by all slave clients on the master side

Clients.normal

All Redis regular clients consume memory bytes

Overhead.total

The additional total overhead memory bytes of the Redis; that is, the total memory allocated by the allocator, minus the memory used for the actual storage of the data.

Keys.count

The number of key in the Redis instance

Keys.bytes-per-key

The average number of bytes consumed per key. Note that the overhead is also evenly distributed to each key, so this value cannot be used to represent the actual average key length of the business.

Dataset.bytes

Represents the amount of memory consumed by Redis data. That is, the total amount of memory allocated, minus the total amount of additional memory overhead.

Dataset.percentage

Represents the percentage of memory occupied by Redis data as a percentage of total memory allocation

Peak.percentage

Ratio of current memory usage to peak time

Fragmentation

Indicates the memory fragmentation rate of the Redis

Doctor

Usage: memory doctor

Used_memory (total.allocated) less than 5m memory usage is considered to be too small to make further diagnosis. Redis gives diagnostic results and recommendations when one of the following points is met:

The memory allocated by peak is 1.5 times larger than that of current total_allocated, that is, peak.allocated/total.allocated > 1.5, which indicates that the fragmentation rate of memory is high, and RSS is much larger than used_memory.

High fragmentation/fragmentation greater than 1.4indicates high fragmentation rate of memory.

The average memory usage of each Normal Client is larger than that of 200KB, indicating that pipeline may be improperly used, or Pub/Sub clients may not process messages in a timely manner.

The average memory usage per Slave Client is larger than that of 10MB, indicating that the write traffic of master is too high.

Purge

Usage: memory purge

Purpose: to release memory by calling jemalloc internal commands. Freed objects include memory occupied by the Redis process but not used effectively, which is often referred to as memory fragmentation.

Memory purge applies only to Redis instances that use jemalloc as the allocator.

Lazy free mechanism

A solved pain point / problem

Redis is a single-threaded program, when running a time-consuming request, it will cause all requests to wait in queue. Before the request processing is completed, Redis can not respond to other requests, so it is easy to cause performance problems. When Redis deletes a large collection key, it is a more time-consuming request.

Principle

A lazy deletion or delayed release mechanism provided by Redis4.x is mainly used to solve the blocking of Redis processes caused by deleting large key, so as to avoid performance and availability problems.

When the key is deleted, the Redis delays the memory of the key asynchronously, and the key release operation is placed in a separate child thread processing of the bio (Background I Pot O).

Usage

Active deletion

− unlink

Unlink has the same purpose as the del command to delete a key. When unlink deletes a collection class key, if the number of elements of the collection key is greater than 64, it will release the memory operation to a separate bio (Background I _ Unip O) thread to perform. Therefore, the unlink delete operation can complete the deletion of a large key containing millions of elements in a very short time.

− flushall/flushdb

By adding the ASYNC asynchronous cleanup option to flushall/flushdb, Redis operates asynchronously when cleaning up the entire instance or a single DB.

Expired key deletion, large key eviction deletion

There are four scenarios for passive deletion, each of which corresponds to a configuration parameter, which is disabled by default:

Lazyfree-lazy-eviction no / / whether to adopt the lazyfree mechanism when the memory usage of redis reaches maxmeory and the elimination policy is set.

Lazyfree-lazy-expire no / / for keys with TTL set, whether to use lazyfree mechanism when being cleaned and deleted by redis after it expires

Lazyfree-lazy-server-del no / / for some instructions that deal with existing keys with an implicit Del key

Slave-lazy-flush no / / perform full data synchronization for slave. Slave will run flushall to clean up its data scene before loading the RDB file of master.

If you need to use the above configuration, please submit a ticket to consult the technical service personnel.

Other new commands

Swapdb

Purpose: exchange data of 2 db in the same Redis instance.

Usage: swapdb dbindex1 dbindex2

Zlexcount

Purpose: return the number of elements that meet the criteria in an ordered collection.

Usage: zlexcount key min max

Memory usage and performance improvement

Use less memory to store the same amount of data

The memory used can be defragmented and gradually recycled

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.