Shulou(Shulou.com)05/31 Report--

This article introduces the relevant knowledge of "Redis memory object model analysis". Many people will encounter such a dilemma in the operation of actual cases, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Redis memory statistics

Statement: info memory

Output description:

Used_memory: Total number of bytes allocated by Redis using its allocator

(the total amount of memory allocated by the Redis allocator in bytes, including virtual memory used)

Used_memory_human: Human readable representation of previous value

(just for friendly display of used_memory, easy to read)

Used_memory_rss: Number of bytes that Redis allocated as seen by the operating system (a.k.a resident set size). This is the number reported by tools such as top (1) and ps (1)

(the Redis process occupies the operating system's memory in bytes, which is consistent with the memory seen by the top and ps commands; except for allocation

In addition to the memory allocated by the device, used_memory_rss also includes the memory needed by the process to run itself, memory fragments, etc., but does not include virtual memory)

Therefore, used_memory and used_memory_rss, the former is the quantity seen from the redis perspective, and the latter is the quantity seen from the operating system. Both

The reason for the difference is that, on the one hand, memory fragments and Redis processes need to occupy memory, so the former may be smaller than the latter, on the other hand, the existence of virtual memory makes the former larger than the latter.

In practical applications, the amount of data of Redis will be relatively large, and the memory consumed by the process will be much smaller than the amount of Redis data and memory fragments. Therefore, the ratio of used_memory_rss to used_memory becomes a parameter to measure the fragmentation rate of Redis memory; this parameter is mem_fragmentation_ratio.

Mem_fragmentation_ratio: Ratio between used_memory_rss and used_memory

(memory fragmentation ratio, which is the ratio of used_memory_rss to used_memory)

Mem_fragmentation_ratio is generally greater than 1, and the higher the value, the greater the percentage of memory fragments. Mem_fragmentation_ratio1), called a shared object. Redis in order to save memory, when some objects are repeated, the new program

Instead of creating a new object, the original object is still used. The object that is reused is the shared object. Currently, shared objects only support integer-valued

String object.

Redis's shared objects currently only support string objects with integer values. The reason for this is actually a balance between memory and CPU (time): sharing objects

Although it reduces memory consumption, it takes extra time to determine whether two objects are equal. For integer values, the operation complexity is judged to be O (1); for

For ordinary strings, the judgment complexity is O (n), while for hashes, lists, sets and ordered sets, the judgment complexity is O (n ^ 2).

Although shared objects can only be string objects with integer values, all five types can use shared objects (elements such as hashes, lists, and so on).

In the current implementation, when the Redis server initializes, it creates 10000 string objects, each with an integer value of 09999; when Redis requires

When you want to use string objects with a value of 0,9999, you can use these shared objects directly. The number 10000 can be adjusted by adjusting the parameter.

Change the value of REDIS_SHARED_INTEGERS (OBJ_SHARED_INTEGERS in 4.0).

The number of references to a shared object can be viewed through the object refcount command, as shown in the following figure. The results page of the execution of the command confirms that only between 09999

Integers are used as shared objects.

(3.5) ptr

The ptr pointer points to specific data, as in the previous example, set hello world,ptr points to the SDS that contains the string world.

(3.6) Summary

To sum up, the structure of redisObject is related to object type, encoding, memory collection, and shared objects; the size of a redisObject object is 16.

Bytes:

4bit+4bit+24bit+4Byte+8Byte=16Byte .

4,SDS

(1) SDS structure

Instead of directly using the C string (that is, an array of characters ending with the empty character'\ 0') as the default string representation, Redis uses SDS. SDS is a simple dynamic string

An abbreviation for (SimpleDynamic String).

From the structure of the SDS, you can see that the length of the buf array = free+len+1 (where 1 represents the empty character at the end of the string); therefore, the space occupied by a SDS structure is:

Length occupied by free + length occupied by len + length of buf array = 4+4+free+len+1=free+len+9.

(2) comparison between SDS and C string

SDS adds free and len fields to the C string, which brings a lot of benefits:

Get string length: SDS is O (1), C string is O (n)

Buffer overflow: when using API of C string, if the string length increases (such as strcat operation) and forgets to reallocate memory, it is easy to cause buffer overflow; while SDS records the length, the corresponding API will automatically reallocate memory when it may cause buffer overflow to eliminate buffer overflow.

Memory reallocation when modifying strings: for C strings, if you want to modify strings, you must reallocate memory (release before applying), because if there is no reallocation, increasing string length will cause memory buffer overflow and decreasing string length will cause memory leak. For SDS, because len and free can be recorded, the association between string length and spatial array length is disassociated, which can be optimized on this basis: the space pre-allocation strategy (that is, allocating memory more than is actually needed) greatly reduces the probability of memory reallocation when the string length increases; the lazy space release strategy greatly reduces the probability of redistributing memory when the string length decreases hours.

Access to binary data: SDS can, C string can not. Because the C string ends with an empty character, and for some binary files (such as pictures, etc.), the content may include an empty string, so the C string cannot be accessed correctly, while SDS uses the string length len as the end of the string, so there is no problem.

In addition, because buf in SDS still uses the C string (that is, ending with'\ 0'), SDS can use some functions in the C string library; however, it is important to note that

Yes, this can only be used when SDS is used to store text data, but not when storing binary data ('\ 0' is not necessarily the end).

(3) the application of SDS and C string

Redis always uses SDS instead of C strings when storing objects. For example, the set hello world command, hello and world are stored in the form of SDS.

The sadd myset member1 member2 member3 command, whether it is a key ("myset") or an element in the collection ("member1", "member2")

And "member3"), are stored in the form of SDS. In addition to storing objects, SDS is also used to store various buffers.

The C string is used only if the string does not change, such as when printing a log.

Object types and Internal Encoding of Redis

As mentioned earlier, Redis supports five object types, and each structure has at least two encodings. The advantage of this is: on the one hand, the interface is separated from the implementation, and when you need to add or change the internal coding, the user's use will not be affected; on the other hand, you can switch the internal coding according to different application scenarios to improve efficiency.

The internal encodings supported by various object types of Redis are shown in the figure below (internal encoding is added in later versions of Redis3.0,www.baohuayule.cn Redis, not to mention; all of the internal encodings introduced in this chapter are based on 3.0):

The conversion of Redis internal coding is in accordance with the following rules: the coding conversion is completed when Redis writes data, and the conversion process is irreversible, so it can only be converted from small memory coding to large memory coding.

1. Overview of string (1)

Strings are the most basic type, because all keys are string types, and several complex types of elements other than strings are also strings.

The length of the string cannot exceed 512MB.

(2) Internal coding

There are three internal encodings of string types, and their application scenarios are as follows:

A long integer of int:8 bytes. When a string value is an integer, this value is represented by an long integer.

Embstr:www.006665.cn

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.