Shulou(Shulou.com)06/01 Report--

Preface

To dig has been more than two years has been a small transparent, today finally posted an article.

Recently, I have been watching Redis and feel that I have gained a lot. Write a blog and record it.

Redis has five basic data structures: string,list,set,zset,hash. Among them, string is the simplest and most commonly used. Although this data type is simple, the internal structure design is very exquisite.

Basic introduction

Compared with Java, string can be modified in Redis, which is a dynamic string (Simple Dynamic String referred to as SDS). Its internal structure is more like an ArrayList, maintaining a byte array and pre-allocating redundant space to reduce frequent memory allocation. When the length of the string is less than 1MB, each expansion will double the existing space. If the length of the string exceeds 1MB, only the space of 1MB will be expanded.

Ps: the string length is the maximum length 512MB.

> set name testOK > get name "test" > mset name1 test1 name2 test2OK > mget name1 name21) "test1" 2) "test2" > del name (integer) 1

Above are the basic string operation commands mset and mget that can save network overhead for reading and writing multiple strings.

Not only that, redis strings can also be used to store integers (let alone Java strings) and can be self-incremented. Strings that hold integer types range from to

If the saved number is larger than this value range, it will become a normal character type can not be self-incremented operation. This will be determined by the string encoding format.

A string consists of multiple bytes, each with a 8bit. Such data structures can also be used as bitmap.

> set foo 1OK > get foo "1" > incr foo (integer) 2 > get foo "2"

Internal principle

Basic realization

The above figure shows the basic structure of the string, in which content stores the contents of the string and uses 0x\ 0 as the ending character like c. This closing character will not be counted in the len. The code is as follows:

Struct SDS {T capacity; / / Array capacity T len; / / actual length byte flages; / / flag bits, the lower three bits represent the type byte [] content; / / array content}

You can see that both capacity and len are generic, so why not just use int? Because there are many optimizations within Redis, strings of different lengths are represented by different data types in order to reduce the use of memory. And len will be as big as capacity when creating strings, and there is no redundant space, because there are very few scenarios for modifying strings. (Redis really optimizes memory to the extreme)

Coding format

There are several Redis string encoding formats: int encoding, embstr encoding and raw encoding. The differences between these encodings are described in detail below.

Before we do that, let's talk about RedisObject. The object header of Redis, all Redis objects have the following header structure.

Struct RedisObject {int4 type; / / data types 5 int4 encoding; / / key internal encoding formats such as int or embstr, etc. Int24 lru; / / when memory exceeds the limit, use LRU algorithm to clear the number of objects in memory int32 refcount; / / change key value referenced void * ptr; / / object content}

Int coding

Int encoding will be used when the stored value is a 64-bit signed integer type, so you can use the key value self-increment operation. Redis will create 1w redisObject shared objects at startup, as described in the following article, the value is between [0prime1000). If the value stored in the integer is in [0Power1000), Redis will not create a new object, but will point directly to the shared object, and the key value will not take up extra space.

Use the object encoding command to view the encoding format and use the debug object command to view more information

> set foo 1OK > object encoding foo "int" > set foo2 1OK > debug object fooValue at:0x7f44b020aca0 refcount:2147483647 encoding:int serializedlength:2 lru:14691591 lru_seconds_idle:72588 > debug object foo2Value at:0x7f44b020aca0 refcount:2147483647 encoding:int serializedlength:2 lru:14691591 lru_seconds_idle:72594

You can see that both foo and foo2 point to the same object here in 0x7f44b020aca0.

Embstr coding

When the length of the stored string is short (len44 bytes), Redis will be encoded in raw. The biggest difference from embstr is that RedisObject and SDS are no longer together, and memory addresses are no longer contiguous.

As shown in the figure:

Thinking

Why do strings have two formats: embstr and format and the dividing line between raw and 44 bytes?

Redis default memory allocator jemalloc allocates memory size in power. In order to accommodate a complete embstr object, it will allocate at least 32 bytes of space, and then 64 bytes longer. After that, it is considered that this is a large string that is not suitable for embstr storage. Instead, it uses raw encoding.

So the question is, what is the length of a 64-byte space string? The answer is 44 bytes.

The length of the content in the following figure is 45 bytes minus the ending 0x\ 0, leaving only 44 bytes.

Summary

The above is the whole content of this article. I hope the content of this article has a certain reference and learning value for everyone's study or work. Thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.