Shulou(Shulou.com)05/31 Report--

This article will explain in detail what the bitmap in redis means. Xiaobian thinks it is quite practical, so share it with you for reference. I hope you can gain something after reading this article.

bitmap

Bitmap, that is, a data structure composed of a large number of bits (each bit can only be 0 and 1), is mainly suitable for saving space in some scenes and recording data meaningfully.

For example, a large number of bool type accesses, a user's 365-day check-in record, check-in is 1, not check-in is 0, if you use ordinary key/value storage, when the number of users is large, the storage space required is very large.

If bitmap is used for storage, 365 days a year can be stored with 365 bits, and 365 bits are converted into 46 bytes (a slightly longer string), thus saving a lot of storage space.

The essence of a bitmap is actually an ordinary string, that is, a byte array. You can use get/set to directly obtain and set the contents of the entire bitmap, or you can use getbit/setbit to treat a byte array as a bit array. [Recommended: Redis Video Tutorial]

Set string using bit operations

Normal setting strings use the set command, below we use setbit to set the bit array, and finally get in the form of a string,

First of all, we get the h, e two ASCII codes using binary representation as follows,

We can see that the binary code of h is 01101000, and the binary code of e is 01100101. We only need to pay attention to the position where bit is 1, and then carry out setbit.

It should be noted that the order of the bit array and the bit order of the characters are reversed. According to this principle, we calculate that the position of each 1 of the h character is 1/2/4, and the position of the e character is 9/10/13/15.

So we're going to set a bit array using setbit and set the corresponding 1 at each position (1/2/4/9/10/13/15),

setbit data 1 1setbit data 2 1setbit data 4 1setbit data 9 1setbit data 10 1setbit data 13 1setbit data 15 1

zero deposit and withdrawal

Finally, directly get the data key, you will find that you just get the he.

The combination of setbit + get is called zero-storage and whole fetch. Zero storage is the setting of one bit and one bit. Whole fetch is to get all the data directly through the key name.

Similarly, we can also carry out zero storage and zero extraction, whole storage and zero extraction, whole storage is to directly use the string to set the entire bit array, zero extraction is to obtain bit by bit position.

zero out

We can see that according to setbit, we set the bit array of key called w, and only set the value of 1/2/4 of these three positions to 1. In the figure below, there is getbit w 3 to get the value of the third position, which is 0 by default. If triggered from the business point of view, it can be understood that there are 4 days of check-in, and no check-in on the third day.

zero out

As shown in the afternoon, we directly set an h character for this key of w, and then get each bit in the bit array of w through getbit. You can see that the obtained content is the same as the binary content of the h character above. The position of 1/2/4 is 1, and the rest is 0.

note

The bit array of redis is automatically extended. If a certain offset position is set beyond the existing content range, the bit array will be automatically zero-extended, that is, the expanded bits are all 0 by default.

If the byte corresponding to the bit is a non-printable character, redis-cli displays the hexadecimal version of the character.

A byte is 8 bits, and it is necessary to distinguish between bytes and bits.

Statistics and lookups (bitcount/bitpos)

redis provides the statistics instruction bitcount and bitmap lookup instruction bitpos,

Bitcount counts the number of 1s in a specified range, bitpos finds the first 0 or 1 in a specified range.

We can count how many days users have checked in through bitcount, and find out from which day users first checked in through bitpos instructions.

If you specify the range parameter [start, end], you can count how many days the user checked in within a certain time range and what day the user started checking in from a certain day.

Note, however, that the start and end arguments are byte indexes, meaning that the specified bit range must be a multiple of 8,

We cannot specify it arbitrarily, so we cannot directly calculate how many days users sign in in a certain month. If we need to calculate it,

You can use the getrange command to extract the byte content covered by the month, and then make statistics in memory, for example, February covers 10 - 12 bytes, use getrange w 8 12.

127.0.0.1:6379> set w hello OK127.0.0.1:6379> bitcount w #How many 1 's in all characters (integer) 21127.0.0.1: 6379> bitcount w 0 0 #Digits of 1 in the first character (integer) 3127.0.0.1: 6379> bitcount w 0 1 #Digits of 1 in the first two characters (integer) 7127.0.0.1: 6379> bitpos w 0 #first 0 digit (integer) 0127.0.0.1: 6379> bitpos w 1 #first digit (integer) 1127.0.0.1: 6379> bitpos w 1 1 1 #First digit from the second character (integer) 9127.0.0.1: 6379> bitpos w 1 2 2 #From the third character, the first 1 digit (integer) 17 bitfield

The setbit/getbit values described earlier are all single bits. If you want to operate on multiple bits at once, you must use pipes to process them.

After redis3.2, bitfield instruction is provided, which can operate on multiple bits at once. Bitfield has three sub-instructions, namely get/set/incrby, which can read and write specified bit fragments.

However, bitfield can only handle up to 64 consecutive bits. If it exceeds 64 bits, multiple subinstructions are required. Bitfield can execute multiple subinstructions at once.

example

Let's do something with bitfield on the bit array shown below

127.0.0.1:6379> bitfield w get u4 0 #Take 4 bits from the first bit and the result is unsigned (u) 1)(integer) 6127.0.0.1: 6379> bitfield w get u3 2 #Take 3 bits from the 3rd bit and the result is unsigned (u) 1)(integer) 5127.0.0.1: 6379> bitfield w get i4 0 #Take 4 bits from the first bit and the result is signed (i) 1)(integer) 6127.0.0.1: 6379> bitfield w get i3 2 #Take 3 bits from the 3rd bit and the result is signed number (i) 1)(integer)-3

signed number means that the first bit in the acquired bit array is the sign bit, and the rest are the values. If the first bit is 1, it is negative.

unsigned number means nonnegative number, no sign bit, all bit arrays obtained are values, signed number can obtain up to 64 bits,

unsigned number can only get 63 bits, because integer in redis protocol is signed number, maximum 64 bits, can not pass 64-bit unsigned value,

If the digit limit is exceeded, redis will tell you that the parameter is wrong.

All of these instructions can be combined into one instruction and you can see that the result is the same.

bitfield w get u4 0 get u3 2 get i4 0 get i3 2

set modification

We start with the 9th digit and replace the existing 8 digits with 8 unsigned digits, which is actually replacing the second character, from e to a (its ASCII code is 97), and you can see that the result also becomes hallo.

127.0.0.1:6379> bitfield w set u8 8 971) (integer) 101127.0.0.1:6379> get w"hallo"

incrby

incrby increments the specified range of bits, i.e.++, which may overflow, if positive, there will be an overflow, if negative, there will be an overflow,

Redis default processing is foldback, that is, if there is an overflow, the overflow sign bit is discarded, for example, if it is an 8-bit unsigned number 255, after adding 1, all become 0, if it is an 8-bit signed number 127, after adding 1, overflow becomes-128.

Again, based on the hello character, demonstrate incrby.

127.0.0.1:6379> set w helloOK127.0.0.1:6379> bitfield w get u4 2 #Take 4 unsigned integers starting from the 3rd digit, the first is 101)(integer) 10127.0.0.1: 6379> bitfield w incrby u4 2 11)(integer) 11127.0.0.1: 6379> bitfield w incrby u4 2 11)(integer) 12127.0.0.1: 6379> bitfield w incrby u4 2 11)(integer) 13127.0.0.1: 6379> bitfield w incrby u4 2 11)(integer) 14127.0.0.1: 6379> bitfield w incrby u4 2 11)(integer) 15127.0.0.1: 6379> bitfield w incrby u4 2 1 #When it gets here, it has already overflowed and turned back to 0 1)(integer) 0

Bitfield instruction provides overflow strategy subinstruction overflow, users can choose overflow behavior, default is wrap, can also choose fail (error not executed), and saturation truncation (sat)(beyond the range to stay at the maximum or minimum value,

The overflow instruction affects only the first instruction that follows, after which the overflow policy changes to the default value wrap.

saturation truncation

127.0.0.1: 6379> set w helloOK127.0.0.1: 6379> bitfield w overflow sat incrby u4 2 11)(integer) 11127.0.0.1: 6379> bitfield w overflow sat incrby u4 2 11)(integer) 12127.0.0.1: 6379> bitfield w overflow sat incrby u4 2 11)(integer) 13127.0.0.1: 6379> bitfield w overflow sat incrby u4 2 11)(integer) 14127.0.0.1: 6379> bitfield w overflow sat incrby u4 2 11)(integer) 15127.0.0.1: 6379> bitfield w overflow sat incrby u4 2 1 #Next all will be hold max 1)(integer) 15127.0.0.1: 6379> bitfield w overflow sat incrby u4 2 11)(integer) 15127.0.0.1: 6379> bitfield w overflow sat incrby u4 2 11)(integer) 15

Failure to execute

127.0.0.1: 6379> set w helloOK127.0.0.1: 6379> bitfield w overflow fail incrby u4 2 11)(integer) 11127.0.0.1: 6379> bitfield w overflow fail incrby u4 2 11)(integer) 12127.0.0.1: 6379> bitfield w overflow fail incrby u4 2 11)(integer) 13127.0.0.1: 6379> bitfield w overflow fail incrby u4 2 11)(integer) 14127.0.0.1: 6379> bitfield w overflow fail incrby u4 2 11)(integer) 15127.0.0.1: 6379> bitfield w overflow fail incrby u4 2 1 #Next is all lost 1)(nil) 127.0.0.1:6379> bitfield w overflow fail incrby u4 2 11)(nil) 127.0.0.1:6379> bitfield w overflow fail incrby u4 2 11)(nil)

get/set/incrby

127.0.0.1: 6379> bitfield w set u4 1 0 get u4 1 incrby u4 2 11)(integer) 02)(integer) 03)(integer) 1127.0.0.1: 6379> get w "\x04ello" about "redis bitmap is what it means" this article is shared here, I hope the above content can be of some help to everyone, so that you can learn more knowledge, if you think the article is good, please share it to let more people see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.