Shulou(Shulou.com)06/01 Report--

This article editor for you to introduce in detail "bitmap case analysis in redis", the content is detailed, the steps are clear, the details are handled properly, I hope this "bitmap case analysis in redis" article can help you solve your doubts, the following follow the editor's ideas slowly in depth, together to learn new knowledge.

1. What is BitMap

The value or state of an element is represented by a bit bit, where the key is the corresponding element itself. We know that eight bit can make up a Byte, so the bitmap itself will greatly save storage space. 2 ^ 32 power 4 billion data only needs 500m memory, which is 8 times less memory.

2. The setbit command describes that setbit key offset value # users who set bitmapkey to 20220328 uid have signed in to 1setbit 20220320 1setbit 20220320 200 1setbit 20220321 1setbit 20220321 300 1 getbit 20220320 100 #, indicating that this user has signed in to bitcount 20220320 # to obtain the number of bitmap

Bitmap's pit

127.0.0.1 offset 6400 > setbit bittest 1001 # setting does not exist offset returns 0 (integer) 0127.0.0.1 integer > setbit bittest 1001 # setting existing offset returns 1 (integer) 1

Setbit maxKey 4000000000 1 # directly got you more than 600m memory.

/ * * Bloom filter bloom Filter * 1. The probability of hash conflict is 1/1000000, so the existence does not necessarily exist, but the non-existence does not exist * 2. Can not delete, delete can not be simply set to 0 directly, may affect the judgment of other elements, in fact, the problem is not general production data will not be deleted, are soft deletion * 3. When adding new data, it takes only 600m to write bloom Filter * 4.2 ^ 32th power 4 billion data memory, super provincial memory, and the search speed is very fast. 160m memory can achieve 1% misjudgment in 10 million-level data * 5.bitmap applies for memory according to offset, so it is necessary to limit the offset value * / public function bloomAction () {$T1 = time (); for ($imemory). / / the probability of two hash 3sd5 hash repeats is 1/1000000 p ($res1);} / p ($res); $T2 = time (); echo $T2 echo; * Bloom filter initializes bloom Filter to execute php index.php "index/demo/loadDb2bloom" * / public function isExistBloomAction () {$redis = redisCursor () $email = input ("email", "", "trim"); $tel = input ("tel", ""); $result = false; $msg = ""; if (filter_var ($email,FILTER_VALIDATE_EMAIL)) {$key1 = "bloom_user_email"; $offset = BloomFilter::JSHash ($email); $result = $redis- > getbit ($key1,$offset) $msg = $email;} elseif ($tel) {$key2 = "bloom_user_telephone"; $offset = BloomFilter::JSHash ($tel); $result = $redis- > getbit ($key2,$offset); $msg = $tel;} $result?apiSuccess ($msg. ", already exists"): apiError ($msg. ", does not exist") } / * Bloom filter initializes bloom Filter to execute php index.php "index/demo/loadDb2bloom" * / public function loadDb2bloomAction () {$time1 = time (); $redis = redisCursor (); $key1 = "bloom_user_email"; $key2 = "bloom_user_telephone" / / setbit () offset must be a number, value must be 1 or 0 / / $redis- > setbit ($key,30,1); $table = "user"; $pkid = "id"; $field1 = "email"; $field2 = "telephone"; $maxid = Db::name ($table)-> max ($pkid); $size = 5000; $page = ceil ($maxid/$size); for ($item0) $iwhere ($where)-> field ("$field1,$field2")-> select (); if ($res) {/ / synchronize to bitmap foreach ($res as $k = > $v) {/ / Bloom filter 1. Existence does not necessarily exist, 2. 100% nonexistent (cause, hash conflict may be repeated with 1 / 100W) / / so it is judged that it does not exist at the time of registration, 100% can be registered, and if there is, you can check whether the database really does not exist $value1 = BloomFilter::JSHash ($v ["$field1"]). $value2 = BloomFilter::JSHash ($v ["$field2"]); $redis- > setbit ($key1,$value1,1); / / email de-weight $redis- > setbit ($key2,$value2,1); / / mobile de-weight} $time2 = time (); echo $where. " Elapsed time ". ($time2-$time1). PHP_EOL;} $time3 = time (); echo" total elapsed time ". ($time3-$time1) .PHP_EOL;}

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.