Shulou(Shulou.com)06/01 Report--

Today, I will talk to you about how to understand the implementation of the redis Bloom algorithm + locks, which may not be well understood by many people. In order to make you understand better, the editor has summarized the following contents for you. I hope you can get something according to this article.

Today's Qixi Festival, I do not know whether everyone is living in sweet honey or suffering, married old man is not worthy of Qixi Festival, can only quietly squat in front of the computer tapping code, writing documents, this is not, taking advantage of the company's afternoon holiday, I also sorted out the relevant knowledge about Redis Bloom filter and lock.

To put it simply, for Internet development, we go from single computer to multi-computer to micro-service. As the amount of data increases, the pressure on the database increases. At this time, hardworking programmers divide the data into cold data and hot data, and then store the hot data in the cache to improve query efficiency and reduce the pressure on the database. However, this is an ideal environment. After all, the Internet is not as calm as we thought.

For example, in the following situation

Hackers, a crowd that programmers yearn for but a headache for the security department, what to do when there are a large number of different key to obtain background data

At this time, I have to brag about the wisdom of my ancestors. I would rather kill three thousand by mistake and never let go of the pure idea of one, thus producing a magical Bloom filter. How does the Bloom filter perform? Keep looking down.

Bloom filter is the ─ algorithm that uses the error rate in exchange for space and time. The error rate is mainly reflected in: he said that if the data exists, it does not necessarily exist, it must not exist.

The code shows only part of it, because there is so much complete display that there is no way / * to determine whether keys exists in the collection where * / public boolean isExist (String where, String key) {long [] indexs = getIndexs (key); boolean result; / / the Redis pipeline is used here to reduce the number of visits to Redis during filter operation and the Redis concurrency Pipeline pipeline = jedis.pipelined () Try {for (long index: indexs) {pipeline.getbit (where, index);} result =! pipeline.syncAndReturnAll (). Contains (false);} finally {pipeline.close ();} / if (! result) {/ / put (where, key); / /} return result } / * key is stored in redis bitmap * / private void put (String where, String key) {long [] indexs = getIndexs (key); / / the Redis pipeline is used to reduce the number of Redis accesses during filter operation and the Redis concurrency Pipeline pipeline = jedis.pipelined () Try {for (long index: indexs) {pipeline.setbit (where, index, true);} pipeline.sync (); / * store data in mysql * /} finally {pipeline.close () }} / * the method of obtaining bitmap subscript based on key comes from guava * / public long [] getIndexs (String key) {long hash2 = hash (key); long hash3 = hash2 > 16; long [] result = new long [numHashFunctions]; for (int I = 0; I < numHashFunctions; isubscription +) {long combinedHash = hash2 + I * hash3 If (combinedHash < 0) {combinedHash = ~ combinedHash;} result [I] = combinedHash% numBits;} return result;} / * get a hash value method from guava * / private long hash (String key) {Charset charset = Charset.forName ("UTF-8") Return Hashing.murmur3_128 () .hashObject (key, Funnels.stringFunnel (charset)) .asLong ();} private static int optimalNumOfHashFunctions (long n, long m) {return Math.max (1, (int) Math.round ((double) m / n * Math.log (2));}

Of course, this situation is not just hacker attacks, there is another kind, I think many people have participated, it is called time-limited second kill, such as Taobao Singles' Day, one of the situations that will be caused at this time is that a large number of clients go to the background with the same Key to obtain data, but in this case, you can't go through the Bloom filter and not let go of one, so what should we do? Lock him up!

Maybe these pictures are a little blurry from this point of view, and individuals still have daily work, so they are not fully organized to form documents, so they are briefly described with pictures and simple text.

After reading the above, do you have any further understanding of how to understand the redis Bloom algorithm implementation + locking? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.