Shulou(Shulou.com)06/01 Report--

This article mainly introduces the example analysis of Redis find hot key, has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let the editor take you to understand it.

Before

Cache avalanche, that is, a large area of cache failure at the same time, when there is another wave of requests, resulting in requests to the database, resulting in abnormal database connection.

Solution:

(1) add a random value to the cache expiration time to avoid collective failure.

(2) mutexes are used, but the throughput of the scheme is significantly reduced.

(3) double cache. We have two caches, cache An and cache B. The expiration time of cache An is 20 minutes, and cache B has no expiration time. Do your own cache warm-up operation. Then subdivide the following dots

I read the database from cache A, and return it directly.

II A has no data, reads the data directly from B, returns directly, and starts an update thread asynchronously.

The III update thread updates both cache An and cache B.

After

For any sudden hot spot data that cannot be perceived in advance, including not limited to hot spot data (such as sudden massive requests for the same product), hot users (such as malicious crawler brushes), hot interfaces (sudden massive requests for the same interface), accurate detection at the millisecond level. The hot data and hot users are then pushed to the JVM memory of all servers to significantly reduce the impact on the back-end data storage layer, and it is up to the user to decide how to allocate and use these hot key (such as local caching of hot goods, denial of access to hot users, fusing of hot interfaces or returning to default values). These hot data are consistent throughout the server cluster, and the business is isolated, and the performance of the worker side is strong.

After many pressure tests, the worker terminal of the 8-core single machine can receive and process 160000 key detection tasks per second, the 16-core single machine can smoothly process more than 300000 per second, the actual pressure test reaches 370000, the CPU is supported smoothly, and the frame is normal.

It is suggested that the design and practice of JD.com millisecond hot key detection framework for consumption with official account content.

Core function: hot data is detected and pushed to each server in the cluster

Applicable scenarios:

1 mysql hot data local cache

2 redis hot data local cache

3. Local cache for blacklisted users

4 crawler users limit current

5 interface, user dimension current limit

6 stand-alone interface, user dimension current limit

7 the user dimension of cluster is limited.

8. Current limit in the dimension of cluster interface

The open source project strategy is of great significance and has experienced millions of concurrency. I have been waiting for you to participate in the construction of JD.com open source middleware project.

Gitee source code address https://gitee.com/jd-platform-opensource/hotkey

It is also recommended to read the parallel framework to solve any multi-thread parallelism, serial, blocking, dependency and callback. You can combine the execution order of each thread arbitrarily, with full-link execution result callback. One-stop solution for multi-thread choreography

Gitee address: https://gitee.com/jd-platform-opensource/asyncTool?_from=gitee_search thank you for reading this article carefully. I hope the article "sample Analysis of Redis find hot key" shared by the editor will be helpful to you. At the same time, I also hope that you will support and follow the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.