Shulou(Shulou.com)05/31 Report--

This article mainly explains "how to solve the Redis cache problem". The content of the article is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn how to solve the Redis cache problem.

Here comes LevelDB!

It is Google's open source NOSQL storage engine library and is an atomic bomb in the field of modern distributed storage. On the basis of it, Facebook developed another NOSQL storage engine library, RocksDB, which not only uses the advanced technical architecture of LevelDB, but also solves some of the shortcomings of LevelDB. You can compare RocksDB to a hydrogen bomb, which is a little more powerful than LevelDB. There are many databases in the modern open source market that use RocksDB as the underlying storage engine, such as the famous TiDB.

But why should I talk about LevelDB instead of RocksDB? The reason is that the LevelDB technology architecture is simpler, clearer and easier to understand. If we eat through LevelDB first and then nibble on RocksDB, it will be very easy to understand, and RocksDB is just a series of optimizations based on LevelDB. By the time we broke the RocksDB hydrogen bomb, the TiDB nuclear-powered spaceship was waiting for us not far ahead.

What's wrong with Redis caching?

When we use Redis for caching, there must be a persistence database that records all the hot and cold data. Data consistency between Redis and persistence layer databases is controlled by the application itself. The application will first get the data from the cache, and when there is no data in the cache, the application needs to load the data from the persistence layer and then put it into the cache. When a data update occurs, the cache needs to be set to invalidate.

Function getUser (String userId) User {

User user = redis.get (userId)

If user = = null {

User = db.get (userId)

If user! = null {

Redis.set (userId, user)

}

}

Return user

}

Function updateUser (String userId, User user) {

Db.update (userId, user)

Redis.expire (userId)

}

Friends who have developed experience in this area know that writing such code is still tedious, and all business code involving caching needs to add this part of the logic.

Strictly speaking, we also need to carefully consider the issue of cache consistency. For example, in the updateUser method, the database performs updates correctly, but the cache redis is set to fail due to network jitter and other reasons, then the data in the cache becomes expired data. If you reverse the order of setting the cache and updating the persistence, there will be other problems, and the reader can think for himself.

Cache inconsistencies can also be caused in situations where multiple processes are highly concurrent. For example, a process calls the getUser () method on a userId because it is not in the cache and needs to be loaded from the database. The result has just been loaded and is about to set up the cache, when the in-memory fullgc code pauses for a while, while another process calls the updateUser method to update the database, setting the cache to invalidate (in fact, there is no data in the cache). Then the previous process finally ends the fullgc and starts to set up the cache, at which time the expired data is cached.

How does LevelDB solve it?

LevelDB combines the Redis cache and the persistence layer into one to help you fix the cache and persistence layer at once. With LevelDB, your code can be simplified to the following

Function getUser (String userId) User {

Return leveldb.get (userId)

}

Function updateUser (String userId, User user) {

Leveldb.set (userId, user)

}

And you no longer have to worry about caching consistency issues. LevelDB's data updates are either successful or unsuccessful, and there is no intermediate Schrodinger state. LevelDB has built-in memory cache and persistence layer of disk files, users do not have to worry about how the internal data is consistent.

What exactly is LevelDB?

As we mentioned earlier, it is a NOSQL storage engine, which is not the same concept as Redis. Redis is a complete database, while LevelDB is just an engine. If the database must be a high-end sports car, then the storage engine is its engine, the core is the heart. With this engine, we pack it with a series of accessories and decorations, and it can become a database. However, do not underestimate accessories and decorations, it is also very difficult to achieve the extreme, packaging LevelDB into an easy-to-use database requires too many exquisite accessories. LevelDB and RocksDB have been out for so many years, very few of them have been able to make a very complete production database on the basis of it.

When using LevelDB, we can also think of it as an Key/Value in-memory database. It provides the basic Get/Set API through which we can read and write data in the code. You can also think of it as an infinite size advanced HashMap, and we can stuff it with unlimited strips of Key/Value data, as long as the disk can hold it.

Precisely because it can only be counted as an in-memory database, the data contained in it cannot be shared across processes and machines. How can LevelDB show its skills in the distributed field?

This depends on packaging technology, packaging a layer of network API on the basis of LevelDB in-memory database. When different processes on different machines want to access it, they all use the network API interface. This forms a simple database. If we use the Redis protocol to wrap it at the network layer, clients using Redis can read and write to the database.

If we want to consider the high availability of the database, we can transform it into a master-slave distributed NOSQL database by adding the master-slave replication function to the above stand-alone database. By adding a layer of forwarding agents (load balancers such as LVS, F5, etc.) in front of the master-slave database, the real-time master-slave switching can be realized.

If the data capacity you need is so large that there is no room for the hard disk of a single machine, you need a data slicing mechanism to distribute the data of the entire database to multiple machines, and each machine is only responsible for reading and writing part of the data. There are many schemes for data sharding, such as Codis through forwarding agent, client forwarding mechanism like Redis-Cluster, and Raft distributed consistency algorithm of TiDB to manage sharding. The simplest and easiest to understand is the forwarding agent shard of Codis.

When the amount of data continues to grow and needs to add new nodes, the data part of the old node must be migrated to the new node. It is a new advanced accessory-data equalizer that manages the balance and migration of data.

Thank you for your reading, the above is the content of "how to solve the Redis cache problem". After the study of this article, I believe you have a deeper understanding of how to solve the Redis cache problem, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.