Shulou(Shulou.com)06/01 Report--

In this article Xiaobian for you to introduce in detail the "Java concurrent programming database and cache data consistency program is what", detailed content, clear steps, details handled properly, I hope that this "Java concurrent programming database and cache data consistency plan is what" article can help you solve doubts, following the editor's ideas slowly in-depth, together to learn new knowledge.

I. Preface

In distributed concurrent systems, the consistency of database and cached data is a challenging technical difficulty. Assuming that there is a perfect industrial distributed transaction solution, then the consistency of database and cached data can be easily solved. in fact, the current distributed transaction is not mature.

Second, different voices

There are all kinds of voices in the consistent solution of database and cached data.

Operate the database before caching or caching before the database

Whether the cache is updated or deleted

1. The sequence of operation

In the concurrent system, in the double write scenario of database and cache, in order to pursue a larger amount of concurrency, it is obvious that the operation of database and cache will not be carried out synchronously. The former is operated successfully and the latter is carried out asynchronously.

As a mature industrial data storage scheme, relational database has a perfect transaction processing mechanism. once the data is down, regardless of hardware failure, it can be said responsibly that the data will not be lost.

The so-called cache is nothing more than the data stored in memory. Once the service is restarted, all the cached data is lost. Since it is called caching, you are always ready to lose cached data. Although Redis has a persistence mechanism, is it possible to guarantee 100% persistence? It is impossible for Redis to persist data asynchronously to disk, cache is cache, database is database, two different things. It is extremely dangerous to use the cache as a database.

From the point of view of data security, we first operate the database, and then operate the cache asynchronously to respond to user requests.

2. Attitude towards caching

Whether the cache is updated or deleted, corresponding to lazy and full Chinese, it is relatively less difficult to delete cache from the practice of dealing with thread safety. If query performance is satisfied on the premise of deleting the cache, then deleting the cache is preferred.

Although updating the cache can improve the query efficiency, it is more troublesome to deal with the thread and dirty data. The preface introduces other message middleware such as MQ, so it is not recommended.

Third, thread concurrency analysis

The key to understand the problems caused by thread concurrency is to understand the system interrupt first. When the operating system is scheduling tasks, interrupts occur at any time, which is the root cause of thread data inconsistency. The 4 and 8 thread CPU, for example, can handle up to eight threads at a time, but the operating system manages far more than eight threads, so the threads proceed in a seemingly parallel manner.

Query data 1. Non-concurrent environment

In a non-concurrent environment, there is nothing wrong with querying the data in the following way: first query the cache, if the cached data does not exist, query the database, update the cache, and return the results.

Public BuOrder getOrder (Long orderId) {String key = ORDER_KEY_PREFIX + orderId; BuOrder buOrder = RedisUtils.getObject (key, BuOrder.class); if (buOrder! = null) {return buOrder;} BuOrder order = getById (orderId); RedisUtils.setObject (key, order, 5, TimeUnit.MINUTES); return order;}

If there is a serious defect in a high concurrency environment: when the cache expires, a large number of query requests pour in and all of them are immediately called to the DB, the database connection resources are exhausted, the client responds to 500errors, and the database is under too much pressure and the service is down.

2. Concurrent environment

Therefore, in a concurrent environment, the above code needs to be modified to use distributed locks. When a large number of requests pour in, the thread that acquired the lock has the opportunity to access the database query data, and the rest of the threads block. When the data is queried and the cache is updated, the lock is released. The waiting thread re-examines the cache and finds that it can get the data and respond to the cached data directly.

Distributed locks are mentioned here, so do you use table locks or row locks? Use distributed row locks to increase concurrency; use a secondary check mechanism to ensure that threads waiting for locks can quickly return results

@ Overridepublic BuOrder getOrder (Long orderId) {/ * if cache does not exist, add distributed lock update cache * / String key = ORDER_KEY_PREFIX + orderId; BuOrder order = RedisUtils.getObject (key, BuOrder.class); if (order! = null) {return order;} String orderLock = ORDER_LOCK + orderId; RLock lock = redissonClient.getLock (orderLock) If (lock.tryLock ()) {order = RedisUtils.getObject (key, BuOrder.class); if (order! = null) {LockOptional.ofNullable (lock) .ifLocked (RLock::unlock); return order;} BuOrder buOrder = getById (orderId); RedisUtils.setObject (key, buOrder, 5, TimeUnit.MINUTES); LockOptional.ofNullable (lock) .ifLocked (RLock::unlock) } return RedisUtils.getObject (key, BuOrder.class);} update data 1. Non-concurrent environment

In a non-concurrent environment, the following code may cause data inconsistencies (data overwriting). Although the use of database-level optimistic locks can solve the problem of data overwriting, invalid update traffic will still flow to the database.

Public Boolean editOrder (BuOrder order) {/ * Update database * / updateById (order); / * delete cache * / RedisUtils.deleteObject (OrderServiceImpl.ORDER_KEY_PREFIX + order.getOrderId ()); return true;} 2, concurrent environment

The use of database optimistic locks in the above analysis can solve the problem that data is overwritten in concurrent updates. However, when the same row of records is modified, the version number changes, and subsequent concurrent requests to the database are invalid traffic. The first strategy to reduce the pressure on the database is to block invalid traffic in front of the database.

The use of distributed locks can ensure that concurrent traffic accesses the database in an orderly manner. considering that optimistic locks have been used at the database level, the second and subsequent threads that acquire locks operate on the database as invalid traffic.

When acquiring the lock, the thread adopts the strategy of exiting during the timeout, waits for the thread that acquired the lock to exit quickly, responds quickly to the user's request, and retries to update the data operation.

Public Boolean editOrder (BuOrder order) {String orderLock = ORDER_LOCK + order.getOrderId (); RLock lock = redissonClient.getLock (orderLock); try {/ * timeout failed to acquire lock quickly, client retry * / if (lock.tryLock (1, TimeUnit.SECONDS)) {/ * update database * / updateById (order) / * delete cache * / RedisUtils.deleteObject (OrderServiceImpl.ORDER_KEY_PREFIX + order.getOrderId ()); / * release locks * / LockOptional.ofNullable (lock) .ifLocked (RLock::unlock); return true;}} catch (InterruptedException e) {e.printStackTrace ();} return false;} dependent environment

The above code uses the utility class that encapsulates the lock.

Xin.altitude.cms ucode-cms-common 1.4.3.2

LockOptional performs subsequent actions based on the state of the lock.

Database before caching data consistency 1. Problem description

Next, we discuss whether there is a concurrency problem of updating the database and then deleting the cache.

(1) the cache just expires

(2) request A to query the database to get an old value.

(3) request B to write the new value to the database

(4) request B to delete the cache

(5) request A to write the old values found to the cache

The key to the above concurrency problem is that step 5 occurs more than step 3 and step 4, which is known by the uncertain factors of operating system interruption, but this situation is likely to happen.

2. Solution

In practice, it takes much less time to write data to Redis than to write data to a database, and although it is less likely to happen, it still happens.

(1) increase cache expiration time

Increasing cache expiration time allows visceral data to exist for a certain period of time, until the next concurrent update occurs, dirty data may occur. Dirty data will exist periodically.

(2) share a row lock for updates and queries

Updates and queries share a row distributed lock, and the above problems no longer exist. When the read request acquires the lock, the write request is blocked (the timeout will quickly fail to return), ensuring that step 5 is carried out before step 3.

(3) delay deletion of cache

Use RabbitMQ to delay the deletion of the cache to remove the impact of step 5. It is done asynchronously with little impact on performance.

Special circumstances

The database has a transaction mechanism to ensure the success of the operation; the Redis single instruction is atomic, and then combined without atomic characteristics, specifically, the database operation is successful, and then the application exception fails, resulting in the Redis cache not deleted. This problem occurs when the Redis service network connection timed out.

If the cache expiration time is set, dirty data exists until the cache expires. If the expiration time is not set, dirty data exists until the next time the data is modified. (the database data has changed and the cache has not been updated)

Solution method

Before operating on the database, write a message to RabbitMQ that delays deleting the cache, and then perform the database operation to perform the cache delete operation. Regardless of whether the code-level cache is deleted successfully or not, MQ deletes the cache as a guaranteed operation.

Read here, this article "Java concurrent programming database and cache data consistency solution" article has been introduced, want to master the knowledge of this article still need to practice and use to understand, if you want to know more about the article, welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.