Shulou(Shulou.com)06/01 Report--

In this issue, the editor will bring you about how to analyze the consistency of multithreaded data access and its solutions in big data. The article is rich in content and analyzes and narrates it from a professional point of view. I hope you can get something after reading this article.

Both servers and laptops use multicore CPU. For performance reasons, each CPU uses a unique Cache mechanism (usually there are two levels of exclusive cache and one level of shared cache), which leads to concurrent access to the same data in memory, visibility or consistency on different CPU. CPU uses Cache mechanism to bring the consistency or data visibility problems of multi-thread access to shared data, which can not be bypassed by engineers engaged in multi-thread programming.

Knowing the problem, there are many ways to solve the problem of data consistency under concurrent access. The most commonly used method is locking, for example, thread locks are directly used to achieve access mutual exclusion under Chammer Clipper +, while ensuring the visibility of modified data. For counter scenarios, you can also use higher-performance atomic operations (about six times the performance of thread locks), such as gcc's built-in _ _ sync_add_and_fetch / _ _ sync_fetch_and_add or _ _ sync_sub_and_fetch / _ _ sync_fetch_and_sub functions. In addition, lock-free programming can be achieved by using CAS (Compare And Swap) atomic operations, such as _ _ sync_bool_compare_and_swap and _ _ sync_val_compare_and_swap built into gcc. For example, we can use CAS to implement lock-free queues.

For the threading model of a producer and a consumer, using data structures such as ring buffer, lock-free programming can be used because there is no access conflict. For example, the Linux kernel handles ring buffer, combined with a memory barrier, to ensure data visibility without locks. The memory barrier is more esoteric, I have no research and practice in this area, just throw a brick here.

For a scenario in which a thread modifies data and multiple threads read, a more extreme lock-free programming approach: not only is there no lock at all (no CAS is required), but also no memory barrier is used to ensure data visibility. This approach obviously violates the data consistency of concurrent access, but it is perfectly feasible for scenarios where acceptable data is ultimately consistent. Why is that? After one thread modifies the data, other threads will delay sensing the modified data because of the CPU cache mechanism. According to my experience, this delay is usually within 1ms. Because there is no mechanism such as locking or memory barrier, the visibility of data in multithreading is uncertain. Data modification for most data structures (such as balanced binary tree, red-black tree, skiplist, hashtable, etc.) is usually completed in multiple steps. Here, we would like to remind you that a prerequisite for adopting this approach is that when the reading thread reads part of the updated data, it will not lead to abnormal conditions such as program logic confusion or even crash. It is difficult to use this programming model for data structures such as balanced binary trees (because maintaining balance involves rotation operations and requires a large lock protection), while carefully implemented skiplist and hashtable can be done.

The above is how to analyze the consistency of multithreaded data access in big data, which is shared by the editor. If you happen to have similar doubts, you might as well refer to the above analysis to understand. If you want to know more about it, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.