Shulou(Shulou.com)06/01 Report--

Big data how to thoroughly solve the distributed system consistency problem, many novices are not very clear about this, in order to help you solve this problem, the following editor will explain for you in detail, people with this need can come to learn, I hope you can gain something.

In fact, when personal understanding, more hope to be able to get code-level implementation, simple theoretical knowledge is still not enough to land, easy to sum up, the real implementation still needs the accumulation of the project.

A model that ensures ultimate consistency

1. Query mode

Any service operation needs to provide a query interface to output the status of the execution of the operation. That is, for a single query, in order to make the query operation have a unique identity, we need an ID in a distributed environment, the available distributed lock, redis increment, and the unique code of the machine is saved as the machine id, so that each query operation is relatively faster. It can solve the problem of synchronous call timeout and asynchronous callback timeout.

two。 Compensation mode

If the operation fails, we need to correct the problematic operation, which is to achieve consistency in the distributed system, and the efforts made to make the system consistent are called compensation operations. This scene is the same as the water problem! Compensation operations are divided into: automatic response, notification operation, and skill operation.

3. Async assurance mode

Then, for scenarios that do not require high response time, deal with them asynchronously, and then inform the user of the results. The biggest advantage of this solution is that it can eliminate the peak of high concurrent traffic, such as logistics, distribution and payment system in e-commerce.

4. Periodic proofreading mode

Then perform proofreading operations between mainstream systems, and then asynchronously proofread the status of batch proofreading operations. If you don't make compensation all the time, this sentence is a little difficult to understand. Check the system regularly as shown in the figure.

Regular proofreading system is mostly used in the financial system, aiming at the accuracy of the data.

5. Reliable message mode

Using message queue to realize asynchronization

Reliable delivery of ① messages

Before sending the message, the message is persisted to the database, the status is marked as unsent, and then the message is sent. If the message is sent successfully, the message is marked, and the message that has not been sent within a certain period of time is fetched from the database and sent.

Idempotency of ② message processor

Idempotency, idempotency, interview and doing exercises you will always encounter, if you want to solve it, you have to know what idempotence is.

In the network delay transmission, it will cause the message queue to retry, and in the process of enrichment, the message will be duplicated.

Solution:

1. If it is a database insert operation, make a primary key to the message to avoid dirty data.

two。 Use a third party to record consumption, such as Redis. Global id is K, message is V, and write to Redis. Check whether Redis exists before consumption.

3. Use the row-level lock of the database

6. Cache consistency mode

If you need non-relational databases to resist traffic in the face of 100 million-level reading requirements, is there a problem here? What is a relational database? (tabular databases that can be connected to each other)

The order of reading is to read the cache first, and then to read the database if you cannot read it. The order of writing is to write to the database first and then to the cache.

Timeout processing mode

1Interactive mode of microservices

① synchronous invocation mode

Service 1 invokes service 2, and service 1 waits for service 2 to return results.

② asynchronous invocation mode

Service 1 invokes service 2. After service 2 is processed, service 2 is notified in reverse.

③ message queuing Asynchronous processing Mode

Service 1 is passed to service 2, and you don't need to care about the return result.

Solution of ① synchronous invocation pattern

Two states

Three states

Solution of ② Asynchronous invocation pattern

Solution of ③ message queuing Asynchronous processing Mode

This paper analyzes the application scenarios such as synchronous invocation, asynchronous invocation and message queuing in the service-oriented system, and there are still some questions, such as when will it lead to message retransmission? What if the message retransmission still fails? What if I fail more than expected? There is how to achieve from the code level, these are too general in the interview, can not solve the actual problem, or from the actual scene to solve the problem.

Is it helpful for you to read the above content? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.