Shulou(Shulou.com)06/03 Report--

This article mainly explains "how to understand distributed transaction ideas and methods of 100 million-level traffic architecture". The content of the explanation in this article is simple and clear, and it is easy to learn and understand. let's study and learn how to understand distributed transaction ideas and methods of billion-level traffic architecture.

What is a distributed transaction

In daily life, many things are either all done or not done at all, or only some of them can be done, otherwise other complicated problems will arise. Many people like to cite the example of money transfer. For the same account, A transfers from Hubei to 500Magi B and withdraws 500 from Guangdong. After An is transferred out, the amount of money in An account should be deducted, and the number of B account will be increased: transaction = (An account deducts 500Query B account increased by 500)

See, if many steps like this are put together, that is, transactions are either executed or not executed, if our data is stored in multiple databases, that is, there are cross-library calls, due to the insecurity and latency of the network, how to ensure the distributed execution of transactions? What if there is a power outage in the middle of the execution? Before explaining distributed transactions, we will briefly review some of the features of transactions, commonly known as ACID. Let's explain them one by one:

Atomicity (Atomic)

In chemistry, a substance made up of molecules, the molecule is the smallest unit that maintains its chemical properties, such as H2O.CO2H2O, CO2, etc., a substance made up of atoms, and the atoms keep the properties of matter, such as FeFe, which means inseparable, and then divided into protons and neutrons is not what we think of as matter, and the atomicity here is the same truth, that is, things can no longer be separated, such as the above. It seems that it can be a transaction composed of two processes, but if you take it apart, it is not a process that we think it should have, so the transaction can no longer be divided and is atomic.

Consistency (Consistency)

Consistency is also easy to understand. For the above two accounts, if the bank wants to know how much money has been deposited here, then before the execution of this transaction, the An account has 500 yuan, the B account has no money, and the bank account has a total of 500 yuan. after the execution of the transaction, An account has no money, B account has 500 yuan, that is, this 500 yuan is certain, it is impossible to have 500 yuan for An account and 500 yuan for B account, then the data are inconsistent. In this case, it means that there is a problem with the execution of some steps in the transaction, resulting in intermediate data, then it is inconsistent.

In distributed, for one result, multiple queries at the same time, the results should be consistent.

Isolation (Isolation)

When one transaction is not completed, another transaction will not affect it, that is, if B transfers 1000 to C, it will be recorded as transaction 2:

Transaction 1 = (An account deducts 500jue B account increases by 500)

Transaction 2 = (B account deducts 1000 focus C account increases by 1000)

There will be no impact between the two transactions, that is, the $500 transferred by A will not arrive at the C account.

Persistence (Durability)

Persistence generally means that the data is written to disk and will not be changed easily. Here, after the transaction is committed, it will affect the database and will not be lost. This means that with the increasing size of the system, in order to improve the availability, maintainability, throughput and other technical indicators, even if we improve the original architecture, after the problem of business computing is solved, the database will still become the bottleneck of the whole system.

Discussion of consistency

In essence, the purpose of ACID is to protect the consistency of data, and the persistence of data will trigger database operations, resulting in low efficiency, so there are some discussions around consistency (efficiency), which are strong consistency, weak consistency and final consistency.

Strong consistency

You can read the last written data of a data at any read. All processes in the system, see the order of operation, are consistent with the order under the global clock. In short, at any time, the data in all nodes is the same, which requires that the data is written to the database as soon as it changes.

Weak consistency

After the data is updated, it is not required to write to the database and synchronize to all nodes in time, that is, there may be some differences between the data and the real data at this time. For the architecture, it is weak consistency if it can tolerate that only some or all of the subsequent access can be accessed.

Final consistency

It is not guaranteed that the same data on any node at any time is the same, that is, some node data may be accurate, some may be inaccurate, but with the migration of time, the same data on different nodes is always changing in the direction of convergence. To put it simply, after a period of time, the data between nodes will finally reach a consistent state.

Among the three kinds of consistency, strong consistency data is more reliable, but because all databases are required to keep data consistent all the time, the efficiency is low, the data is not unified, and the request cannot be responded to. In high concurrency scenarios, the experience is not very good. Therefore, in practical use, according to different business choices, the consistency is also different, and the account payment must be strong consistency when shopping. But the commodity inventory data does not have to be strong consistency, as for the comments below, you can even choose weak consistency.

Sub-database sub-table

I talked about the AKF split principle of the cluster (AKF of the Redis cluster split principle), which probably means that the hardware performance is limited by the upper limit. When the hardware cannot support the request traffic, the traffic can be distributed to different servers. The Y-axis and Z-axis split of AKF are business split and data split, which will involve splitting and storing the data in the database in different places. This is called split database table. Different types of data are stored in different databases for multi-computer storage and load, so the traditional transaction mechanism ACID can not work properly.

The content of sub-database and sub-table is data segmentation (Sharding), as well as the positioning and integration of data after segmentation. Specifically, data segmentation is to store data in multiple databases, so that the amount of data in a single database becomes smaller, and the performance problem of a single database is alleviated by expanding the number of hosts, so as to achieve the purpose of improving the operational performance of the database.

According to its segmentation type, data segmentation can be divided into two ways: vertical (vertical) segmentation and horizontal (horizontal) segmentation.

Vertical split

There are two kinds of vertical segmentation: vertical sub-database and vertical sub-table, and the two meanings are similar.

Vertical sub-database is to store different tables with low correlation in different databases according to business coupling. The practice is similar to the split of a large system into a number of small systems, which are divided independently according to the business classification. Similar to the practice of "microservice governance", each microservice uses a separate database. As shown in the figure:

A vertical sub-table is similar, for example, if a table contains all the information of a person, such as name, ID card, gender, height, weight, province, city, district, village, major, G-spot, etc., then it can be divided into three tables:

The first table contains only basic information (name, ID card, gender, height, weight)

The second table contains place of origin information (province, city, district, village)

The third table contains learning information (major, G-spot).

Advantages and disadvantages of vertical split

Advantages of vertical slicing:

Solve the coupling at the business system level, the business is clear

Similar to the governance of micro-services, it can also manage, maintain, monitor and expand the data of different businesses.

In high concurrency scenarios, vertical sharding can improve the bottleneck of IO, database connections and stand-alone hardware resources to a certain extent.

Disadvantages of vertical slicing:

Some tables cannot be join, so they can only be solved by interface aggregation, which increases the complexity of development.

Distributed transaction processing is complex

There is still the problem of excessive amount of data in a single table (horizontal segmentation is required)

Horizontal split

After the vertical split of the database, if a database is still large, for example, the stored data is extremely large, then you can split the database horizontally:

The above horizontal split is divided according to the ID interval. For example, assign records with a userId of 1 to 10000 to the first library, 10001to 20000 to the second library, and so on. In a sense, the "hot and cold data separation" used in some systems migrates some less used historical data to other libraries, and the business function only provides queries for hot data, which is a similar practice.

In addition to the above split according to the user ID interval, you can also do Hash operation split, here will not expand in detail.

Advantages and disadvantages of horizontal split

The advantages of horizontal split are:

The size of single table can be controlled.

Naturally, it is easy to scale horizontally. If you want to expand the capacity of the entire shard cluster later, you only need to add nodes, and there is no need to migrate other shard data.

When using fragment fields for range search, continuous fragments can quickly locate fragments for fast query, which can effectively avoid the problem of cross-fragment query.

Horizontal split disadvantages:

Hot data has become a performance bottleneck. Continuous sharding may have data hotspots, such as slicing according to time field, some fragments store data in the most recent time period, which may be read and written frequently, while some historical data stored in fragments are rarely queried.

Problems caused by sub-database and sub-table

Sub-library and sub-table can effectively alleviate the performance bottleneck and pressure caused by single machine and single library, and break through the bottleneck of network IO, hardware resources and connection number. At the same time, it also brings some problems. As mentioned earlier, the transaction contains a set of sub-operations, which are either all executed or not executed, but after dividing the database, a transaction may involve multiple databases or multiple tables to expand the database for execution. The network is unstable, that is, transaction execution is more difficult. Transactions after sub-table and sub-database are called distributed transactions (cross-fragment transactions) in order to distinguish them from traditional transactions.

Cross-fragment transactions are also distributed transactions, there is no simple solution, generally can use the "XA protocol" and "two-phase commit" processing.

Distributed transactions can maximize the atomicity of database operations. However, multiple nodes need to be coordinated when the transaction is committed, which postpones the time point of committing the transaction and prolongs the execution time of the transaction. Increases the probability of conflicts or deadlocks when a transaction accesses a shared resource. With the increase of database nodes, this trend will become more and more serious, which will become the shackle of the horizontal expansion of the system at the database level.

Final consistency

For those systems with high performance requirements but low consistency requirements, the real-time consistency of the system is often not demanding, as long as the final consistency is achieved within the allowable period of time, transaction compensation can be used. Different from the way in which transactions are rolled back immediately after an error occurs in execution, transaction compensation is a remedial measure for checking afterwards. Some common implementation methods are: reconciliation of data, comparison based on logs, regular synchronization with standard data sources, and so on. Transaction compensation should also be considered in combination with the business system.

Distributed transaction solution

Before we talk about this, we need to briefly review the CAP principle and Base theory. Because the distributed transaction is different from the rigid transaction of ACID, the concept of flexible transaction is put forward based on the BASE theory in the distributed scenario. In order to achieve the ultimate consistency through flexible transactions, we need to rely on some characteristics, which do not necessarily have to be satisfied in specific schemes, because different schemes have different requirements; but if they are not satisfied, it is impossible to do flexible transactions.

CAP principle

The average person of CAP may have heard it more than a hundred times. Many people say that CAP is a relationship of "three choices and two choices", which makes people mistakenly think that there is such a situation as AC. But in fact, CAP is a relationship of choosing one of two. This has already been explained in a paper in 2012: CAP Twelve Years Later: How the "Rules" Have Changed.

It is equivalent to revising the previous statement that P (partition fault tolerance) in CAP is a must. Under the premise of satisfying P, it is difficult to satisfy both A (usability) and C (consistency), but after that, there is another article: Harvest, yield, and scalable tolerant systems, which is based on the above paper "12 years after CAP", which mainly puts forward the concepts of Harvest and Yield. And talk about what is discussed in the above paper in more detail. To put it simply, after satisfying P, C and A can be taken into account after relaxing the constraints. It is not an either-or relationship.

Why is P necessary?

Why is partition fault tolerance necessary in the CAP principle? first of all, we should understand what partition fault tolerance is. Here we are talking about the network. The network cluster is designed to many servers. If the network is instantly unstable, it is tantamount to dividing the network into different zones. Suppose it is divided into two zones. At this time, if there is a transaction:

Send a message to partition 1: a transfers 100 yuan to B, and send a message to partition 2: a transfers 200 yuan to B.

So for the two partitions, there are two situations:

A) No availability, that is, at least one of the two transactions will not be accepted

B) there is no consistency. Half of them see A transfer 100 yuan to B and the other half see A transfer 200 yuan to B.

Therefore, the partition tolerance must be satisfied, and the solution strategy is to copy a data item to multiple nodes, then after the partition appears, the data item may be distributed to each zone. Tolerance is improved.

Base theory

In many cases, we do not need a strongly consistent system, so later, when people argue about data consistency and availability, they mainly focus on the strongly consistent ACID or the final consistent BASE. BASE is the result of the tradeoff between consistency and availability in CAP, which comes from the summary of the practice of large-scale Internet distributed systems and is gradually evolved based on CAP's law. Its core idea is that even if strong consistency can not be achieved, each application can use an appropriate way to achieve the final consistency of the system according to its own business characteristics.

BASE theory is the abbreviation of three phrases: Basically Available (basic availability), Soft State (soft state) and Eventually Consistent (ultimate consistency).

Basic availability

Suppose the system has an unpredictable failure, but it still works, compared to a normal system:

Loss of response time: normally, search engines return results to users in 0.5 seconds, while basic available search engines can return results in 2 seconds.

Functional loss: on an e-commerce website, under normal circumstances, users can successfully complete every order. But during the promotion period, in order to protect the stability of the shopping system, some consumers may be directed to a downgraded page.

This is called basic availability.

Soft state

Relative to atomicity, data copies of multiple nodes are required to be consistent, which is a "hard state". The soft state means that the data in the system is allowed to have an intermediate state, and it is considered that the state does not affect the overall availability of the system, that is, it allows the system to have data delay in the data copies of many different nodes.

Final consistency

It says soft state, and then it can't be soft all the time, there must be a time limit. After the expiration of the period, all copies should be ensured to maintain data consistency, so as to achieve the ultimate consistency of the data. This time period depends on network latency, system load, data replication scheme design, and so on.

The core idea of Base is:

Since it is impossible to achieve strong consistency (Strong consistency), each application can adopt appropriate ways to make the system achieve final consistency (Eventual consistency) according to its own business characteristics. With Base theory, we can begin to talk about the idea of distributed transaction processing.

Two-phase submission protocol

Two-phase commit (2PC:Two-Phase Commit), as its name implies, splits a distributed transaction process into two phases: voting and transaction commit. In order to make the whole database cluster run normally, the protocol specifies a single point of coordinator, which is used to coordinate the operation of each node of the whole database cluster. In order to simplify the description, we refer to each node in the database cluster as participants, and the three-phase commit protocol also includes two role definitions: coordinator and participant, which will be discussed later.

Stage one: voting

The main purpose of this phase is to find out whether each participant in the database cluster can execute the transaction normally. The specific steps are as follows:

The coordinator sends transaction execution requests to all participants and waits for participants to feedback the transaction execution results.

After receiving the request, the transaction participant executes the transaction without committing, and records the transaction log

The participant feeds back the execution of his transaction to the coordinator while blocking the subsequent instructions of the coordinator.

Phase 2: transaction commit

After the inquiry from the coordinator in the first phase, each participant will reply to the execution of his or her own transaction, at this time there are three possibilities:

All participants returned to be able to perform the transaction normally.

One or more participants replied that the transaction execution failed.

The coordinator waits for a timeout.

In case 1, the coordinator will notify all participants of the commit of the transaction, as follows:

The coordinator sends a commit notification to each participant, requesting that the transaction be submitted

The participant performs the commit operation after receiving the transaction commit notification, and then releases the occupied resources

The participant returns the transaction commit result information to the coordinator.

For cases 2 and 3, the coordinator believes that the participant cannot execute the transaction successfully, so a transaction rollback notification is sent to each participant for the sake of consistency of the entire cluster data. The specific steps are as follows:

The coordinator sends a transaction rollback notification to each participant, requesting that the transaction be rolled back

After receiving the transaction rollback notification, the participant performs the rollback operation and then releases the occupied resources

The participant returns the transaction rollback result information to the coordinator.

The two-phase commit protocol solves the problem of strong data consistency in distributed databases, and in practical applications, it is more used to solve the atomicity of transaction operations. The following figure depicts the state transition between coordinators and participants.

From the coordinator's point of view, after initiating the vote, we enter the WAIT waiting state, waiting for all participants to return to their transaction execution status, and after receiving responses from all participants, the next step is to send commit submission or rollback rollback messages.

From the point of view of the participants, after replying to the coordinator's voting request, they enter the READY state (the transaction can be executed normally), and then wait for the coordinator's final decision notification, and once they receive the notification, they can perform commit or rollback operations according to the decision.

The two-phase commit protocol is simple in principle and easy to implement, but its disadvantages are obvious, including the following:

Single point problem

The coordinator plays an important role in the whole two-phase commit process. Once the server of the coordinator goes down, it will affect the normal operation of the whole database cluster. For example, in the second phase, if the coordinator cannot properly send transaction commit or rollback notifications due to failure, the participants will remain blocked and the entire database cluster will not be able to provide services.

Synchronous blocking

In the process of two-phase commit execution, all participants need to listen to the unified scheduling of the coordinator, during which they are blocked and can not engage in other operations, which is extremely inefficient.

Data inconsistency

Although the two-phase commit protocol is designed for strong consistency of distributed data, there is still the possibility of data inconsistency. For example, in the second phase, suppose the coordinator sends out a transaction commit notification, but because of network problems, only some participants receive and perform commit operations, while the rest of the participants remain blocked because they do not receive the notification, which leads to data inconsistency.

In order to solve the above problems, timeout mechanism and mutual inquiry mechanism can be introduced to solve them to a great extent.

Timeout mechanism

For the coordinator, if they do not receive a reply from all participants within a specified period of time, they can automatically exit the WAIT status and send an rollback notification to all participants. For participants who are in the READY state but do not receive the second phase notification from the coordinator within a specified period of time, the rollback operation cannot be performed arbitrarily, because the coordinator may be sending a commit notification, at which point the execution of rollback will result in data inconsistency.

Mutual inquiry mechanism

At this point, we can intervene in the mutual inquiry mechanism and let participant An inquire about the implementation of other participants B. If B performs a rollback or commit operation, A can boldly perform the same operation as B; if B has not reached the READY state at this time, it can be inferred that the coordinator must have sent a rollback notification; if B is also in the READY state, A can continue to ask other participants. Only when all participants are in the READY state, the two-phase commit protocol cannot be processed and will be blocked for a long time.

Three-phase submission agreement

Three-phase commit protocol (3PC:Three-Phase Commit), in view of the problems of two-phase commit, three-phase commit protocol introduces a pre-inquiry phase and timeout strategy to reduce the blocking time of the whole cluster and improve system performance. The three stages of commit are: pre-inquiry (can_commit), pre-commit (pre_commit), and transactional commit (do_commit).

The first stage: pre-inquiry

At this stage, the coordinator will ask each participant whether the transaction can be executed normally, and the participant will reply to an estimated value according to his or her own situation. This process is light compared to the actual transaction execution. The specific steps are as follows:

The coordinator sends a transaction inquiry notification to each participant, asks if the transaction operation can be performed, and waits for a reply

Each participant returns an estimated value according to his or her own situation, and returns certain information if it is estimated that he can perform the transaction normally, and enters the preparatory state, otherwise it returns negative information.

Phase 2: pre-submission

At this stage, the coordinator will take corresponding actions according to the inquiry results of the first stage. There are three main types of inquiry results:

All participants return confirmation information.

One or more participants returned negative information.

The coordinator waits for a timeout.

In case 1, the coordinator sends a transaction execution request to all participants as follows:

The coordinator sends transaction execution notifications to all transaction participants

After receiving the notification, the participant executes the transaction but does not commit

The participant returns the transaction execution to the client.

In the above steps, if the participant waits for a timeout, the transaction is interrupted. For scenarios 2 and 3, the coordinator decided that the transaction could not be executed properly, so he issued an abort notification to each participant requesting to exit the standby state, the specific steps are as follows:

The coordinator sends abort notifications to all transaction participants

The participant interrupts the transaction after receiving the notification.

Phase 3: transaction commit

If the second phase of the transaction is not interrupted, the coordinator of this phase will decide whether to commit or roll back the transaction based on the result returned by the transaction execution, which can be divided into three cases:

All participants can perform transactions normally.

One or more participants failed to execute the transaction.

The coordinator waits for a timeout.

In case 1, the coordinator initiates a transaction commit request to each participant, as follows:

The coordinator sends transaction commit notifications to all participants

All participants perform the commit operation after receiving the notification and release the occupied resources

The participant feedback the result of the transaction submission to the coordinator.

In cases 2 and 3, the coordinator believes that the transaction cannot be executed successfully, so a transaction rollback request is sent to each participant as follows:

The coordinator sends transaction rollback notifications to all participants

All participants perform the rollback operation after receiving the notification and release the occupied resources

Participants feed back the results of the transaction rollback to the coordinator.

At this stage, if the participant is unable to receive the commit or rollback request from the coordinator because of the coordinator or network problem, the participant will not be blocked as in the two-phase commit, but will continue to commit after waiting for the timeout. Although the synchronization blocking is reduced compared to the two-phase commit, the data inconsistency can not be completely avoided. The probability of long-term blocking in two-phase commit protocols is still very low, so although three-phase commit protocols are more guaranteed for strong data consistency than two-phase commit protocols, because of efficiency, two-phase commit protocols are more popular in practical systems.

TCC mode

TCC is an acronym for Try, Confirm, and Cancel. Their respective responsibilities are:

Try: responsible for reserving resources (such as creating an order with status = PENDING)

To do business inspection, to put it simply, resources that have been occupied cannot be reserved.

Isolate reserved resources.

Confirm: responsible for the resources reserved for landing

The real execution business uses the resources reserved in the try phase, idempotent.

Cancel: responsible for revoking reserved resources

Users are required to implement Try, Confirm and Cancel operations according to their own business scenarios; the transaction initiator executes the Try mode in the first phase, the Confirm method in the second phase commit, and the Cancel method in the second phase rollback.

Say a few more words about reserved resources, resources are limited, so reserved resources are time-limited, if reserved resources do not get Confirm-- for a long time, we call this situation as timeout-- participants will Cancel them themselves. In other words, the participants have the ability to self-manage the resources, so as to avoid the long-term occupation of resources due to the problems of the initiators.

TCC adds the ability to check business and undo transactions. At the same time, TCC promotes the actions at the 2PC database level to the service level, except that all the actions in TCC are a local transaction, and each local transaction is commit to the database after the action is completed:

Try is equivalent to 2PC's Commit request phase, plus business check logic

Confirm is equivalent to the commit action of 2PC's Commit phase.

Cancel is equivalent to the rollback action of 2PC's Commit phase.

Process steps:

Initiator sends Try to all participants

Each participant executes Try to reserve resources

The initiator receives Try results from all participants

The initiator sends Confirm/Cancel to all participating rooms.

Each participant executes the Confirm/Cancel

The initiator receives Confirm/Cancel results from all participants

The process is very similar to a two-phase commit.

Thank you for your reading. the above is the content of "how to understand distributed transaction ideas and methods of 100 million-level traffic architecture". After the study of this article, I believe you have a deeper understanding of how to understand distributed transaction ideas and methods of 100 million-level traffic architecture, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.