Shulou(Shulou.com)05/31 Report--

This article mainly introduces "how to understand Trafodion transaction management". In daily operation, I believe many people have doubts about how to understand Trafodion transaction management. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful for you to answer the doubts about "how to understand Trafodion transaction management". Next, please follow the editor to study!

The original meaning of the word Trafodion is "transaction", which shows how much importance the project team attaches to transaction processing.

Transactions are mainly used to prevent and process inconsistent errors in data. First of all, it is too difficult for the author to understand what is data consistency and give a specific definition. Let me give you an example. When the author was young, everyone knew the "four heavenly kings of Hong Kong". They were Andy Lau, Jacky Cheung, Liming and Aaron Kwok. I define these four names as "consistent", while "Liu Xueyou" or "Zhang Dehua" are not consistent data. Or the introduction is incomplete: for example, "the four heavenly kings of Hong Kong are: Andy Lau," and other heavenly kings will be unhappy. In addition, Andy Lau sang "forget the water", if the database query found that Andy Lau sang "kiss goodbye", it is also inconsistent data.

There are two main factors that cause data inconsistency, which are solved by two transaction techniques respectively. In Trafodion, one technology is logging; the other is multi-version concurrency control MVCC.

Journal

The first cause of data inconsistency is unpredictable failures. A failure can cause an abnormal interruption in the process of processing data, such as a power outage when the data is being written to disk, then the result is unknown and incomplete data may be written. For example, "Zhang Xueyou" should be written to cover the original "Liu Dehua", but only "Zhang" was written, and the remaining two bytes were not written in time, so it was still old data, so the disk data became "Zhang Dehua", resulting in data inconsistency. There has never been such a king. In other words, the names of four people are being written in sequence, the power is cut off, only Andy Lau is written, and the others do not write, which is also inconsistent, either all four or none of them, what does it mean to write one without the other, and look down on the other heavenly kings?

The way to handle such errors is the WAL log, write ahead log. That is, before each data is written, the changes are written to the log, and then the data is written. In this way, if an error occurs, you can recover by querying the log. For example, in the previous example, "Jacky Cheung" is written to the log before writing it to the data file. In this way, even if there is a failure, there will be no problem. If the failure occurs in the process of writing a log, then the data file is still "Andy Lau". If the failure occurs in the process of writing the data file, then although the data file is "Zhang Dehua", the correct data in the WAL log is "Zhang Xueyou". We will replay the log and overwrite "Zhang Dehua" with "Zhang Xueyou".

If you encounter a write interrupt error, you can use the log to undo all the operations you have done, such as writing to Andy Lau, and then make an error and query the log and find that Andy Lau has been written, then rollback is to delete "Andy Lau".

Through logging, the consistency of transactions is ensured in any error case, that is, An and D in ACID.

Traditional logging techniques include Redo,undo, redo/undo and so on. Like all other databases, Trafodion uses redo/undo logs.

Trafodion uses the WAL of HBase as the Redo log. When the region is restored, HBase is responsible for playing back the WAL for recovery.

In version R1.0, Trafodion caches write data in memory and writes the data in memory to disk only when it is submitted, so undo logs are not required. If the transaction needs to be rolled back, you only need to discard the data in memory.

Currently planned for version R1.2, Trafodion will use the latest technology called SSCC, and SSCC uses multiple versions of HBase to store the change history of data to provide repeated read isolation levels, which are also used as undo logs.

Concurrency Control method of MVCC

Another kind of error, which doesn't work on logs, is that multiple transactions write the same data concurrently, which is the same kind of problem that occurs when we modify shared variables in multithreaded programs. For example, we have two threads to write the database, both to update the number one of the song charts. Thread A writes one line of data: [singer= > Andy Lau, song= > forget water]; thread B writes another line of data [singer= > Jacky Cheung, song= > kiss goodbye]. An and B write together, one possible way to write is that A writes the singer and is interrupted by B, B writes the singer and song, A resumes execution and writes the song. In the end, it became [singer= > Jacky Cheung, song= > forget the water]. But Jacky Cheung never sang "forget Love".

The method of handling the second type of error is called concurrency control. There are usually two solutions, the first is the method commonly used by traditional databases, the lock manager, and the second is MVCC (multi-version concurrency control). Trafodion uses the MVCC approach and does not use lock managers. Therefore, there are no read-write blocking and deadlock problems caused by locks in Trafodion.

The basic idea of MVCC is the same as when people use SVN version control systems. Every programmer is a transaction, and the modified code is the table in the database. If you use sourcesafe, then it is similar to lock management. When Zhang San rewrites file A, no one else can read or write until Zhang San modifies the check in. After using SVN, if Zhang San is writing file A, he is writing a local version, and other people can read it at any time; other programmers can also modify file A. But you can't both commit. The person who commit first succeeds, and the person who commit later needs to do merge. If two people modify different rows (similar to two transactions modifying different data rows), they can generally commit,SVN and merge automatically. Otherwise, there will be conflicts, requiring a second commit person to manually merge, or simply give up (Abort). Trafodion's MVCC method is exactly the same as this process.

HBase itself provides multi-version services, so it is very suitable to adopt MVCC technology. Each write operation produces a new version of the data without overwriting the old version of the data. The read operation reads the correct version of the data, which is a historical snapshot of the data, based on the start time of the transaction. This avoids illusion and provides a guarantee of repeated read isolation level. The author is ill-informed and secretly thinks that MVCC was first adopted by Oracle, and Oracle engineers are often very proud of it because reading is not blocked by writes and provides a high degree of parallelism. Later, innodb copied all the implementation details of Oracle, making MySQL have similar capabilities; PostgreSql uses a different MVCC implementation method from Oracle/innoDb, and the implementation of Trafodion is more similar to Postgresql, that is, saving multi-version data to provide snapshot reading, rather than being used as undo logs, and then garbage collection on a regular basis to delete expired snapshots. While Oracle uses undo logs as historical snapshots, the former reads historical versions very fast because the data and snapshots are all together, but needs to process each piece of historical data and check their visibility, which is relatively inefficient. Oracle needs to read historical data in undo segment and rolls back the data using undo logs, which seems to be slow. But you don't need to think about GC, and it saves space, because undo space kills two birds with one stone. So the author thinks that the mode of Oracle/InnoDB is better, but it mainly depends on the specific application scenario. If the concurrency is high and the proportion of reading historical versions is higher than the frequency of reading recent data, then the mode of PostgreSQL/Trafodion should be better, because they are more efficient in reading snapshots.

However, the author's level of experience is very limited, talking about other products here is a bit bumpy, if you do not think so, please do not hesitate to comment.

In Commit, write-write conflict detection is carried out. If it is found that two concurrent transactions write a row of data at the same time, it is processed according to the principle of "first committer win". The first committed transaction succeeds, while the other transactions fail to roll back. After writing singer message Andy Lau, thread An and BMagee An are interrupted, and B writes Jacky Cheung to sing "kiss goodbye". This data does not overwrite the original data, but generates a new record (some people like to call it COW). This record marks it with thread B's transaction ID. At this time, thread An is dispatched back and continues to write. Similarly, Liu Dehua's singing "forget Water" written by A will not overwrite the original data, but will generate a new record and mark it with A's transaction ID. While at commit, Trafodion found that there were two new records for the same row of data, and according to the principle of first commit win, A won. So if thread A commits, it will succeed, while thread B will report an error and roll back. The final record of the database is Liu Dehua's song "forgetting Love". Is consistent.

The current version of Trafodion is R1.0, and a new concurrency control algorithm, SSCC, is enabled in future releases (planned for R1.2). SSCC is the research result of the Chinese Academy of Sciences. The Trafodion team and the Chinese Academy of Sciences work closely to launch this new concurrency control mechanism. SSCC provides a higher isolation level than SnapShot Isolation and highly efficient support for stateless write operations.

SSCC believes that there are two kinds of SQL write operations: stateful writes, such as a=a+1, are stateful; stateless writes, such as delete, or overwrite old data, which have nothing to do with historical status.

Stateless writing is very common in web applications, such as Google's index generating. Using this mechanism, Trafodion can efficiently provide strong support for related web applications.

The author will also try to introduce SSCC, an innovative concurrency control algorithm, in subsequent blogs.

Concurrency control ensures the I in ACID.

Finally, there is ACID C which is not mentioned, which means "consistent", but this C is not guaranteed by the transaction processor. The Consistency of ACID is guaranteed by database constraints. Therefore, the author does not like to use ACID to describe transactions. But this is a classic, and I have to mention it.

Two-phase submission

For stand-alone databases, such as MySQL,Oracle (not RAC), SQL Server,PostgreSql (not xl version), etc., the above two technologies can fully implement ACID transactions. But Trafodion is a distributed database, operations are performed on different nodes of the cluster, for example, two write operations may be run by two different ESP on different nodes, so Trafodion also needs to handle distributed transactions.

The transaction processor of Trafodion divides the responsibility of concurrency control into two parts: TM (Transaction Manager) is responsible for the distributed concurrency control of the whole transaction to ensure the consistency of the transaction itself; RM (Resource Manager) is responsible for the concurrency control of data access to ensure the consistency of data.

Specifically, RM uses the aforementioned MVCC technology to provide Snapshot Isolation-level data consistency.

In Trafodion, the Region of each HBase is a RM. A transaction is likely to operate on multiple Region, for example, the transaction needs to update two rows of data, and each belongs to two different Region. In this case, more than one RM is involved in a transaction. HBase's Region is likely to run on different physical nodes, so this is a distributed transaction. Distributed transaction ensures the consistency of the whole transaction by TM.

TM uses a two-phase commit protocol to ensure the consistency of distributed transactions. Two-phase submission is a very simple and intuitive agreement, similar to a meeting arranged by a company secretary that several people must attend. The first stage: the secretary sends a letter to all participants: "the meeting will be held at 3pm on Monday, OK?". If all people reply "OK", it is finally decided that the meeting will proceed to the second stage; if someone answers no, or if someone does not answer, the meeting will not proceed to the second stage. The second stage will be dealt with differently according to the results of the first stage. If it is a meeting, the secretary will send a letter to everyone: 'make sure to have a meeting at 3 p.m. Monday'; if there is no meeting, the secretary will send a letter to everyone:'no meeting'.

Each transaction is initiated by a TM, and at the beginning of the transaction, TM creates an ID for the transaction. Since then, every DML operation in the transaction has this ID, which is used to determine the data version read within the RM.

At the time of submission, TM first performs the first phase of prepare commit, sending prepare messages to each RM. RM detects write-write conflicts when it receives a prepare message, and returns a conflict if there is a conflict, otherwise it returns a successful submission. After receiving replies from all RM, TM judges that if all RM commits successfully, the transaction succeeds, otherwise the transaction fails. In the second phase, if the transaction is successful, TM sends commit messages to all RM. After each RM receives the commit message, all operations of the transaction are written to the physical disk to complete the transaction. If any RM returns a conflict, or any RM timeout does not respond, TM considers the transaction failed and sends abort messages to all RM in the second phase; each RM performs a transaction rollback operation after receiving the abort message.

DDL transaction

Starting with version R1.1, Trafodion also supports DDL transactions, such as creating two tables, the first succeeding, the second failing, and neither table exists after the rollback.

At this point, the study on "how to understand Trafodion transaction management" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.