Shulou(Shulou.com)06/01 Report--

Transaction Transactions

[TOC]

Preface of the translator:

Data model definition comparison:

MySQLMongoDB line document table collection library group replication replication set

Translation convention of proper nouns:

Chinese original text read consistency readConcern write consistency writeConcern introduction:

Added in version 4.0

New in version 4.0

In MongoDB, operations on a single document are atomic. Because MongoDB allows embedding subdocuments and arrays to represent relationships in a single document to replace join operations across documents and collections, this compromise indirectly implements the characteristics of multi-document transactions in many scenarios.

In MongoDB, an operation on a single document is atomic. Because you can use embedded documents and arrays to capture relationships between data in a single document structure instead of normalizing across multiple documents and collections, this single-document atomicity obviates the need for multi-document transactions for many practical use cases.

However, this method is stretched in the face of simultaneous updating of multiple documents or consistent reading of multiple documents. The new version of MongoDB provides a multi-document transaction feature for replication sets. It can satisfy the transactionality among multiple operations, documents, collections and databases, the characteristics of transactions: several operations in a transaction are either completed or rolled back, the atomicity of operations A, the consistency of data updates C. When the transaction commits, all data changes are permanently saved D. The data changes of a transaction will not be obtained externally until it is committed.

However, for situations that require atomicity for updates to multiple documents or consistency between reads to multiple documents, MongoDB provides the ability to perform multi-document transactions against replica sets. Multi-document transactions can be used across multiple operations, collections, databases, and documents. Multi-document transactions provide an "all-or-nothing" proposition. When a transaction commits, all data changes made in the transaction are saved. If any operation in the transaction fails, the transaction aborts and all data changes made in the transaction are discarded without ever becoming visible. Until a transaction commits, no write operations in the transaction are visible outside the transaction.

Note:

In most scenarios, the performance loss of multi-document transactions is more serious than that of single-document data changes. And don't try to relax the requirements for data structure design just because it supports multi-document transactions. No matter what kind of database, clever table design is always more efficient than headiron forcing the use of unnecessary low-level data associations.

In general, due to the reduced need for multi-document transactions, non-stereotyped (that is, embedded subdocuments and arrays) data models will still play an important role.

In most cases, multi-document transaction incurs a greater performance cost over single document writes, and the availability of multi-document transaction should not be a replacement for effective schema design. For many scenarios, the denormalized data model (embedded documents and arrays) will continue to be optimal for your data and use cases. That is, for many scenarios, modeling your data appropriately will minimize the need for multi-document transactions.

Transaction and replication set Transactions and Replica Sets

Multi-document transactions only support replication sets in version 4.0, and transactional support for sharding clusters is planned to be implemented in version 4.2.

Multi-document transactions are available for replica sets only. Transactions for sharded clusters are scheduled for MongoDB 4.2 [1].

Disclaimer, no translation of The development, release, and timing of any features or functionality described for our products remains at our sole discretion. This information is merely intended to outline our general product direction and it should not be relied on in making a purchasing decision nor is this a commitment, promise or legal obligation to deliver any material, code, or functionality. New feature version backward compatibility Feature Compatibility Version (FCV)

To use the multi-document transaction feature, the featureCompatibilityVersion value must be set to more than 4. 0. (when enabled, the existing data cannot be started with mongod below version 4.0)

The featureCompatibilityVersion (fCV) of all members of the replica set must be 4.0 or greater. To check the fCV for a member, connect to the member and run the following command:

Db.adminCommand ({getParameter: 1, featureCompatibilityVersion: 1})

For more information on fCV, see setFeatureCompatibilityVersion.

Storage engine Storage Engines

The multi-document transaction feature only supports WiredTiger storage engine

Multi-document transactions are only available for deployments that use WiredTiger storage engine.

Multi-document transactions are not available for deployments that use in-memory storage engine or the deprecated MMAPv1 storage engine.

Transaction and document operation Transactions and Operations

Note:

For transactions:

​ can perform CURD operations on any table that crosses the library

You can specify read/write (CRUD) operations on existing collections. The collections can be in different databases.

Tables in the config,admin,local library do not support multi-document transactions

You cannot read/write to collections in the config, admin, or local databases.

System.* tables in each library do not support multi-document transactions.

You cannot write to system.* collections.

A query plan that returns the current operation cannot be made in the transaction of the current session

You cannot return the supported operation's query plan (i.e. Explain).

Cursors created outside the transaction cannot perform getMore operations in the transaction

For cursors created outside of transactions, you cannot call getMore inside a transaction.

Cursors created in a transaction cannot perform getMore operations outside the transaction

For cursors created in a transaction, you cannot call getMore outside the transaction.

Operations such as creating or deleting collections, adding indexes, and other operations to update database metadata cannot be performed in a multi-document transaction. Furthermore, operations that may implicitly create collections are also prohibited in multi-document transactions.

Operations that affect the database catalog, such as creating or dropping a collection or an index, are not allowed in multi-document transactions. For example, a multi-document transaction cannot include an insert operation that would result in the creation of a new collection. See Restricted Operations.

Tip:

If a transaction that will operate on the collection is opened immediately after the collection is created or deleted, the operation of creating or deleting the collection requires appending the parameter of write consistency (writeConcern) spread to most nodes to ensure that the transaction can successfully acquire the locks associated with the collection.

(note: this may be a bit of a mouthful. In fact, it is considered from the consistency of the cluster, which is related to the read consistency of the transaction readConcern)

When creating or dropping a collection immediately before starting a transaction, if the collection is accessed within the transaction, issue the create or drop operation with write concern "majority" to ensure that the transaction can acquire the required locks.

Commands and methods that currently support multi-document transactions

For multi-document transactions:

MethodCommandNotedb.collection.aggregate () aggregateExcluding the following stages:$collStats$currentOp$indexStats$listLocalSessions$listSessions$outdb.collection.distinct () distinctdb.collection.find () findgeoSearchdb.collection.deleteMany () db.collection.deleteOne () db.collection.remove () deletedb.collection.findOneAndDelete () db.collection.findOneAndReplace () db.collection.findOneAndUpdate () findAndModifyFor upsert, only when run against an existing collection.db.collection.insertMany () db.collection.insertOne () db.collection.insert () insertOnly when run against an existing collection.db.collection.save () If an insert Only when run against an existing collection.db.collection.updateOne () db.collection.updateMany () db.collection.replaceOne () db.collection.update () updateFor upsert, only when run against an existing collection.db.collection.bulkWrite () Various Bulk Operation MethodsFor insert operations, only when run against an existing collection. Count operation Count Operation

If you need to count in a transaction, you need to use the $count operator or combine $group with $sum

To perform a count operation within a transaction, use the $count aggregation stage or the $group (with a$sum expression) aggregation stage.

The version 4.0 compatible MongoDB driver contains a countDocuments (filter,options) counting interface packaged by $group and $sum at the collection level.

MongoDB drivers compatible with the 4.0features provide a collection-level API countDocuments (filter,options) as a helper method that uses the $group with a $sum expression to perform a count.

Get the operation Informational Operations related to the parameter and information

Commands that get parameters or information, such as isMaster, buildInfo, connectionStatus (and their associated help commands), can be used in a transaction, but not as the first action in a transaction.

Informational commands, such as isMaster, buildInfo, connectionStatus (and their helper methods) are allowed in transactions; however, they cannot be the first operation in the transaction.

Restricted operation Restricted Operations

The following actions are not allowed in multi-document transactions:

The following operations are not allowed in multi-document transactions:

​ may affect the operation of database meta-information, such as creating or deleting collections and adding indexes. (note: it's actually a DDL operation.)

Operations that affect the database catalog, such as creating or dropping a collection or an index. For example, a multi-document transaction cannot include an insert operation that would result in the creation of a new collection.

ListCollections and listIndexes and their related help commands are prohibited above. (note: this is a bit unexpected)

The listCollections and listIndexes commands and their helper methods are also excluded.

Operations that do not add, delete, modify and query parameters and information, such as createUser, getParameter, count and their help commands, etc.

Non-CRUD and non-informational operations, such as createUser, getParameter, count, etc. And their helpers. Transaction and database security related Transactions and Security

If ​ enables authentication and wants to use multi-document transactions, it must have permission to operate transactions.

If running with access control, you must have privileges for the operations in the transaction. [2]

If the audit is turned on, the rolled-back transaction will still be recorded.

If running with auditing, operations in an aborted transaction are still audited. If additional authentication is used, the user name cannot exceed 10KIf using $external authentication users (i.e. Kerberos, LDAP, x.509 users), the usernames cannot be greater than 10k bytes. Transaction and session related Transactions and Sessions

Transactions are closely related to sessions, and a session can only be concurrently with at most one active transaction during its life cycle.

Transations are associated with a session. That is, you start a transaction for a session. At any given time, you can have at most one open transaction for a session.

Key IMPORTANT

When using a driver to connect (a non-mongo shell connection), the session must be mapped to each transaction. (note: multiple connection session classes should be opened for multiple transactions). If the session exits while the transaction is in progress, the transaction will be rolled back

When using the drivers, you must pass the session to each operation in the transaction.

If a session ends and it has an open transaction, the transaction aborts.

Compatible Transactions and MongoDB Drivers of transaction and driver

The following are the minimum driver version numbers for each language that support multi-document transactions:

Clients require MongoDB drivers updated for MongoDB 4.0.

JavaPythonC#NodeRubyPerlPHPCScala3.8.03.7.0C 1.11.02.73.1.02.6.02.0.01.5.02.4.0

Key point: IMPORTANT

If you want to combine read and write operations in a transaction, you must also correspond each operation to the session.

To associate read and write operations with a transaction, you must pass the session to each operation in the transaction. For examples, see Transactions and Retryable Writes.

Transaction and mongo shell tool Transactions and the mongo Shell

The following mongo shell methods can be used to manipulate transactions:

The following mongo shell methods are available for transactions:

Session.startTransaction () Session.commitTransaction () Session.abortTransaction () transactions and rewrite Transactions and Retryable Writes

HIGHLY AVAILABLE APPLICATIONS for highly available applications

Whether it is MongoDB or relational database, the application connected to it should try to deal with the errors in the transaction commit process and design the redo logic of the transaction.

Regardless of the database system, whether MongoDB or relational databases, applications should take measures to handle errors during transaction commits and incorporate retry logic for transactions.

Transaction redo Retry Transaction

Independent writes in a transaction cannot always be redone, whether retryWrites is set to true or not

The individual write operations inside the transaction are not retryable, regardless of whether retryWrites is set to true.

If an error occurs during the execution of the operation, an array field named errorLabels is returned. If the error is temporary, the errorLabels contains a "TransientTransactionError" element, indicating that the entire transaction can be redone.

If an operation encounters an error, the returned error may have an errorLabels array field. If the error is a transient error, the errorLabels array field contains "TransientTransactionError" as an element and the transaction as a whole can be retried.

For example, the following JAVA function implements the execution of the transaction and redoes the errors that contain "TransientTransactionError".

(note: the official documentation has routines in several other languages. Here we choose the JAVA with the largest number of users.)

For example, the following helper runs a function and retries the function if a "TransientTransactionError" is encountered:

Void runTransactionWithRetry (Runnable transactional) {while (true) {try {transactional.run (); break;} catch (MongoException e) {System.out.println ("Transaction aborted. Caught exception during transaction. "); if (e.hasErrorLabel (MongoException.TRANSIENT_TRANSACTION_ERROR_LABEL)) {System.out.println (" TransientTransactionError, aborting transaction and retrying... "); continue;} else {throw e;} submit operation can be redone Retry Commit Operation

The commit operation itself can be redone. If the transaction encounters an error during commit, the driver automatically ignores the setting of retryWrites for a retry.

The commit operations are retryable write operations. If the commit operation operation encounters an error, MongoDB drivers retry the operation a single time regardless of whether retryWrites is set to true.

If an error occurs during the commit of the transaction, MongoDB returns an array field containing errorLabels. If the error is temporary, the errorLabels field contains a "UnknownTransactionCommitResult" element indicating that the transaction can be resubmitted

If the commit operation encounters an error, MongoDB returns an error with an errorLabels array field. If the error is a transient commit error, the errorLabels array field contains "UnknownTransactionCommitResult" as an element and the commit operation can be retried.

Although the MongoDB driver provides a transaction resubmission mechanism, the application should design a method to handle errors that occur during transaction commit.

In addition to the single retry behavior provided by the MongoDB drivers, applications should take measures to handle "UnknownTransactionCommitResult" errors during transaction commits.

Still take JAVA as an example:

Void commitWithRetry (ClientSession clientSession) {while (true) {try {clientSession.commitTransaction (); System.out.println ("Transaction committed"); break } catch (MongoException e) {/ / can retry commit if (e.hasErrorLabel (MongoException.UNKNOWN_TRANSACTION_COMMIT_RESULT_LABEL)) {System.out.println ("UnknownTransactionCommitResult, retrying commit operation..."); continue;} else {System.out.println ("Exception during commit..."); throw e } transaction and commit operation redo Retry Transaction and Commit Operation

Combining the above two logic, look at the following routine: (JAVA)

Void runTransactionWithRetry (Runnable transactional) {while (true) {try {transactional.run (); break;} catch (MongoException e) {System.out.println ("Transaction aborted. Caught exception during transaction. "); if (e.hasErrorLabel (MongoException.TRANSIENT_TRANSACTION_ERROR_LABEL)) {System.out.println (" TransientTransactionError, aborting transaction and retrying... "); continue;} else {throw e } void commitWithRetry (ClientSession clientSession) {while (true) {try {clientSession.commitTransaction (); System.out.println ("Transaction committed"); break } catch (MongoException e) {/ / can retry commit if (e.hasErrorLabel (MongoException.UNKNOWN_TRANSACTION_COMMIT_RESULT_LABEL)) {System.out.println ("UnknownTransactionCommitResult, retrying commit operation..."); continue;} else {System.out.println ("Exception during commit..."); throw e } void updateEmployeeInfo () {MongoCollection employeesCollection = client.getDatabase ("hr"). GetCollection ("employees"); MongoCollection eventsCollection = client.getDatabase ("hr"). GetCollection ("events"); try (ClientSession clientSession = client.startSession ()) {clientSession.startTransaction () EmployeesCollection.updateOne (clientSession, Filters.eq ("employee", 3), Updates.set ("status", "Inactive"); eventsCollection.insertOne (clientSession, new Document ("employee", 3). Append ("status", new Document ("new", "Inactive"). Append ("old", "Active")); commitWithRetry (clientSession) }} void updateEmployeeInfoWithRetry () {runTransactionWithRetry (this::updateEmployeeInfo);} atomicity Atomicity

Multiple document transactions satisfy the atomicity of ACID

Multi-document transactions are atomic:

​ when a transaction is committed, all changes within the transaction are saved and can be read by transactions of other sessions.

When a transaction commits, all data changes made in the transaction are saved and visible outside the transaction. Until a transaction commits, the data changes made in the transaction are not visible outside the transaction.

When the transaction is rolled back, all changes within the transaction are cleanly discarded. For example, after any operation in a transaction fails, the transaction rolls back and all changes in the current transaction are discarded and are not known to other sessions.

When a transaction aborts, all data changes made in the transaction are discarded without ever becoming visible. For example, if any operation in the transaction fails, the transaction aborts and all data changes made in the transaction are discarded without ever becoming visible. Read-oriented Read Preference

Read operations in a multi-document transaction must use primary bias, that is, from the primary instance of the replication set

Multi-document transactions that contain read operations must use read preference primary.

All operations in a given transaction must route to the same member.

Transaction parameter options Transaction Options (Read Concern/Write Concern) session.startTransaction ({readConcern: {level:}, writeConcern: {w:, j:, wtimeout:}); read consistency Read Concern

Multi-document transactions support three levels of read consistency: "snapshot", "local", and "majority", that is, snapshot, native, and majority

Multi-document transactions support read concern "snapshot", "local", and "majority":

​ is for native and majority read consistency levels, and MongoDB sometimes replaces them with stronger levels. (note: why? )

For "local" and "majority" read concern, MongoDB may sometimes substitute a stronger read concern.

For the majority read consistency level, if a transaction commits at this level, the transaction operation reads the data that has been committed by the majority in the replica set.

For "majority" read concern, if the transaction commits with write concern "majority", transaction operations are guaranteed to have read majority-committed data. Otherwise, the "majority" read concern provides no guarantees that read operations read majority-committed data.

Read consistency for snapshot level, if. [is there a mistake in the official document here], if the transaction is committed with snapshot-level read consistency, you can ensure that the operations in the transaction read the majority transaction snapshot at the beginning of the transaction.

For "snapshot" read concern, if the transaction commits with write concern "majority", the transaction operations are guaranteed to have read from a snapshot of majority committed data. Otherwise, the "snapshot" read concern provides no guarantee that read operations used a snapshot of majority-committed data.

If you set read consistency for the current transaction instead of setting consistency for individual operations in the transaction, the consistency covers all transactions in the transaction. And in a transaction, the consistency of transaction granularity covers read consistency at the set level, the database level, and the client level.

You set the read concern at the transaction level, not at the individual operation level. The operations in the transaction will use the transaction-level read concern. Any read concern set at the collection and database level is ignored inside the transaction. If the transaction-level read concern is explicitly specified, the client level read concern is also ignored inside the transaction.

Read consistency can be set at the beginning of the transaction.

You can set the transaction read concern at the transaction start.

If the read consistency setting at the transaction level is set by default, the transaction inherits the read consistency at the session level, and if the read consistency is not set at the session level, it continues to inherit the read consistency set up the client / client level.

If unspecified at the transaction start, transactions use the session-level read concern or, if that is unset, the client-level read concern.

Write consistency Write Concern

If you set write consistency for the current transaction instead of setting consistency for separate operations in the transaction. Then when committing, the transaction uses its own consistency to commit the write operation. Write consistency is ignored by individual operations in the transaction, please do not set it in vain. (note: the original text of this paragraph is ambiguous. It is considered that the original text of this paragraph is different from the original text of the previous paragraph in order to distinguish it from the situation in the previous paragraph.)

You set the write concern at the transaction level, not at the individual operation level. At the time of the commit, transactions use the transaction level write concern to commit the write operations. Individual operations inside the transaction ignore write concerns. Do not explicitly set the write concern for the individual write operations inside transactions.

Read consistency can be set at the beginning of the transaction.

You can set the write concern for the transaction commit at the transaction start.

If ​ defaults to the transaction-level write consistency setting, the transaction inherits session-level write consistency when committing, and if the session level is set by default, it continues to inherit client-level write consistency upward.

If unspecified at the transaction start, transactions use the session-level write concern for the commit or, if that is unset, the client-level write concern.

Write consistency setting to w: 0 is not supported

Write concern w: 0 is not supported for transactions.

If w: 1 is used for commit, only the native oK, that is, PrimaryOK, will be confirmed when the transaction is committed. If the primary node dies and the transaction is not transferred to the secondary node at that time. Well, congratulations, there is a good chance that the transaction will be rolled back. It will cause data inconsistency and lack of data between the database side and the application side.

If you commit using w: 1 write concern, your transaction can be rolled back if there is a failover.

If the transaction commits using majority write consistency, and snapshot-level read consistency is specified. Then the transaction operation ensures that it is read from the snapshot data that the majority has committed at the beginning of the transaction. (note: the meaning of this paragraph is unclear. What is the official document expressing? is it wrong?)

If the transaction commits with write concern "majority" and has specified read concern "snapshot" read concern, transaction operations are guaranteed to have read from a snapshot of majority-committed data. Otherwise, the "snapshot" read concern provides no guarantees that read operations used a snapshot of majority-committed data.

If the transaction commits using majority write consistency and the read consistency at the majority level is specified, the operations in the transaction are guaranteed to be read from the data committed by the majority. (note: is this the wrong copy and paste in the editing of the official document and put the consistency of the above reading over)

If the transaction commits with write concern "majority" and has specified read concern "majority" read concern, transaction operations are guaranteed to have read majority-committed data. Otherwise, the "majority" read concern provides no guarantees that read operations read majority-committed data. Transaction and lock Transactions and Locks

When an operation in a transaction acquires a lock on the data it needs, the default wait timeout is 5 milliseconds, after which the current transaction is rolled back. (note: this is too short, fortunately it can be set manually)

By default, transactions waits 5 milliseconds to acquire locks required by the operations in the transaction. If the transaction cannot acquire its required locks within the 5 milliseconds, the transaction aborts.

Tip:

If a transaction that will operate on the collection is opened immediately after the collection is created or deleted, the operation of creating or deleting the collection requires appending the parameter of write consistency (writeConcern) spread to most nodes to ensure that the transaction can successfully acquire the locks associated with the collection.

(note: this may be a bit of a mouthful. In fact, it is considered from the consistency of the cluster, which is related to the read consistency of the transaction readConcern)

When creating or dropping a collection immediately before starting a transaction, if the collection is accessed within the transaction, issue the create or drop operation with write concern "majority" to ensure that the transaction can acquire the required locks.

You can use the maxTransactionLockRequestTimeoutMillis parameter to set the lock wait timeout length.

You can use the maxTransactionLockRequestTimeoutMillis parameter to adjust how long transactions wait to acquire locks.

You can also set maxTransactionLockRequestTimeoutMillis to-1 to disable the lock waiting for a timeout rollback, and the transaction waits for the lock it needs to be released by another transaction or session.

You can also use operation-specific timeout by setting maxTransactionLockRequestTimeoutMillis to-1.

Transactions release all locks upon abort or commit.

Original link: MongoDB document:Transactions

Translator: Zhang Ruizhi

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.