Shulou(Shulou.com)05/31 Report--

This article introduces how to realize the transaction in MongoDB 4.0. the content is very detailed. Interested friends can use it for reference. I hope it will be helpful to you.

The transaction function introduced by MongoDB 4.0supports multi-document ACID features, such as using mongo shell for transaction operations.

> s = db.getMongo () .startSession () session {"id": UUID ("3bf55e90-5e88-44aa-a59e-a30f777f1d89")} > s.startTransaction () > db.coll01.insert ({x: 1, y: 1}) WriteResult ({"nInserted": 1}) > db.coll02.insert ({x: 1, y: 1}) WriteResult ({"nInserted": 1}) > s.commitTransaction () (or s.abortTransaction () rollback transaction)

Driver, other languages that support MongoDB 4.0, also encapsulate transaction-related interfaces. Users need to create a Session, then open the transaction on Session and commit the transaction. For example

Python version

With client.start_session () as s: s.start_transaction () collection_one.insert_one (doc_one, session=s) collection_two.insert_one (doc_two, session=s) s.commit_transaction () java version try (ClientSession clientSession = client.startSession ()) {clientSession.startTransaction (); collection.insertOne (clientSession, docOne); collection.insertOne (clientSession, docTwo); clientSession.commitTransaction ();} Session

Session is a concept introduced by MongoDB version 3.6, and this feature is mainly introduced to prepare for the implementation of multiple document transactions. Session is essentially a "context".

In previous versions, MongoDB only managed the context of a single operation, and the mongod service process received a request, created a context for the request (corresponding to OperationContext in the source code), and then used this context throughout the service of the request, including the statistics of the time consumed by the request, the lock resources consumed by the request, the storage snapshot used by the request, and so on. With Session, you can make multiple requests share a context and correlate multiple requests, thus being able to support multiple document transactions.

Each Session contains a unique identity lsid. In version 4.0, users can specify additional extension fields for each request, mainly including:

Lsid: the ID of the Session to which the request resides, also known as logic session id

TxnNmuber: the transaction number corresponding to the request, which must be monotonously incremented within a Session

StmtIds: operate ID corresponding to each operation in the request (take insert as an example, an insert command can insert multiple documents)

In fact, users do not need to understand these details when using transactions. MongoDB Driver will automatically process them, and Driver will assign lsid when creating Session. For all operations in this Session, Driver will automatically add lsid to these operations, and if it is a transaction, it will automatically bring txnNumber.

It is worth mentioning that Session lsid can be allocated by the server side by calling the startSession command, or by the client itself, which can save network overhead; as for the identification of the transaction, MongoDB does not provide a separate command for startTransaction, and the txnNumber is allocated directly by the Driver. Driver only needs to ensure that the txnNumber is incremented within a Session, and the server side will actively start a new transaction when it receives a new transaction request.

When MongoDB is in startSession, you can specify a series of options to control the access behavior of Session, mainly including:

CausalConsistency: whether to provide the semantics of causal consistency. If set to true, MongoDB will guarantee the semantics of "read your own write" no matter which node it is read from. Reference causal consistency

ReadConcern: analysis with reference to MongoDB readConcern principle

WriteConcern: analysis with reference to MongoDB writeConcern principle

ReadPreference: set the rules for selecting nodes when reading. Refer to read preference.

RetryWrites: if set to true, MongoDB will automatically retry the scenario where the re-election occurs in the replication set scenario. Refer to retryable write

ACIDAtomic

For multi-document transaction operations, MongoDB provides the atomic semantic guarantee of "All or nothing".

Consistency

It's too hard to explain, and there are databases that abandon the Consistency feature?

Isolation

MongoDB provides the snapshot isolation level, creates a WiredTiger snapshot at the beginning of the transaction, and then uses this snapshot to provide transaction reads throughout the transaction.

Durability

When a transaction uses WriteConcern {j: ture}, MongoDB will ensure that the transaction log is committed before it returns, and even if crash,MongoDB occurs, it can be recovered according to the transaction log; and if the {j: true} level is not specified, even if the transaction is successfully committed, the transaction may be rolled back after crash recovery.

Transaction and replication

In the replication set configuration, when the entire transaction of MongoDB commits, an oplog will be recorded (oplog is a normal document, so the modification of the transaction in the current version cannot exceed the 16MB limit of the document size), including all the operations in the transaction. The slave node pulls the oplog and replays the transaction operation locally.

Transaction oplog example, which contains the lsid,txnNumber of the transaction operation and all operation logs within the transaction (applyOps field)

"ts": Timestamp (1530696933, 1), "t": NumberLong (1), "h": NumberLong ("4217817601701821530"), "v": 2, "op": "c", "ns": "admin.$cmd", "wall": ISODate ("2018-07-04T09:35:33.549Z"), "lsid": {"id": UUID ("e675c046-d70b-44c2-ad8d-3f34f2019a7e"), "uid": BinData (0) "47DEQpj8HBSABSARkm5NMpJWZG3hSuFU =")}, "txnNumber": NumberLong (0), "stmtId": 0, "prevOpTime": {"ts": Timestamp (0,0), "t": NumberLong (- 1)}, "o": {"applyOps": [{"op": "I", "ns": "test.coll2", "ui": UUID ("a49ccd80-6cfc-4896-9740-c5bff41e7cce") "o": {"_ id": ObjectId ("5b3c94d4624d615ede6097ae"), "x": 20000}}, {"op": "I", "ns": "test.coll3", "ui": UUID ("31d7ae62-fe78-44f5-ba06-595ae3b871fc"), "o": {"_ id": ObjectId ("5b3c94d9624d615ede6097af"), "x": 20000}}]}

The whole playback process is as follows:

Get the current Batch (the backend keeps pulling oplog and putting it into Batch)

Set the OplogTruncateAfterPoint timestamp to the first oplog timestamp in Batch (stored in the local.replset.oplogTruncateAfterPoint collection)

Write all oplog to local.oplog.rs collections in Batch. According to the number of oplog entries, if the number is large, writes will be accelerated concurrently.

Clean up the OplogTruncateAfterPoint to identify the complete success of the oplog write; if you find that the oplogTruncateAfterPoint is set when you restart and recover the crash before the completion of this step, the oplog will be truncated to the timestamp to restore to a consistent state point.

The oplog is divided into multiple threads for concurrent playback. In order to improve the concurrency efficiency, all the modification operations contained in the oplog generated by the transaction, like a normal oplog with a single operation, are divided into multiple threads according to the document ID.

The update ApplyThrough timestamp is the last oplog timestamp in the Batch, indicating that after the next reboot, the synchronization will be resynchronized from this location. If the previous step fails, the oplog will be pulled from the last value of the ApplyThrough (the last oplog of the previous Batch) when the restart resumes.

Update the oplog visible timestamp. If other nodes synchronize from the slave node, you can read this part of the newly written oplog.

Update the local Snapshot (timestamp) and the new writes will be visible to the user.

Transaction and storage engine transaction timing is unified

WiredTiger has supported transactions for a long time. In version 3.x, MongoDB used WiredTiger transactions to ensure the atomicity of changes to data, index and oplog. But in fact, MongoDB provides the transaction interface after several versions of iteration, and the core difficulty is the timing problem.

MongoDB identifies the global order through the oplog timestamp, while WiredTiger identifies the global order through the internal transaction ID. In implementation, there is no correlation between the two. This causes the transaction commit order seen by MongoDB to be inconsistent with the transaction commit order seen by WiredTiger in the case of concurrency.

To solve this problem, WiredTier 3.0 introduces a transaction timestamp (transaction timestamp) mechanism, where applications can explicitly assign commit timestmap to WiredTiger transactions through the WT_SESSION::timestamp_transaction interface, and then read "as of" a timestamp. With the read "as of" a timestamp feature, when the oplog is replayed, the read on the standby node will no longer conflict with the playback oplog, and the read request will not be blocked by replaying the oplog, which is a huge improvement in version 4.0.

/ * * _ _ wt_txn_visible-- * Can the current transaction see the given ID / timestamp? * / static inline bool__wt_txn_visible (WT_SESSION_IMPL * session, uint64_t id, const wt_timestamp_t * timestamp) {if (! _ txn_visible_id (session, id)) return (false); / * Transactions read their writes, regardless of timestamps. * / if (F_ISSET (& session- > txn, WT_TXN_HAS_ID) & & id = = session- > txn.id) return (true); # ifdef HAVE_TIMESTAMPS {WT_TXN * txn = & session- > txn; / * Timestamp check. * / if (! F_ISSET (txn, WT_TXN_HAS_TS_READ) | | timestamp = = NULL) return (true); return (_ _ wt_timestamp_cmp (timestamp, & txn- > read_timestamp)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.