Oplog & ACID Transactions

Oplog & ACID Transactions

While MongoDB has always guaranteed atomicity at the single-document level, modern applications often require cross-collection or multi-document consistency. This is achieved through Multi-Document ACID Transactions.

🏗️ 1. The Oplog: Foundation of Consistency

The Oplog (Operation Log) is a capped collection in the local database that stores all write operations applied to a Replica Set’s Primary.

• Idempotency: Every operation in the Oplog is idempotent, meaning applying it multiple times results in the same state.
• Tailing the Oplog: Secondary nodes continuously poll the Primary’s Oplog to replicate changes.
• Retention: Because it is a capped collection, the Oplog has a fixed size. If it overflows, Secondaries that have fallen too far behind will need a full resync.

Tip

You can check your Oplog size and window (time before it overflows) using rs.printReplicationInfo().

---

🚀 2. Multi-Document ACID Transactions

Since version 4.0, MongoDB supports full ACID (Atomicity, Consistency, Isolation, Durability) transactions across multiple collections.

Core Features

• All-or-Nothing: All operations within a transaction either succeed and are committed together, or all are aborted.
• Isolation: Operations in a transaction are not visible to other clients until they are committed (Snapshot Isolation).

Technical Requirements

• Replica Set: Transactions require a Replica Set (or a Sharded Cluster).
• Driver Support: You must use a modern driver (e.g., Pymongo 3.7+).

---

⚡ 3. Implementation Examples

Pymongo Example: Banking Transaction

from pymongo import MongoClient

client = MongoClient('mongodb://localhost:27017/?replicaSet=rs0')
db = client['bank']

def transfer_funds(session, from_id, to_id, amount):
    accounts = db['accounts']
    
    # Step 1: Deduct from A
    accounts.update_one(
        {"_id": from_id, "balance": {"$gte": amount}},
        {"$inc": {"balance": -amount}},
        session=session
    )
    
    # Step 2: Add to B
    accounts.update_one(
        {"_id": to_id},
        {"$inc": {"balance": amount}},
        session=session
    )

with client.start_session() as session:
    # Use with_transaction to automatically handle retries
    session.with_transaction(
        lambda s: transfer_funds(s, "account_A", "account_B", 500)
    )

Mongo Shell Example: Manual Transaction

const session = db.getMongo().startSession();
session.startTransaction();

try {
    db.inventory.updateOne(
        { _id: "sku_123" }, 
        { $inc: { quantity: -1 } }, 
        { session }
    );
    
    db.orders.insertOne(
        { order_id: "order_abc", item: "sku_123" }, 
        { session }
    );
    
    session.commitTransaction();
} catch (error) {
    session.abortTransaction();
} finally {
    session.endSession();
}

---

🛡️ 4. Performance and Constraints

Caution

While transactions are powerful, they should not be your default choice. MongoDB’s “Document-Oriented” model encourages embedding related data into a single document, which provides single-document atomicity without the overhead of transactions.

1. Locking: Transactions hold locks on modified documents until they are committed or aborted, which can impact performance.
2. Time Limits: By default, transactions must complete within 60 seconds (transactionLifetimeLimitSeconds).
3. Execution Cost: Transactions incur additional overhead in the WiredTiger engine to maintain multiple snapshots of the data.

💡 Best Practices

1. Design for Single-Document Atomicity: Always prefer Embedded Documents over transactions when possible.
2. Keep Transactions Short: Avoid long-running logic or external API calls inside a transaction block.
3. Handle Transient Errors: Some transaction failures (like “Write Conflict”) are transient and should be retried automatically (as shown in the Pymongo example).