Transactions

❗️

This is a legacy Apache Ignite documentation

The new documentation is hosted here: https://ignite.apache.org/docs/latest/

Atomicity Mode

Ignite supports several modes for cache operations, two transactional modes and an atomic mode. The atomic mode supports multiple atomic operations, one at a time. In the transactional modes, you can group multiple cache operations, on one or more keys, in to a single logical operation, known as a transaction. These operations will be executed without any other interleaved operations on the specified keys, and will either all succeed or all fail. There is no partial execution of the operations.

Atomicity mode is defined in CacheAtomicityMode enum (TRANSACTIONAL, TRANSACTIONAL_SNAPSHOT, ATOMIC) and can be configured via the atomicityMode property of CacheConfiguration.

The TRANSACTIONAL and TRANSACTIONAL_SNAPSHOT modes enable fully ACID-compliant transactions.

The TRANSACTIONAL mode only supports key-value transactions executed via Java API. Transactions in this mode can have different concurrency modes and isolation levels.

The TRANSACTIONAL_SNAPSHOT mode supports both key-value transactions and SQL transactions. It enables multiversion concurrency control (MVCC) for both types of transactions. See Multiversion Concurrency Control for details about and limitations of this mode.

🚧

Beta version of Transactional SQL and MVCC

In Ignite v2.7, Transactional SQL and MVCC are released as beta versions to allow users to experiment and share feedback. This version of Transactional SQL and MVCC should not be considered for production.

Enable TRANSACTIONAL or TRANSACTIONAL_SNAPSHOT mode only if you require ACID-compliant operation.

The ATOMIC mode provides better performance by avoiding transactional locks, whilst providing data atomicity and consistency for each single operation. Another difference in ATOMIC mode is that bulk writes, such as PutAll(...) and RemoveAll(...) methods are not executed in one transaction and can partially fail. If this partial failure occurs, a CachePartialUpdateException will be thrown and will contain a list of keys for which the update failed.

<bean class="org.apache.ignite.configuration.IgniteConfiguration">
    ...
    <property name="cacheConfiguration">
        <bean class="org.apache.ignite.configuration.CacheConfiguration">
            <!-- Set a cache name. -->
            <property name="name" value="myCache"/>

            <!-- Set atomicity mode, can be ATOMIC, TRANSACTIONAL or TRANSACTIONAL_SNAPSHOT. 
                ATOMIC is default. -->
            <property name="atomicityMode" value="TRANSACTIONAL"/>
            ... 
        </bean>
    </property>
     
    <!-- Optional transaction configuration. -->
    <property name="transactionConfiguration">
        <bean class="org.apache.ignite.configuration.TransactionConfiguration">
            <!-- Configure TM lookup here. -->
        </bean>
    </property>
</bean>

CacheConfiguration cacheCfg = new CacheConfiguration();

cacheCfg.setName("cacheName");

cacheCfg.setAtomicityMode(CacheAtomicityMode.ATOMIC);

IgniteConfiguration cfg = new IgniteConfiguration();

cfg.setCacheConfiguration(cacheCfg);

// Optional transaction configuration. Configure TM lookup here.
TransactionConfiguration txCfg = new TransactionConfiguration();

cfg.setTransactionConfiguration(txCfg);

// Start Ignite node.
Ignition.start(cfg);

📘

Performance

Note that transactions are disabled whenever ATOMIC mode is used, providing much higher performance and throughput in cases where transactions are not needed.

IgniteTransactions

IgniteTransactions interface contains functionality for starting and completing transactions, as well as subscribing listeners or getting metrics.

📘

Cross-Cache Transactions

You can combine multiple operations from different caches into one transaction. Note that this allows you to update caches of different types, like REPLICATED and PARTITIONED caches, in one transaction.

🚧

Near Cache Transactions

Near caches are fully transactional (in the TRANSACTIONAL mode) and get updated or invalidated automatically whenever the data changes on the servers. The TRANSACTIONAL_SNAPSHOT mode does not support near caches.

❗️

You cannot execute transactions over multiple caches with different atomicity modes. All caches covered in a single transaction must have the same atomicity mode (either TRANSACTIONAL or TRANSACTIONAL_SNAPSHOT).

You can obtain an instance of IgniteTransactions as follows:

Ignite ignite = Ignition.ignite();

IgniteTransactions transactions = ignite.transactions();

Here is an example of how transactions can be performed in Ignite:

try (Transaction tx = transactions.txStart()) {
    Integer hello = cache.get("Hello");
  
    if (hello == 1)
        cache.put("Hello", 11);
  
    cache.put("World", 22);
  
    tx.commit();
}

🚧

Transaction Object Lifecycle

A Transaction object needs to be closed after use. To ensure this always happens, do one of the following:

• Start the Transaction in a try-with-resources statement that calls the close() method at the end.
• Use the finally block and call the tx.close() method manually.

🚧

Transactional Methods

Not all of the methods in the IgniteCache API are fully transactional when this mode is enabled for a cache. Methods that have "throws TransactionException" in their method signature satisfy the ACID principle and can be safely used inside of a distributed transaction.

Two-Phase-Commit (2PC)

Ignite utilizes a Two-Phase-Commit (2PC) protocol for its transactions and optimizes to one-phase-commit whenever possible. Whenever data is updated within a transaction, Ignite will keep a transactional state in a local transaction map until commit() is called, at which point, if needed, the data is transferred to participating remote nodes.

As the name implies, there are two phases: prepare and commit. These are explained in the following blog posts.

Original series:

• Two-Phase-Commit for Distributed In-Memory Caches
• Two-Phase-Commit for In-Memory Caches - Part II
• One-Phase-Commit - Fast Transactions For In-Memory Caches

New series that takes Ignite persistence into consideration:

• Apache Ignite Transactions Architecture: 2-phase commit protocol
• Apache Ignite Transactions Architecture: Concurrency Modes and Isolation Levels
• Apache Ignite Transactions Architecture: Failover and Recovery
• Apache Ignite transactions architecture: Ignite Persistence Transaction Handling
• Apache Ignite Transactions Architecture: Transaction Handling at the Level of 3rd Party Persistence

Also​, see this page that covers internals of Ignite's transactional subsystem.

👍

ACID Compliance

Ignite provides fully ACID (Atomicity, Consistency, Isolation, Durability) compliant transactions that guarantee consistency.

🚧

3rd Party Persistence Commit

If a transaction updates a cache that uses a 3rd Party Persistence, the changes will be written to the underlying database from the node that initiated this transaction, even if it is a client node. Make sure that the node has write permissions in the underlying database.

❗️

If a transaction updates multiple caches that use 3rd Party Persistence databases, the commit stage will happen sequentially for each database. If any database fails to commit the changes, e.g. when a node fails, all previous databases will not be able to roll them back.

Therefore, we do not recommend using caches with external persistent storage and caches without one in a single transaction. Moreover, if you need to use transactions to update caches with a 3rd party persistent storage, make sure that all caches use the same database.

Deadlock Detection

One major rule that you must follow when working with distributed transactions is that locks for keys participating in a transaction must be acquired in the same order. Violating this rule can lead to a distributed deadlock.

Ignite does not avoid distributed deadlocks, but rather has built-in functionality that makes it easier to debug and fix such situations.

In the code snippet below, a transaction has been started with a timeout. If the timeout expires, the deadlock detection procedure will try to find a possible deadlock that might have caused the timeout. When the timeout expires, TransactionTimeoutException is generated and propagated to the application code as the cause of CacheException regardless of a deadlock. However, if a deadlock is detected, the cause of the returned TransactionTimeoutException will be TransactionDeadlockException (at least for one transaction involved in the deadlock).

try (Transaction tx = ignite.transactions().txStart(TransactionConcurrency.PESSIMISTIC,
    TransactionIsolation.READ_COMMITTED, 300, 0)) {
    cache.put(1, 1);

    cache.put(2, 1);

    tx.commit();
}
catch (CacheException e) {
    if (e.getCause() instanceof TransactionTimeoutException &&
        e.getCause().getCause() instanceof TransactionDeadlockException)    
        
        System.out.println(e.getCause().getCause().getMessage());
}

The TransactionDeadlockException message contains useful information that can help you find the reason for the deadlock.

Deadlock detected:

K1: TX1 holds lock, TX2 waits lock.
K2: TX2 holds lock, TX1 waits lock.

Transactions:

TX1 [txId=GridCacheVersion [topVer=74949328, time=1463469328421, order=1463469326211, nodeOrder=1], nodeId=ad68354d-07b8-4be5-85bb-f5f2362fbb88, threadId=73]
TX2 [txId=GridCacheVersion [topVer=74949328, time=1463469328421, order=1463469326210, nodeOrder=1], nodeId=ad68354d-07b8-4be5-85bb-f5f2362fbb88, threadId=74]

Keys:

K1 [key=1, cache=default]
K2 [key=2, cache=default]

Deadlock detection is a multi-step procedure that can take many iterations depending on the number of nodes in the cluster, keys, and transactions that are involved in a possible deadlock. A deadlock detection initiator is a node where a transaction was started and failed with a TransactionTimeoutException. This node will investigate if a deadlock has occurred by exchanging requests/responses with other remote nodes, and then prepare a deadlock related report that is provided with the TransactionDeadlockException. Each such message (request/response) is known as an iteration.

Since a transaction is not rolled back until the deadlock detection procedure is completed, it sometimes makes sense to tune the parameters (shown below), if you want to have a predictable time for a transaction's rollback.

• IgniteSystemProperties.IGNITE_TX_DEADLOCK_DETECTION_MAX_ITERS - Specifies the maximum number of iterations for the deadlock detection procedure. If the value of this property is less than or equal to zero, deadlock detection will be disabled (1000 by default);
• IgniteSystemProperties.IGNITE_TX_DEADLOCK_DETECTION_TIMEOUT - Specifies the timeout for the deadlock detection mechanism (1 minute by default).

Note that if there are too few iterations, you may get an incomplete deadlock-report.

👍

If you want to completely avoid deadlocks, refer to Deadlock-free Transactions below.

Deadlock-Free Transactions

For OPTIMISTIC SERIALIZABLE transactions, locks are not acquired sequentially. In this mode, keys can be accessed in any order because transaction locks are acquired in parallel with an additional check allowing Ignite to avoid deadlocks.

We need to introduce some concepts in order to describe how locks in SERIALIZABLE transactions work. In Ignite, each transaction is assigned a comparable version called XidVersion. Upon transaction commit, each entry that is written in the transaction is assigned a new comparable version called EntryVersion. An OPTIMISTIC SERIALIZABLE transaction with version XidVersionA will fail with a TransactionOptimisticException if:

• There is an ongoing PESSIMISTIC or non-serializable OPTIMISTIC transaction holding a lock on an entry of the SERIALIZABLE transaction.
• There is another ongoing OPTIMISTIC SERIALIZABLE transaction with version XidVersionB such that XidVersionB > XidVersionA and this transaction holds a lock on an entry of the SERIALIZABLE transaction.
• By the time the OPTIMISTIC SERIALIZABLE transaction acquires all required locks, there exists an entry with the current version different from the observed version before commit.

📘

In a highly concurrent environment, optimistic locking might lead to a high transaction failure rate but pessimistic locking can lead to deadlocks if locks are acquired in a different order by transactions.

However, in a contention-free environment optimistic serializable locking may provide better performance for large transactions because the number of network trips depends only on the number of nodes that the transaction spans and does not depend on the number of keys in the transaction.

Handling Failed Transactions

A transaction might fail with the following exceptions:

📘

Retrying Transactions

It's reasonable to retry both failed optimistic and pessimistic transactions. Even if there is a failed transaction due to some issue caused by network problems or node failures, Ignite will use backup copies or the data available on disk to enforce data consistency.

However, don't try to re-run the transaction infinitely on the application level. Include​ logic that limits retry attempts to a finite amount of time.

Long Running Transactions Termination

Some cluster events trigger partition map exchange process and data rebalancing within an Ignite cluster to ensure even data distribution cluster-wide. An example of one such event is the cluster-topology-change event that takes place whenever a new node joins the cluster or an existing one leaves it. Plus, every time a new cache or SQL table is created, the partition map exchange gets triggered.

When the partition map exchange starts, Ignite acquires a global lock at a particular stage. The lock can't be obtained while incomplete transactions are running in parallel. These transactions prevent the partition map exchange process from moving forward​, thus, blocking some operations such as a new node join process.

Use the TransactionConfiguration.setTxTimeoutOnPartitionMapExchange(...) method to set the maximum time allowed for your long-running transactions to block the partition map exchange. Once the timeout fires, all the incomplete transactions will be rolled back letting the partition map exchange proceed.

This example shows how to configure the timeout:

<bean class="org.apache.ignite.configuration.IgniteConfiguration">
  	...
    <property name="transactionConfiguration">
        <bean class="org.apache.ignite.configuration.TransactionConfiguration">
        <!--Set the timeout to 20 seconds-->
        <property name="TxTimeoutOnPartitionMapExchange" value="20000"/>
        <!--Other trasaction configurations-->
        ...
        </bean>
    </property>
</bean>

// Create Ignite configuration
IgniteConfiguration cfg = new IgniteConfiguration();
        
// Create Ignite Transactions configuration
TransactionConfiguration txCfg = new TransactionConfiguration();

// Set the timeout to 20 seconds
txCfg.setTxTimeoutOnPartitionMapExchange(20000);

cfg.setTransactionConfiguration(txCfg);

// Start the cluster node
Ignition.start(cfg);

If a transaction is rolled back due to the timeout firing, you'll be able to catch and process TransactionTimeoutException.

Integration With JTA

Ignite can be configured with a JTA transaction manager lookup class using the TransactionConfiguration#setTxManagerFactory method. Transaction manager factory is a factory that provides Ignite with an instance of JTA transaction manager.

Ignite provides CacheJndiTmFactory factory. It's an out-of-the-box transaction manager factory implementation that uses JNDI names to find TM.

When set, Ignite will check to see if there is an ongoing JTA transaction during each cache operation on a transactional cache. If JTA transaction is started, Ignite will also start a transaction and will enlist it into the JTA transaction using its own internal implementation of XAResource. The Ignite transaction will be prepared, committed, or rolled back altogether with the corresponding JTA transaction.

Below is an example of using JTA transaction manager together with Ignite.

// Get an instance of JTA transaction manager.
TMService tms = appCtx.getComponent(TMService.class);

// Get an instance of Ignite cache.
IgniteCache<String, Integer> cache = cache();

UserTransaction jtaTx = tms.getUserTransaction();

// Start JTA transaction.
jtaTx.begin();

try {
    // Do some cache operations.
    cache.put("key1", 1);
    cache.put("key2", 2);

    // Commit the transaction.
    jtaTx.commit();
}
finally {
    // Rollback in a case of exception.
    if (jtaTx.getStatus() == Status.STATUS_ACTIVE)
        jtaTx.rollback();
}