Skip to content

Avoid MongoDB write conflicts in MongoJobRepository by replacing sequence-based ID with TSID#5144

Open
diydriller wants to merge 2 commits intospring-projects:mainfrom
diydriller:GH-4960
Open

Avoid MongoDB write conflicts in MongoJobRepository by replacing sequence-based ID with TSID#5144
diydriller wants to merge 2 commits intospring-projects:mainfrom
diydriller:GH-4960

Conversation

@diydriller
Copy link
Copy Markdown

@diydriller diydriller commented Dec 7, 2025

Summary

This PR addresses a concurrency issue in the MongoDBJobRepository.

The current implementation generates numeric identifiers for job/step executions through a sequence document.
Under concurrency, especially with transactional writes, this causes frequent WriteConflict errors from MongoDB.

This PR replaces the numeric sequence generation with application-generated, unique, time-ordered identifier.
This eliminates write conflicts caused by sequence updates.

Problem

MongoDB uses optimistic concurrency control, and concurrent writes to the same document cause conflicts.

collection.findOneAndUpdate(
    new Document("_id", sequenceName),
    new Document("$inc", new Document("count", 1)),
    options
).getLong("count");

This produces high contention on a single document under parallel workloads.

Solution

Replace sequence-based numeric ID generation with TSID.
Implements TSID algorithm(42-bit timestamp + 10-bit node ID + 12-bit sequence)

TSID characteristics

  • time-ordered
  • unique
  • 64-bit long value

Fixes #4960


private final String sequenceName;
public MongoSequenceIncrementer() {
this.nodeId = (int) (System.nanoTime() & NODE_MASK);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's still possible same nodeId is used by multiple instances resulting in conflicts.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

enhanced the nodeId generation by incorporating the hostname, process ID, and a random value.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

WriteConflict in MongoSequenceIncrementer during parallel job executions

3 participants