Avoid MongoDB write conflicts in MongoJobRepository by replacing sequence-based ID with TSID#5144
Open
diydriller wants to merge 2 commits intospring-projects:mainfrom
Open
Avoid MongoDB write conflicts in MongoJobRepository by replacing sequence-based ID with TSID#5144diydriller wants to merge 2 commits intospring-projects:mainfrom
diydriller wants to merge 2 commits intospring-projects:mainfrom
Conversation
Signed-off-by: Hyun Jong Park <[email protected]>
quaff
reviewed
Dec 8, 2025
|
|
||
| private final String sequenceName; | ||
| public MongoSequenceIncrementer() { | ||
| this.nodeId = (int) (System.nanoTime() & NODE_MASK); |
Contributor
There was a problem hiding this comment.
It's still possible same nodeId is used by multiple instances resulting in conflicts.
Author
There was a problem hiding this comment.
enhanced the nodeId generation by incorporating the hostname, process ID, and a random value.
…ments Signed-off-by: Hyun Jong Park <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR addresses a concurrency issue in the MongoDBJobRepository.
The current implementation generates numeric identifiers for job/step executions through a sequence document.
Under concurrency, especially with transactional writes, this causes frequent WriteConflict errors from MongoDB.
This PR replaces the numeric sequence generation with application-generated, unique, time-ordered identifier.
This eliminates write conflicts caused by sequence updates.
Problem
MongoDB uses optimistic concurrency control, and concurrent writes to the same document cause conflicts.
This produces high contention on a single document under parallel workloads.
Solution
Replace sequence-based numeric ID generation with TSID.
Implements TSID algorithm(42-bit timestamp + 10-bit node ID + 12-bit sequence)
TSID characteristics
Fixes #4960