Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cherry pick PR 7462
When there are no user commits, versions advance at a minimum rate governed by knob MAX_COMMIT_BATCH_INTERVAL. FDB needs continuously generated commits, e.g. in shard movement where new versions are "piggy backed" to make progress.
The actors commitBatcher and commitProxyServerCore generate these commits. The problem is they run asynchronously to each other and can get out of sync. The former times out every MAX_COMMIT_BATCH_INTERVAL seconds and wakes the later, which may have sent an empty message less than MAX_COMMIT_BATCH_INTERVAL seconds ago (it may have delayed the previous commit). In that case, commitProxyServerCore won't send the empty commit, and a new commit won't be generated until commitBatcher times out again.
This pattern delays version advancement up to MAX_COMMIT_BATCH_INTERVAL*2 seconds and breaks the knob's contract. Shard movement is slowed, eventually leading to the failure in quiet database.
This PR makes a single actor (commitBatcher) regulate the frequency of empty commits.
Fixes
MutationLogReaderCorrectness.toml -b on -s 653691063;MutationLogReaderCorrectness.toml -b on -s 98766466etc. (more).
Joshua
20220701-220644-henrylambright-f99f08bd5fb313efCode-Reviewer Section
The general guidelines can be found here.
Please check each of the following things and check all boxes before accepting a PR.
For Release-Branches
If this PR is made against a release-branch, please also check the following:
release-branchormainif this is the youngest branch)