Repository navigation
catalog: skip consolidation on syncs that apply no updates - #39482
Merged
ggevay merged 2 commits intoOct 2, 2026
Merged
Conversation
ggevay
force-pushed
the
gabor/sql-750-takeover-fence-livelock
branch
from
October 2, 2026 13:39
2ae92dc to
c1bc0b5
Compare
`sync_inner` consolidated the whole in-memory snapshot after every sync, even when the sync only observed upper progress. The 0dt takeover's fence loop syncs before each compare-and-append, and the opening handle's snapshot still holds the full audit log. Once that consolidation outlasts the interval between the serving leader's upper advances, every fence attempt loses the race. Co-Authored-By: Claude Opus 5.5 <[email protected]>
ggevay
force-pushed
the
gabor/sql-750-takeover-fence-livelock
branch
from
October 2, 2026 14:03
c1bc0b5 to
ca78ba2
Compare
ggevay
marked this pull request as ready for review
October 2, 2026 14:44
aljoscha
approved these changes
Oct 2, 2026
aljoscha
left a comment
aljoscha
left a comment
Contributor
There was a problem hiding this comment.
excellent change! I had a nit about the comment being overly verbose
| assert_eq!(updates, BTreeMap::new(), "all updates should be applied"); | ||
| // Always consolidate at the end to ensure the snapshot is clean. | ||
| self.consolidate(); | ||
| // A sync that applied no updates finds the snapshot consolidated: every successful |
Contributor
There was a problem hiding this comment.
hello bot: maybe "Only consolidate when there are actual updates." is enough here. I need to parse all this crap this whole comment that refers to other places in the code that will probably rot. 😅
Co-Authored-By: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
sync_innerconsolidates the in-memory snapshot only when the sync applied updates. A sync that applied none starts from a consolidated snapshot, so the skipped pass was a no-op.open_inner) syncs before each compare-and-append against the serving leader's upper, which the leader advances on every group commit, at least once per second. The opening handle's snapshot still holds the full audit log, 7.4M entries on the largest production catalog. Each sync spent about 1.1 s consolidating, so every attempt lost the race. One cutover waited 3 h 11 min, with about 1,700 failed attempts.test_persist_sync_of_upper_progress_skips_consolidationinsrc/catalog/tests/open.rschecks that a sync over upper-only progress does not consolidate, while a sync that applies a committed item does. It fails on main.Closes: SQL-750
🤖 Generated with Claude Code