fix(sql): cache WAL txn intervals to avoid out-of-disk-space errors in case of manual and timer materialized views#5797
Merged
bluestreak01 merged 42 commits intomasterfrom Jul 7, 2025
Conversation
puzpuzpuz
added
Bug
… in case of manual and timer materialized views
puzpuzpuz
force-pushed
the
puzpuzpuz_mat_view_wal_purge
branch
from
d96baa7 to
55be4e9
Compare
puzpuzpuz
changed the title
ideoma
reviewed
Jul 3, 2025
…into puzpuzpuz_mat_view_wal_purge
puzpuzpuz
force-pushed
the
puzpuzpuz_mat_view_wal_purge
branch
from
656f45d to
1ab536c
Compare
ideoma
previously approved these changes
Jul 4, 2025
ideoma
reviewed
Jul 7, 2025
ideoma
previously approved these changes
Jul 7, 2025
ideoma
approved these changes
Jul 7, 2025
Contributor
[PR Coverage check]😍 pass : 310 / 332 (93.37%) file detail
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes
WalPurgeJobnot being able to make progress in case of materialized views with manual or timer refresh. The problem due to the refresh job having to read txn intervals from WAL transactions (_eventfiles) to perform an incremental refresh. But for manual or timer views the refresh may be triggered very infrequently (or even never) leading to old WAL segments accumulating on disk. This patch introduces a txn intervals cache to mat view state objects. The cache is refreshed on each base table transaction, so that from theWalPurgeJob's perspective manual and timer views now behave similar to the immediate ones.The caching is done via a special timer, so it's throttled to avoid writing too many mat view state WAL transactions with cached intervals to the disk.
Also enables manual refresh type.