Skip to content

fix(sql): improve refresh step estimation for sparse data with small sample by interval#6517

Merged
bluestreak01 merged 6 commits intomasterfrom
mat_view_estimation_fix
Dec 9, 2025
Merged

fix(sql): improve refresh step estimation for sparse data with small sample by interval#6517
bluestreak01 merged 6 commits intomasterfrom
mat_view_estimation_fix

Conversation

@kafka1991
Copy link
Copy Markdown
Collaborator

@kafka1991 kafka1991 commented Dec 9, 2025

This PR fixes inefficient refresh queries when mat views use small sampling intervals (e.g., microsecond) on tables with sparse data distribution. The improved estimation ensures optimal batch sizes for incremental refresh.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Dec 9, 2025

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch mat_view_estimation_fix

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@kafka1991 kafka1991 changed the title fix(sql): update estimation methods to improve accuracy for sparse data fix(sql): improve refresh step estimation for sparse and small sample by interval Dec 9, 2025
@kafka1991 kafka1991 changed the title fix(sql): improve refresh step estimation for sparse and small sample by interval fix(sql): improve refresh step estimation for sparse data with small sample by interval Dec 9, 2025
puzpuzpuz
puzpuzpuz previously approved these changes Dec 9, 2025
@puzpuzpuz puzpuzpuz added Bug Incorrect or unexpected behavior Materialized View labels Dec 9, 2025
@glasstiger
Copy link
Copy Markdown
Contributor

[PR Coverage check]

😍 pass : 26 / 27 (96.30%)

file detail

path covered line new line coverage
🔵 io/questdb/cairo/DefaultCairoConfiguration.java 0 1 00.00%
🔵 io/questdb/griffin/engine/groupby/MonthTimestampNanosSampler.java 1 1 100.00%
🔵 io/questdb/griffin/engine/groupby/YearTimestampMicrosSampler.java 1 1 100.00%
🔵 io/questdb/cairo/TableReader.java 12 12 100.00%
🔵 io/questdb/PropServerConfiguration.java 1 1 100.00%
🔵 io/questdb/griffin/engine/groupby/MonthTimestampMicrosSampler.java 1 1 100.00%
🔵 io/questdb/cairo/mv/MatViewRefreshJob.java 8 8 100.00%
🔵 io/questdb/griffin/engine/groupby/YearTimestampNanosSampler.java 1 1 100.00%
🔵 io/questdb/cairo/pool/ReaderPool.java 1 1 100.00%

@bluestreak01 bluestreak01 merged commit d6bc4a5 into master Dec 9, 2025
41 of 43 checks passed
@bluestreak01 bluestreak01 deleted the mat_view_estimation_fix branch December 9, 2025 13:59
@questdb-butler
Copy link
Copy Markdown

⚠️ Enterprise CI Failed

The enterprise test suite failed for this PR.

Build: View Details
Tested Commit: d277f93790c9c4855c47d4195349841129df0183

Please investigate the failure before merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug Incorrect or unexpected behavior Materialized View

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants