Skip to content

chore(sql): speed up horizon join time frame search for large RHS tables#6814

Merged
bluestreak01 merged 8 commits intomasterfrom
puzpuzpuz_horizon_join_debug
Feb 25, 2026
Merged

chore(sql): speed up horizon join time frame search for large RHS tables#6814
bluestreak01 merged 8 commits intomasterfrom
puzpuzpuz_horizon_join_debug

Conversation

@puzpuzpuz
Copy link
Copy Markdown
Contributor

@puzpuzpuz puzpuzpuz commented Feb 24, 2026 •
edited
Loading

Speeds up HORIZON JOIN in case of very large right-hand-side tables.

Implementation

HorizonJoinTimeFrameHelper.findAsOfRow() now uses a TimeFrameCursor#seekEstimate() call to binary-search to the target timestamp's vicinity before linearly scanning frames, reducing frame iteration from O(N) to O(log N) for large right-hand-side tables.

@puzpuzpuz puzpuzpuz self-assigned this Feb 24, 2026
@puzpuzpuz puzpuzpuz added SQL Issues or changes relating to SQL execution DO NOT MERGE These changes should not be merged to main branch labels Feb 24, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Feb 24, 2026

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch puzpuzpuz_horizon_join_debug

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

puzpuzpuz and others added 4 commits February 24, 2026 18:59
@puzpuzpuz
Merge remote-tracking branch 'upstream/master' into puzpuzpuz_horizon…
f3b65f7 
…_join_debug
@puzpuzpuz
Address review feedback
8e91867
@puzpuzpuz
Add multi-partition tests
a538b58
@bluestreak01 @claude
Add seekEstimate test for split partitions
b9e1be9 
Cover the case where O3 ingestion splits a partition into multiple
frames that share the same ceiling. The test verifies that
seekEstimate returns the last frame in each split group and that
next() correctly advances past it.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
@bluestreak01 bluestreak01 added Performance Performance improvements and removed DO NOT MERGE These changes should not be merged to main branch labels Feb 24, 2026
bluestreak01
bluestreak01 previously approved these changes Feb 24, 2026
View reviewed changes
@puzpuzpuz
Copy link
Copy Markdown
Contributor Author

puzpuzpuz commented Feb 25, 2026

@bluestreak01 testHorizonJoinParallelExecution is getting stuck on linux-griffin-root. I'm debugging it - the fix should be ready soon.

@puzpuzpuz
Copy link
Copy Markdown
Contributor Author

puzpuzpuz commented Feb 25, 2026

@bluestreak01 testHorizonJoinParallelExecution is getting stuck on linux-griffin-root. I'm debugging it - the fix should be ready soon.

The bugfix is in e4d6356. Let's see how CI does now.

@glasstiger
Copy link
Copy Markdown
Contributor

glasstiger commented Feb 25, 2026

[PR Coverage check]

😍 pass : 68 / 76 (89.47%)

file detail

path covered line new line coverage
🔵 io/questdb/griffin/engine/table/ExtraNullColumnCursorFactory.java 0 4 00.00%
🔵 io/questdb/griffin/engine/table/SelectedRecordCursorFactory.java 2 4 50.00%
🔵 io/questdb/griffin/engine/table/HorizonJoinTimeFrameHelper.java 43 45 95.56%
🔵 io/questdb/griffin/engine/table/ConcurrentTimeFrameCursorImpl.java 2 2 100.00%
🔵 io/questdb/cairo/sql/TimeFrameCursor.java 18 18 100.00%
🔵 io/questdb/griffin/engine/table/TimeFrameCursorImpl.java 3 3 100.00%

@bluestreak01 bluestreak01 merged commit 8ab362f into master Feb 25, 2026
44 checks passed
@bluestreak01 bluestreak01 deleted the puzpuzpuz_horizon_join_debug branch February 25, 2026 12:26
@puzpuzpuz
Copy link
Copy Markdown
Contributor Author

puzpuzpuz commented Feb 25, 2026

@bluestreak01 thanks for the review!

@puzpuzpuz puzpuzpuz mentioned this pull request Feb 25, 2026
Merged
bluestreak01 pushed a commit that referenced this pull request Feb 25, 2026
@puzpuzpuz @bluestreak01
chore(sql): speed up horizon join time frame search for large RHS tab…
7377c75 
…les (#6814)
maciulis pushed a commit to maciulis/questdb that referenced this pull request Mar 16, 2026
@puzpuzpuz
chore(sql): speed up horizon join time frame search for large RHS tab…
7e83cb4 
…les (questdb#6814)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bluestreak01 bluestreak01 bluestreak01 approved these changes

Assignees

@puzpuzpuz puzpuzpuz

Labels

Performance Performance improvements SQL Issues or changes relating to SQL execution

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@puzpuzpuz @glasstiger @bluestreak01