Skip to content

fix(sql): NullPointerException when right-hand query in window join has timestamp filter#6806

Merged
bluestreak01 merged 6 commits intomasterfrom
puzpuzpuz_horizon_join_fixes
Feb 24, 2026
Merged

fix(sql): NullPointerException when right-hand query in window join has timestamp filter#6806
bluestreak01 merged 6 commits intomasterfrom
puzpuzpuz_horizon_join_fixes

Conversation

@puzpuzpuz
Copy link
Copy Markdown
Contributor

@puzpuzpuz puzpuzpuz commented Feb 23, 2026

WINDOW JOIN and HORIZON JOIN crash with a NullPointerException when the right-hand side (slave) is a subquery that applies a timestamp interval filter or reorders columns, e.g.:

SELECT ... FROM trades t
WINDOW JOIN (SELECT price, sym, ts FROM prices WHERE ts IN '2024-01-01') p
ON (t.sym = p.sym) RANGE BETWEEN ...

The root cause is that SelectedRecordCursorFactory and ExtraNullColumnCursorFactory did not implement newTimeFrameCursor(), which is required by the concurrent time frame cursor infrastructure used in these joins. Additionally, SelectedPageFrameCursor did not implement TablePageFrameCursor, causing a ClassCastException.

After fixing those, a subtler double column remapping bug remained: both SelectedConcurrentTimeFrameCursor and SelectedPageFrameCursor were independently remapping column indices through columnCrossIndex, causing symbol table lookups to hit wrong columns (returning null) and the timestamp column to be read from the wrong position in the address cache.

Changes

  • Extract ConcurrentTimeFrameCursor into an interface and ConcurrentTimeFrameCursorImpl implementation class.
  • Add newTimeFrameCursor() to SelectedRecordCursorFactory, ExtraNullColumnCursorFactory, and QueryProgress.
  • Make SelectedPageFrameCursor and ExtraNullColumnPageFrameCursor implement TablePageFrameCursor.
  • Add timestampIndex parameter to ConcurrentTimeFrameCursor.of() so wrappers can specify the correct timestamp column position in the (potentially remapped) address cache.
  • Simplify SelectedConcurrentTimeFrameCursor to delegate column remapping entirely to SelectedPageFrameCursor, avoiding double remapping.
  • Add tests for both WINDOW JOIN and HORIZON JOIN with timestamp filters and column reordering on the slave side.

@puzpuzpuz puzpuzpuz self-assigned this Feb 23, 2026
@puzpuzpuz puzpuzpuz added Bug Incorrect or unexpected behavior SQL Issues or changes relating to SQL execution labels Feb 23, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Feb 23, 2026

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Walkthrough

This PR refactors the time-frame cursor architecture by converting ConcurrentTimeFrameCursor from a concrete class to a public interface, introducing ConcurrentTimeFrameCursorImpl as its implementation, and systematically propagating timestamp index information through cursor initialization. Additionally, CairoConfiguration parameters are removed from AbstractPageFrameRecordCursorFactory and related subclasses.

Changes

Cohort / File(s) Summary
Time-Frame Cursor Interface & Implementation
core/src/main/java/io/questdb/griffin/engine/table/ConcurrentTimeFrameCursor.java, core/src/main/java/io/questdb/griffin/engine/table/ConcurrentTimeFrameCursorImpl.java
Converts ConcurrentTimeFrameCursor from a concrete final class to a public interface extending TimeFrameCursor, updates of() signature to include timestampIndex parameter. Introduces new ConcurrentTimeFrameCursorImpl class providing full page-frame cursor implementation with timestamp caching, frame navigation, and record materialization.
Timestamp Index Propagation in Join Atoms
core/src/main/java/io/questdb/griffin/engine/join/AsyncWindowJoinAtom.java, core/src/main/java/io/questdb/griffin/engine/table/BaseAsyncHorizonJoinAtom.java
Retrieves timestampIndex from owner slave time-frame cursor and propagates it as an additional parameter to of() calls on both owner and per-worker cursor initialization paths.
Timestamp Index in Filtered As-Of Join Factories
core/src/main/java/io/questdb/griffin/engine/join/FilteredAsOfJoinFastRecordCursorFactory.java, core/src/main/java/io/questdb/griffin/engine/join/FilteredAsOfJoinNoKeyFastRecordCursorFactory.java
Adds slaveTimestampIndex parameter to SelectedRecordCursorFactory.SelectedTimeFrameCursor constructor calls when slave cross-index is detected.
Configuration Parameter Removal from Base Classes
core/src/main/java/io/questdb/griffin/engine/table/AbstractPageFrameRecordCursorFactory.java, core/src/main/java/io/questdb/griffin/engine/table/AbstractDeferredValueRecordCursorFactory.java
Removes CairoConfiguration from public constructor signatures and updates superclass constructor invocations accordingly. Updates import declarations from wildcards to explicit imports.
Configuration Removal in Subclasses
core/src/main/java/io/questdb/griffin/engine/table/AbstractTreeSetRecordCursorFactory.java, core/src/main/java/io/questdb/griffin/engine/table/FilterOnExcludedValuesRecordCursorFactory.java, core/src/main/java/io/questdb/griffin/engine/table/FilterOnSubQueryRecordCursorFactory.java, core/src/main/java/io/questdb/griffin/engine/table/FilterOnValuesRecordCursorFactory.java, core/src/main/java/io/questdb/griffin/engine/table/SortedSymbolIndexRecordCursorFactory.java
Updates super() calls to remove CairoConfiguration parameter from parent constructor invocations.
Deferred/Latest Factory Updates
core/src/main/java/io/questdb/griffin/engine/table/LatestByDeferredListValuesFilteredRecordCursorFactory.java, core/src/main/java/io/questdb/griffin/engine/table/LatestByValueDeferredFilteredRecordCursorFactory.java, core/src/main/java/io/questdb/griffin/engine/table/LatestByValueDeferredIndexedFilteredRecordCursorFactory.java, core/src/main/java/io/questdb/griffin/engine/table/LatestByValueFilteredRecordCursorFactory.java, core/src/main/java/io/questdb/griffin/engine/table/LatestByValueIndexedFilteredRecordCursorFactory.java
Updates wildcard imports to explicit imports and removes CairoConfiguration from super() calls. Aligns constructor chaining with simplified base class signatures.
New Time-Frame Cursor Methods
core/src/main/java/io/questdb/griffin/engine/QueryProgress.java, core/src/main/java/io/questdb/griffin/engine/table/PageFrameRecordCursorFactory.java
Adds newTimeFrameCursor() method that returns ConcurrentTimeFrameCursor and instantiates ConcurrentTimeFrameCursorImpl when framing is supported.
Enhanced Cursor Factory Wrappers
core/src/main/java/io/questdb/griffin/engine/table/SelectedRecordCursorFactory.java, core/src/main/java/io/questdb/griffin/engine/table/ExtraNullColumnCursorFactory.java
Introduces wrapper classes (SelectedConcurrentTimeFrameCursor, ExtraNullColumnConcurrentTimeFrameCursor) to propagate selectedTimestampIndex through time-frame cursor operations. Updates SelectedPageFrameCursor and ExtraNullColumnPageFrameCursor to implement TablePageFrameCursor with getTableReader() support. Adds newTimeFrameCursor() methods and constructor signatures to accept and store selectedTimestampIndex.
Test Coverage
core/src/test/java/io/questdb/test/griffin/HorizonJoinTest.java, core/src/test/java/io/questdb/test/griffin/WindowJoinTest.java
Adds testHorizonJoinFilterTimestampFilter and testHorizonJoinFilterTimestampFilterSequential to validate horizon joins with timestamp-based filtering. Adds multiple window join tests (testWindowJoinFilterTimestampFilter, testWindowJoinFilterTimestampFilterSequential, testWindowJoinSelfJoinWithAggregatesInSelectAndWhere, testWindowJoinSelfJoinWithVwapCalculation) and removes/replaces test coverage for self-join scenarios.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Suggested labels

Core, Refactoring

Suggested reviewers

  • mtopolnik
  • bluestreak01
🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 4.72% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title 'fix(sql): NullPointerException when right-hand query in window join has timestamp filter' clearly and specifically describes the main issue being fixed—a NullPointerException in WINDOW JOIN/HORIZON JOIN when the right-hand query has a timestamp filter.
Description check ✅ Passed The description provides comprehensive context: it explains the crash scenario with a SQL example, identifies root causes (missing newTimeFrameCursor(), ClassCastException, double column remapping bug), and details all changes made including interface extraction, new methods, parameter additions, and test coverage.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch puzpuzpuz_horizon_join_fixes

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@puzpuzpuz
Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Feb 24, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
core/src/main/java/io/questdb/griffin/engine/table/SelectedRecordCursorFactory.java (1)

135-144: ⚠️ Potential issue | 🟠 Major

Add defensive instanceof check before casting to TablePageFrameCursor.

The cast at line 143 assumes base.getPageFrameCursor() always returns a TablePageFrameCursor, but implementations like ReadParquetPageFrameCursor and RegisteredPageFrameCursor implement PageFrameCursor directly without table backing. This will cause a ClassCastException if a non-table-backed cursor is returned.

Additionally, SelectedPageFrameCursor.of(PartitionFrameCursor, ...) intentionally throws UnsupportedOperationException (lines 377-378) since initialization occurs through wrap() instead. Guard the wrap() call with an instanceof check to ensure the cursor is actually TablePageFrameCursor, or return baseCursor unchanged if it is not.

Same issue applies at lines 373-379.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@core/src/main/java/io/questdb/griffin/engine/table/SelectedRecordCursorFactory.java`
around lines 135 - 144, The getPageFrameCursor method assumes
base.getPageFrameCursor() returns a TablePageFrameCursor and casts
unconditionally; add a defensive instanceof check before casting and calling
SelectedPageFrameCursor.wrap so non-table-backed cursors (e.g.,
ReadParquetPageFrameCursor, RegisteredPageFrameCursor) are returned unchanged.
Specifically, in SelectedRecordCursorFactory.getPageFrameCursor, verify
baseCursor instanceof TablePageFrameCursor before calling
pageFrameCursor.wrap((TablePageFrameCursor) baseCursor) and return baseCursor if
not; apply the same pattern where SelectedPageFrameCursor.of/.wrap is used
(including the PartitionFrameCursor initialization path) to avoid
ClassCastException and the UnsupportedOperationException from incorrect
initialization.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In
`@core/src/main/java/io/questdb/griffin/engine/table/ConcurrentTimeFrameCursorImpl.java`:
- Around line 49-67: Reorder the private instance fields in class
ConcurrentTimeFrameCursorImpl so they are alphabetized by name while keeping
their modifiers, initializers and associated comments intact; specifically
arrange frameCount, frameCursor, frameMemoryPool, framePartitionIndexes,
frameRowCounts, frameTimestampCache, partitionCeilings, partitionTimestamps,
record, timeFrame, and timestampIndex into alphabetical order within the private
instance group and preserve the "// Cursor's lifecycle..." and timestamp-index
comment blocks adjacent to their respective fields.

In
`@core/src/main/java/io/questdb/griffin/engine/table/SelectedRecordCursorFactory.java`:
- Around line 188-195: In SelectedRecordCursorFactory, relocate the public
instance method newTimeFrameCursor() so public methods remain alphabetically
ordered: move the newTimeFrameCursor() method to sit between isProjection() and
recordCursorSupportsLongTopK(); keep the method body unchanged (it constructs a
SelectedConcurrentTimeFrameCursor when crossedIndex is true) and ensure
imports/visibility remain the same.

In `@core/src/test/java/io/questdb/test/griffin/HorizonJoinTest.java`:
- Around line 639-671: Update the SQL literals passed into the
assertQueryNoLeakCheck calls so keywords use uppercase IN and the SQL is written
as a Java text block (multiline string) instead of concatenated string
expressions; specifically modify the two queries used in the HorizonJoinTest's
assertQueryNoLeakCheck calls (the HORIZON JOIN query and the variant with
"SELECT price, price price0, sym, ts FROM prices WHERE ts in '1970-01-01'") to
replace "in" with "IN" and convert the concatenated/quoted SQL to a single
"""...""" text block each, keeping the same expressions for getSecondsDivisor()
and the surrounding Java code (these changes affect the SQL literals used by
SelectedConcurrentTimeFrameCursor test paths).

In `@core/src/test/java/io/questdb/test/griffin/WindowJoinTest.java`:
- Around line 3390-3448: The test method
testWindowJoinSelfJoinWithAggregatesInSelectAndWhere uses long concatenated
string literals for the CREATE/INSERT and the SELECT+expected result; change
these to Java multiline text blocks ("""...""") to improve readability and match
test SQL formatting rules—replace the concatenated execute("INSERT INTO
fx_trades VALUES " + "...") call and the long "SELECT " + "t.timestamp, " + ...
string passed into assertQueryNoLeakCheck, as well as the expected result block,
with proper text blocks while keeping calls to execute(...) and
assertQueryNoLeakCheck(...) and their argument order unchanged.
- Around line 3450-3527: Update the testWindowJoinSelfJoinWithVwapCalculation
test to follow formatting rules: replace large numeric literals 10000 and
2000000 with underscore-separated forms (e.g., 10_000, 2_000_000) in the INSERT
expression and any setProperty values if applicable, and convert the long SQL
string assignments (the "insert into fx_trades ..." and the failing query
assigned to query) to Java text blocks (multiline string literals) to improve
readability; edit the string variables used in engine.execute(...) and the query
variable inside testWindowJoinSelfJoinWithVwapCalculation and keep all semantics
unchanged.

---

Outside diff comments:
In
`@core/src/main/java/io/questdb/griffin/engine/table/SelectedRecordCursorFactory.java`:
- Around line 135-144: The getPageFrameCursor method assumes
base.getPageFrameCursor() returns a TablePageFrameCursor and casts
unconditionally; add a defensive instanceof check before casting and calling
SelectedPageFrameCursor.wrap so non-table-backed cursors (e.g.,
ReadParquetPageFrameCursor, RegisteredPageFrameCursor) are returned unchanged.
Specifically, in SelectedRecordCursorFactory.getPageFrameCursor, verify
baseCursor instanceof TablePageFrameCursor before calling
pageFrameCursor.wrap((TablePageFrameCursor) baseCursor) and return baseCursor if
not; apply the same pattern where SelectedPageFrameCursor.of/.wrap is used
(including the PartitionFrameCursor initialization path) to avoid
ClassCastException and the UnsupportedOperationException from incorrect
initialization.

ℹ️ Review info

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f353964 and 05ee01a.

📒 Files selected for processing (24)
  • core/src/main/java/io/questdb/griffin/engine/QueryProgress.java
  • core/src/main/java/io/questdb/griffin/engine/join/AsyncWindowJoinAtom.java
  • core/src/main/java/io/questdb/griffin/engine/join/FilteredAsOfJoinFastRecordCursorFactory.java
  • core/src/main/java/io/questdb/griffin/engine/join/FilteredAsOfJoinNoKeyFastRecordCursorFactory.java
  • core/src/main/java/io/questdb/griffin/engine/table/AbstractDeferredValueRecordCursorFactory.java
  • core/src/main/java/io/questdb/griffin/engine/table/AbstractPageFrameRecordCursorFactory.java
  • core/src/main/java/io/questdb/griffin/engine/table/AbstractTreeSetRecordCursorFactory.java
  • core/src/main/java/io/questdb/griffin/engine/table/BaseAsyncHorizonJoinAtom.java
  • core/src/main/java/io/questdb/griffin/engine/table/ConcurrentTimeFrameCursor.java
  • core/src/main/java/io/questdb/griffin/engine/table/ConcurrentTimeFrameCursorImpl.java
  • core/src/main/java/io/questdb/griffin/engine/table/ExtraNullColumnCursorFactory.java
  • core/src/main/java/io/questdb/griffin/engine/table/FilterOnExcludedValuesRecordCursorFactory.java
  • core/src/main/java/io/questdb/griffin/engine/table/FilterOnSubQueryRecordCursorFactory.java
  • core/src/main/java/io/questdb/griffin/engine/table/FilterOnValuesRecordCursorFactory.java
  • core/src/main/java/io/questdb/griffin/engine/table/LatestByDeferredListValuesFilteredRecordCursorFactory.java
  • core/src/main/java/io/questdb/griffin/engine/table/LatestByValueDeferredFilteredRecordCursorFactory.java
  • core/src/main/java/io/questdb/griffin/engine/table/LatestByValueDeferredIndexedFilteredRecordCursorFactory.java
  • core/src/main/java/io/questdb/griffin/engine/table/LatestByValueFilteredRecordCursorFactory.java
  • core/src/main/java/io/questdb/griffin/engine/table/LatestByValueIndexedFilteredRecordCursorFactory.java
  • core/src/main/java/io/questdb/griffin/engine/table/PageFrameRecordCursorFactory.java
  • core/src/main/java/io/questdb/griffin/engine/table/SelectedRecordCursorFactory.java
  • core/src/main/java/io/questdb/griffin/engine/table/SortedSymbolIndexRecordCursorFactory.java
  • core/src/test/java/io/questdb/test/griffin/HorizonJoinTest.java
  • core/src/test/java/io/questdb/test/griffin/WindowJoinTest.java
💤 Files with no reviewable changes (1)
  • core/src/main/java/io/questdb/griffin/engine/table/AbstractPageFrameRecordCursorFactory.java

@glasstiger
Copy link
Copy Markdown
Contributor

[PR Coverage check]

😍 pass : 134 / 191 (70.16%)

file detail

path covered line new line coverage
🔵 io/questdb/griffin/engine/QueryProgress.java 0 1 00.00%
🔵 io/questdb/griffin/engine/table/ExtraNullColumnCursorFactory.java 1 42 02.38%
🔵 io/questdb/griffin/engine/table/SelectedRecordCursorFactory.java 32 37 86.49%
🔵 io/questdb/griffin/engine/table/ConcurrentTimeFrameCursorImpl.java 84 94 89.36%
🔵 io/questdb/griffin/engine/table/SortedSymbolIndexRecordCursorFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/join/AsyncWindowJoinAtom.java 1 1 100.00%
🔵 io/questdb/griffin/engine/table/AbstractDeferredValueRecordCursorFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/table/AbstractTreeSetRecordCursorFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/join/FilteredAsOfJoinNoKeyFastRecordCursorFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/table/PageFrameRecordCursorFactory.java 2 2 100.00%
🔵 io/questdb/griffin/engine/table/LatestByValueDeferredFilteredRecordCursorFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/table/FilterOnExcludedValuesRecordCursorFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/table/LatestByValueFilteredRecordCursorFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/table/FilterOnSubQueryRecordCursorFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/table/LatestByValueDeferredIndexedFilteredRecordCursorFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/table/FilterOnValuesRecordCursorFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/table/LatestByDeferredListValuesFilteredRecordCursorFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/table/LatestByValueIndexedFilteredRecordCursorFactory.java 1 1 100.00%
🔵 io/questdb/griffin/engine/table/BaseAsyncHorizonJoinAtom.java 1 1 100.00%
🔵 io/questdb/griffin/engine/join/FilteredAsOfJoinFastRecordCursorFactory.java 1 1 100.00%

@bluestreak01 bluestreak01 merged commit f321e3b into master Feb 24, 2026
44 checks passed
@bluestreak01 bluestreak01 deleted the puzpuzpuz_horizon_join_fixes branch February 24, 2026 13:53
bluestreak01 pushed a commit that referenced this pull request Feb 25, 2026
maciulis pushed a commit to maciulis/questdb that referenced this pull request Mar 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug Incorrect or unexpected behavior SQL Issues or changes relating to SQL execution

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants