fix(sql): NullPointerException when right-hand query in window join has timestamp filter#6806
Conversation
…as timestamp filter
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Use the checkbox below for a quick retry:
WalkthroughThis PR refactors the time-frame cursor architecture by converting ConcurrentTimeFrameCursor from a concrete class to a public interface, introducing ConcurrentTimeFrameCursorImpl as its implementation, and systematically propagating timestamp index information through cursor initialization. Additionally, CairoConfiguration parameters are removed from AbstractPageFrameRecordCursorFactory and related subclasses. Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested labels
Suggested reviewers
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
core/src/main/java/io/questdb/griffin/engine/table/SelectedRecordCursorFactory.java (1)
135-144:⚠️ Potential issue | 🟠 MajorAdd defensive
instanceofcheck before casting toTablePageFrameCursor.The cast at line 143 assumes
base.getPageFrameCursor()always returns aTablePageFrameCursor, but implementations likeReadParquetPageFrameCursorandRegisteredPageFrameCursorimplementPageFrameCursordirectly without table backing. This will cause aClassCastExceptionif a non-table-backed cursor is returned.Additionally,
SelectedPageFrameCursor.of(PartitionFrameCursor, ...)intentionally throwsUnsupportedOperationException(lines 377-378) since initialization occurs throughwrap()instead. Guard thewrap()call with aninstanceofcheck to ensure the cursor is actuallyTablePageFrameCursor, or returnbaseCursorunchanged if it is not.Same issue applies at lines 373-379.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@core/src/main/java/io/questdb/griffin/engine/table/SelectedRecordCursorFactory.java` around lines 135 - 144, The getPageFrameCursor method assumes base.getPageFrameCursor() returns a TablePageFrameCursor and casts unconditionally; add a defensive instanceof check before casting and calling SelectedPageFrameCursor.wrap so non-table-backed cursors (e.g., ReadParquetPageFrameCursor, RegisteredPageFrameCursor) are returned unchanged. Specifically, in SelectedRecordCursorFactory.getPageFrameCursor, verify baseCursor instanceof TablePageFrameCursor before calling pageFrameCursor.wrap((TablePageFrameCursor) baseCursor) and return baseCursor if not; apply the same pattern where SelectedPageFrameCursor.of/.wrap is used (including the PartitionFrameCursor initialization path) to avoid ClassCastException and the UnsupportedOperationException from incorrect initialization.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In
`@core/src/main/java/io/questdb/griffin/engine/table/ConcurrentTimeFrameCursorImpl.java`:
- Around line 49-67: Reorder the private instance fields in class
ConcurrentTimeFrameCursorImpl so they are alphabetized by name while keeping
their modifiers, initializers and associated comments intact; specifically
arrange frameCount, frameCursor, frameMemoryPool, framePartitionIndexes,
frameRowCounts, frameTimestampCache, partitionCeilings, partitionTimestamps,
record, timeFrame, and timestampIndex into alphabetical order within the private
instance group and preserve the "// Cursor's lifecycle..." and timestamp-index
comment blocks adjacent to their respective fields.
In
`@core/src/main/java/io/questdb/griffin/engine/table/SelectedRecordCursorFactory.java`:
- Around line 188-195: In SelectedRecordCursorFactory, relocate the public
instance method newTimeFrameCursor() so public methods remain alphabetically
ordered: move the newTimeFrameCursor() method to sit between isProjection() and
recordCursorSupportsLongTopK(); keep the method body unchanged (it constructs a
SelectedConcurrentTimeFrameCursor when crossedIndex is true) and ensure
imports/visibility remain the same.
In `@core/src/test/java/io/questdb/test/griffin/HorizonJoinTest.java`:
- Around line 639-671: Update the SQL literals passed into the
assertQueryNoLeakCheck calls so keywords use uppercase IN and the SQL is written
as a Java text block (multiline string) instead of concatenated string
expressions; specifically modify the two queries used in the HorizonJoinTest's
assertQueryNoLeakCheck calls (the HORIZON JOIN query and the variant with
"SELECT price, price price0, sym, ts FROM prices WHERE ts in '1970-01-01'") to
replace "in" with "IN" and convert the concatenated/quoted SQL to a single
"""...""" text block each, keeping the same expressions for getSecondsDivisor()
and the surrounding Java code (these changes affect the SQL literals used by
SelectedConcurrentTimeFrameCursor test paths).
In `@core/src/test/java/io/questdb/test/griffin/WindowJoinTest.java`:
- Around line 3390-3448: The test method
testWindowJoinSelfJoinWithAggregatesInSelectAndWhere uses long concatenated
string literals for the CREATE/INSERT and the SELECT+expected result; change
these to Java multiline text blocks ("""...""") to improve readability and match
test SQL formatting rules—replace the concatenated execute("INSERT INTO
fx_trades VALUES " + "...") call and the long "SELECT " + "t.timestamp, " + ...
string passed into assertQueryNoLeakCheck, as well as the expected result block,
with proper text blocks while keeping calls to execute(...) and
assertQueryNoLeakCheck(...) and their argument order unchanged.
- Around line 3450-3527: Update the testWindowJoinSelfJoinWithVwapCalculation
test to follow formatting rules: replace large numeric literals 10000 and
2000000 with underscore-separated forms (e.g., 10_000, 2_000_000) in the INSERT
expression and any setProperty values if applicable, and convert the long SQL
string assignments (the "insert into fx_trades ..." and the failing query
assigned to query) to Java text blocks (multiline string literals) to improve
readability; edit the string variables used in engine.execute(...) and the query
variable inside testWindowJoinSelfJoinWithVwapCalculation and keep all semantics
unchanged.
---
Outside diff comments:
In
`@core/src/main/java/io/questdb/griffin/engine/table/SelectedRecordCursorFactory.java`:
- Around line 135-144: The getPageFrameCursor method assumes
base.getPageFrameCursor() returns a TablePageFrameCursor and casts
unconditionally; add a defensive instanceof check before casting and calling
SelectedPageFrameCursor.wrap so non-table-backed cursors (e.g.,
ReadParquetPageFrameCursor, RegisteredPageFrameCursor) are returned unchanged.
Specifically, in SelectedRecordCursorFactory.getPageFrameCursor, verify
baseCursor instanceof TablePageFrameCursor before calling
pageFrameCursor.wrap((TablePageFrameCursor) baseCursor) and return baseCursor if
not; apply the same pattern where SelectedPageFrameCursor.of/.wrap is used
(including the PartitionFrameCursor initialization path) to avoid
ClassCastException and the UnsupportedOperationException from incorrect
initialization.
ℹ️ Review info
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (24)
core/src/main/java/io/questdb/griffin/engine/QueryProgress.javacore/src/main/java/io/questdb/griffin/engine/join/AsyncWindowJoinAtom.javacore/src/main/java/io/questdb/griffin/engine/join/FilteredAsOfJoinFastRecordCursorFactory.javacore/src/main/java/io/questdb/griffin/engine/join/FilteredAsOfJoinNoKeyFastRecordCursorFactory.javacore/src/main/java/io/questdb/griffin/engine/table/AbstractDeferredValueRecordCursorFactory.javacore/src/main/java/io/questdb/griffin/engine/table/AbstractPageFrameRecordCursorFactory.javacore/src/main/java/io/questdb/griffin/engine/table/AbstractTreeSetRecordCursorFactory.javacore/src/main/java/io/questdb/griffin/engine/table/BaseAsyncHorizonJoinAtom.javacore/src/main/java/io/questdb/griffin/engine/table/ConcurrentTimeFrameCursor.javacore/src/main/java/io/questdb/griffin/engine/table/ConcurrentTimeFrameCursorImpl.javacore/src/main/java/io/questdb/griffin/engine/table/ExtraNullColumnCursorFactory.javacore/src/main/java/io/questdb/griffin/engine/table/FilterOnExcludedValuesRecordCursorFactory.javacore/src/main/java/io/questdb/griffin/engine/table/FilterOnSubQueryRecordCursorFactory.javacore/src/main/java/io/questdb/griffin/engine/table/FilterOnValuesRecordCursorFactory.javacore/src/main/java/io/questdb/griffin/engine/table/LatestByDeferredListValuesFilteredRecordCursorFactory.javacore/src/main/java/io/questdb/griffin/engine/table/LatestByValueDeferredFilteredRecordCursorFactory.javacore/src/main/java/io/questdb/griffin/engine/table/LatestByValueDeferredIndexedFilteredRecordCursorFactory.javacore/src/main/java/io/questdb/griffin/engine/table/LatestByValueFilteredRecordCursorFactory.javacore/src/main/java/io/questdb/griffin/engine/table/LatestByValueIndexedFilteredRecordCursorFactory.javacore/src/main/java/io/questdb/griffin/engine/table/PageFrameRecordCursorFactory.javacore/src/main/java/io/questdb/griffin/engine/table/SelectedRecordCursorFactory.javacore/src/main/java/io/questdb/griffin/engine/table/SortedSymbolIndexRecordCursorFactory.javacore/src/test/java/io/questdb/test/griffin/HorizonJoinTest.javacore/src/test/java/io/questdb/test/griffin/WindowJoinTest.java
💤 Files with no reviewable changes (1)
- core/src/main/java/io/questdb/griffin/engine/table/AbstractPageFrameRecordCursorFactory.java
core/src/main/java/io/questdb/griffin/engine/table/ConcurrentTimeFrameCursorImpl.java
Show resolved
Hide resolved
core/src/main/java/io/questdb/griffin/engine/table/SelectedRecordCursorFactory.java
Outdated
Show resolved
Hide resolved
[PR Coverage check]😍 pass : 134 / 191 (70.16%) file detail
|
…as timestamp filter (#6806)
…as timestamp filter (questdb#6806)
WINDOW JOIN and HORIZON JOIN crash with a NullPointerException when the right-hand side (slave) is a subquery that applies a timestamp interval filter or reorders columns, e.g.:
The root cause is that
SelectedRecordCursorFactoryandExtraNullColumnCursorFactorydid not implementnewTimeFrameCursor(), which is required by the concurrent time frame cursor infrastructure used in these joins. Additionally,SelectedPageFrameCursordid not implementTablePageFrameCursor, causing a ClassCastException.After fixing those, a subtler double column remapping bug remained: both
SelectedConcurrentTimeFrameCursorandSelectedPageFrameCursorwere independently remapping column indices throughcolumnCrossIndex, causing symbol table lookups to hit wrong columns (returning null) and the timestamp column to be read from the wrong position in the address cache.Changes
ConcurrentTimeFrameCursorinto an interface andConcurrentTimeFrameCursorImplimplementation class.newTimeFrameCursor()toSelectedRecordCursorFactory,ExtraNullColumnCursorFactory, andQueryProgress.SelectedPageFrameCursorandExtraNullColumnPageFrameCursorimplementTablePageFrameCursor.timestampIndexparameter toConcurrentTimeFrameCursor.of()so wrappers can specify the correct timestamp column position in the (potentially remapped) address cache.SelectedConcurrentTimeFrameCursorto delegate column remapping entirely toSelectedPageFrameCursor, avoiding double remapping.