Skip to content

fix(sql): fix AssertionError that could be triggered by JOIN SQL#6824

Merged
bluestreak01 merged 5 commits intoquestdb:masterfrom
DHRUV6029:fix/filtered-factory-double-wrap
Mar 6, 2026
Merged

fix(sql): fix AssertionError that could be triggered by JOIN SQL#6824
bluestreak01 merged 5 commits intoquestdb:masterfrom
DHRUV6029:fix/filtered-factory-double-wrap

Conversation

@DHRUV6029
Copy link
Copy Markdown
Contributor

Fixes #6762

Summary

When a JOIN query has a WHERE clause that contains both a column-referencing condition and a non-column constant expression, QuestDB crashes with an internal AssertionError instead of returning results.

Reproducer query:

SELECT 
    T1.event, 
    T1.origin, 
    CASE WHEN T1.event > 50 THEN 'High' ELSE 'Low' END 
FROM telemetry T1 
INNER JOIN telemetry T2 ON T1.created < T2.created 
WHERE (cast(T1.origin as SYMBOL)) IS NULL AND NOW() = NOW() 
ORDER BY (cast(T1.origin as SYMBOL)) NOT IN ('{}') 
LIMIT NULL;

Error:

java.lang.AssertionError
    at io.questdb.griffin.engine.table.FilteredRecordCursorFactory.<init>(FilteredRecordCursorFactory.java:44)
    at io.questdb.griffin.SqlCodeGenerator.generateJoins(SqlCodeGenerator.java:4254)

Root Cause

The bug is in SqlCodeGenerator.generateJoins(). During query compilation, the SQL optimizer splits the WHERE clause into two buckets:

Condition Bucket Reason
(cast(T1.origin as SYMBOL)) IS NULL postJoinWhereClause References column T1.origin
NOW() = NOW() constWhereClause Has no column references, but is not compile-time constant (NOW() is runtime-evaluated)

The code generator then applies each bucket as a separate FilteredRecordCursorFactory wrapper:

Step 1 (SqlCodeGenerator.java:4195): Wraps the join result with the postJoin filter:

master = new FilteredRecordCursorFactory(master, postJoinFilter);
// master is now a FilteredRecordCursorFactory

Step 2 (SqlCodeGenerator.java:4254): Tries to wrap again with the const filter:

master = new FilteredRecordCursorFactory(master, constFilter);
// CRASH: master is already a FilteredRecordCursorFactory

This violates the assertion at FilteredRecordCursorFactory.java:44:

assert !(base instanceof FilteredRecordCursorFactory);

The assertion exists by design — nesting FilteredRecordCursorFactory instances is inefficient and the codebase expects filters to be combined, not stacked.

The Fix

3 files changed, 61 lines added:

1. FilteredRecordCursorFactory.java — Added getFilter() and halfClose() overrides

  • getFilter() exposes the filter so it can be extracted and combined.
  • halfClose() follows the established "filter stealing" lifecycle pattern — closes internal state (the cursor) while leaving base and filter alive for reuse. This is the same pattern used by AsyncFilteredRecordCursorFactory.halfClose() and referenced at 7 other locations in SqlCodeGenerator.java.

2. SqlCodeGenerator.java:4254 — Combine filters instead of nesting

Before creating a new FilteredRecordCursorFactory, check if master is already one. If so, extract the existing filter and base, combine both filters with AND using AndFunctionFactory, and create a single factory.

Instead of:

FilteredRecordCursorFactory(FilteredRecordCursorFactory(join, filterA), filterB)  ← CRASH

It now produces:

FilteredRecordCursorFactory(join, filterA AND filterB)  ← correct

3. JoinTest.java — Regression test

Added testJoinInnerPostJoinAndConstFilter that reproduces the exact scenario: a JOIN with both a column-referencing WHERE condition (T1.val > 0) and a non-constant expression (NOW() = NOW()).

Test plan

  • JoinTest#testJoinInnerPostJoinAndConstFilter (new regression test) passes
  • Full JoinTest suite (171 tests) passes with 0 failures
  • Manual test on master (no fix): AssertionError crash confirmed
  • Manual test on fix branch: query returns correct results, 0 errors in server logs

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Feb 26, 2026

Walkthrough

The PR introduces a centralized filter-composition mechanism in SqlCodeGenerator via a new private wrapWithFilter() helper that safely combines filters using AndFunctionFactory. This replaces nested FilteredRecordCursorFactory instances across three code paths in filter and join handling. FilteredRecordCursorFactory is enhanced with public getFilter() and halfClose() methods for filter access and resource management. Two regression tests validate the fix for post-join filter scenarios.

Changes

Cohort / File(s) Summary
Filter Composition Logic
core/src/main/java/io/questdb/griffin/SqlCodeGenerator.java
Added private wrapWithFilter() method to safely combine filters via AndFunctionFactory. Updated generateFilter0 and two paths in generateJoins to use the new wrapper instead of directly constructing nested FilteredRecordCursorFactory instances. Imported AndFunctionFactory and added private static field.
FilteredRecordCursorFactory API
core/src/main/java/io/questdb/griffin/engine/table/FilteredRecordCursorFactory.java
Added public getFilter() accessor and public halfClose() method for resource lifecycle management. No changes to core getCursor logic.
Regression Tests
core/src/test/java/io/questdb/test/griffin/engine/join/JoinTest.java
Added testJoinInnerPostJoinAndConstFilter and testJoinInnerPostJoinMultipleJoinsFilter to validate that post-join filters are properly combined into a single filter instead of being nested.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested labels

Bug, SQL

Suggested reviewers

  • bluestreak01
  • mtopolnik
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 6.25% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description check ✅ Passed The description comprehensively explains the bug, provides the reproducer query, documents the root cause, and details the fix with clear examples and test coverage.
Linked Issues check ✅ Passed The PR successfully addresses all coding requirements from issue #6762: prevents nested FilteredRecordCursorFactory creation by combining filters with AND logic, fixes the AssertionError crash, and enables queries with both column-referencing and runtime-evaluated conditions to return results correctly.
Out of Scope Changes check ✅ Passed All changes are directly scoped to fixing the AssertionError: FilteredRecordCursorFactory gains filter-exposure methods (getFilter/halfClose), SqlCodeGenerator implements the wrapWithFilter helper and applies it at three FilteredRecordCursorFactory creation sites, and JoinTest adds a regression test.
Title check ✅ Passed The title directly references the bug being fixed (AssertionError on JOIN) which is the main objective of the changeset, but omits the specific cause (post-join and const WHERE filters) mentioned in the commit message.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@DHRUV6029 DHRUV6029 marked this pull request as ready for review February 26, 2026 06:28
@DHRUV6029
Copy link
Copy Markdown
Contributor Author

@nwoolmer @bluestreak01 would you be able to take a look when you get a chance? Happy to make any changes needed.

Thanks

@glasstiger
Copy link
Copy Markdown
Contributor

glasstiger commented Mar 4, 2026

@DHRUV6029

Comments from Claude Code:


  1. Incomplete fix — same bug exists in the post-join filter loop (Medium severity)

  The post-join filter wrapping at line 5041 runs inside a loop (lines 4311-5046). For a multi-table join like:

  SELECT ... FROM A
  INNER JOIN B ON ... WHERE pred_referencing_B
  INNER JOIN C ON ... WHERE pred_referencing_C

  If parallel filter is disabled and both joins have postJoinWhereClause, iteration 1 wraps master in a FilteredRecordCursorFactory at line 5041, and iteration 2 tries to wrap it again
  — triggering the same assertion. The PR only fixes the const-filter-on-top-of-post-join-filter case but not the post-join-on-top-of-post-join case.

  Similarly, generateFilter0() at line 3290 can wrap the result of generateJoins() in yet another FilteredRecordCursorFactory if there's an outer WHERE clause and parallel filter is
  unavailable.

  Recommendation: Apply the same instanceof FilteredRecordCursorFactory check at line 5041, or — better — extract the combine-filters logic into a helper method and use it at all three
  sites.

  2. Exception safety gap between halfClose() and new factory creation (Low severity)

  existingFiltered.halfClose();                    // line 5110 - cursor freed
  // ... allocations happen here ...
  filter = new AndFunctionFactory().newInstance(    // line 5117 - if this threw...
      0, andArgs, argPositions, configuration, executionContext
  );
  master = new FilteredRecordCursorFactory(base, filter);  // line 5120

  After halfClose(), base and existingFilter are orphaned — no factory owns them. If the code between lines 5110-5120 threw, both would leak. Currently AndFunctionFactory.newInstance()
  cannot throw in practice, but the method signature declares throws SqlException. If the implementation ever changes, this becomes a real leak.

  Recommendation: Extract base and filter first, build the combined filter, and only then call halfClose() — or wrap the section in a try/catch that re-closes base and existingFilter on
   failure.

  3. halfClose() doesn't fully follow the lifecycle pattern (Nitpick)

  The existing halfClose() call sites in SqlCodeGenerator follow a consistent pattern with supportsFilterStealing():
  if (factory.supportsFilterStealing()) {
      filter = factory.getFilter();
      base = factory.getBaseFactory();
      factory.halfClose();
      ...
  }

  The PR doesn't add supportsFilterStealing() to FilteredRecordCursorFactory and instead uses instanceof. This works but is inconsistent with the existing pattern. The instanceof check
  is arguably clearer here since this is a specific code path, not the generic filter-stealing mechanism.

  4. Direct instantiation of AndFunctionFactory (Minor style concern)

  filter = new AndFunctionFactory().newInstance(
      0, andArgs, argPositions, configuration, executionContext
  );

  This allocates a new AndFunctionFactory instance just to call newInstance(). While cheap, QuestDB's codebase generally avoids unnecessary allocations. Consider whether the
  functionParser pipeline should be used instead, or at minimum make the factory a static constant.

  Test Coverage

  - The new test testJoinInnerPostJoinAndConstFilter reproduces the exact reported scenario and verifies both no-crash and correct results. Good.
  - Uses assertQueryNoLeakCheck — consistent with the rest of JoinTest.java (all 132 assertions use this variant), so this is fine by convention.

  Missing test coverage:
  - No test for the multi-join nesting scenario (issue #1 above)
  - No test verifying the query plan shows a single combined Filter node rather than nested filters
  - No test with NOW() != NOW() or other non-trivial const expressions to exercise edge cases in the AndFunctionFactory constant-folding paths

  Summary

  ┌─────────────────────────────────────────────────────────────────────┬──────────┐
  │                               Finding                               │ Severity │
  ├─────────────────────────────────────────────────────────────────────┼──────────┤
  │ Same nesting bug exists in the post-join filter loop (line 5041)    │ Medium   │
  ├─────────────────────────────────────────────────────────────────────┼──────────┤
  │ Exception safety gap after halfClose()                              │ Low      │
  ├─────────────────────────────────────────────────────────────────────┼──────────┤
  │ Missing supportsFilterStealing() override (uses instanceof instead) │ Nitpick  │
  ├─────────────────────────────────────────────────────────────────────┼──────────┤
  │ Unnecessary AndFunctionFactory allocation                           │ Nitpick  │
  ├─────────────────────────────────────────────────────────────────────┼──────────┤
  │ Missing tests for multi-join nesting                                │ Medium   │
  └─────────────────────────────────────────────────────────────────────┴──────────┘

  The fix is correct and well-explained for the reported issue. The main concern is that it's a partial fix — the same root problem (nesting FilteredRecordCursorFactory) can be
  triggered via at least one other code path (successive post-join filters in multi-table joins). I'd recommend extracting the combine-and-wrap logic into a helper and applying it at
  all new FilteredRecordCursorFactory(master, ...) sites to make the fix comprehensive.

@DHRUV6029
Copy link
Copy Markdown
Contributor Author

Thanks for the review , will address these comments

Thanks

@DHRUV6029 DHRUV6029 force-pushed the fix/filtered-factory-double-wrap branch from 24ae2e7 to bb64f2d Compare March 5, 2026 06:58
When a JOIN query has a WHERE clause that the optimizer splits into both
a postJoinWhereClause (column-referencing) and a constWhereClause
(non-column-referencing, non-constant like NOW() = NOW()), the code
generator in generateJoins() wraps the master factory in a
FilteredRecordCursorFactory twice. The second wrapping violates the
assertion at FilteredRecordCursorFactory:44 that forbids nesting,
causing an internal server error.

The fix extracts a wrapWithFilter() helper that detects when the factory
is already a FilteredRecordCursorFactory and combines both filters into
a single AND filter using AndFunctionFactory, following the established
filter-stealing lifecycle pattern (getFilter + getBaseFactory + halfClose).
Applied at all three FilteredRecordCursorFactory creation sites for
comprehensive coverage.

Fixes questdb#6762
@DHRUV6029 DHRUV6029 force-pushed the fix/filtered-factory-double-wrap branch from bb64f2d to 45d5981 Compare March 5, 2026 07:17
@DHRUV6029
Copy link
Copy Markdown
Contributor Author

@glasstiger addressed your comments please take a look

Thanks

@glasstiger
Copy link
Copy Markdown
Contributor

@CodeRabbit review the PR

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 5, 2026

@glasstiger Sure, I'll review the PR now!


🧠 Learnings used
Learnt from: javier
Repo: questdb/questdb PR: 0
File: :0-0
Timestamp: 2026-02-06T18:22:28.130Z
Learning: In questdb/questdb, for PRs requested by javier, provide line-cited, push-back style reviews with concrete follow-ups (tests, diffs, and performance considerations).

Learnt from: javier
Repo: questdb/questdb PR: 0
File: :0-0
Timestamp: 2026-02-06T18:22:28.130Z
Learning: For questdb/questdb, javier expects a rigorous, push-back style review; always include concrete line-cited findings and propose targeted tests/fixes when reviewing javier’s PRs.

Learnt from: jerrinot
Repo: questdb/questdb PR: 0
File: :0-0
Timestamp: 2026-02-04T11:18:36.492Z
Learning: For PRs by jerrinot, always use a thorough and critical review mode: question design decisions, challenge assumptions, scrutinize implementations for edge cases and potential issues, and avoid superficial validation. They prefer hard-core reviews that push back on the code.

Learnt from: bluestreak01
Repo: questdb/questdb PR: 6673
File: core/src/test/java/io/questdb/test/griffin/WindowJoinTest.java:2616-2618
Timestamp: 2026-01-19T21:43:19.228Z
Learning: In test `testWindowJoinFailsWhenSlaveDoesNotSupportTimeFrames` in `core/src/test/java/io/questdb/test/griffin/WindowJoinTest.java`, the use of `AND` between timestamp equality conditions (e.g., `ts = 'X' AND ts = 'Y'`) is intentional to prevent intrinsic interval creation. This ensures the window join fails as expected when the slave doesn't support time frames, which is necessary after PR `#6673` added OR timestamp intrinsics recognition.

Learnt from: CR
Repo: questdb/questdb PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-04T02:38:24.210Z
Learning: Applies to **/test/**/*.java : Use assertQueryNoLeakCheck() to assert the results of queries

Learnt from: javier
Repo: questdb/questdb PR: 6830
File: core/src/main/java/io/questdb/griffin/SqlOptimiser.java:3538-3609
Timestamp: 2026-03-02T17:04:59.384Z
Learning: In questdb/questdb, SqlParser enforces the maximum number of UNNEST column aliases (including WITH ORDINALITY) and throws at the first excess alias token; downstream components like SqlOptimiser.enumerateUnnestColumns() should assume this invariant and not revalidate alias counts.
✅ Actions performed

Full review triggered.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
core/src/main/java/io/questdb/griffin/engine/table/FilteredRecordCursorFactory.java (1)

55-58: Move getFilter() to preserve required member ordering.

getFilter() is currently placed before getCursor(). With alphabetical ordering for public instance members, getCursor() should come first, so this new method should be repositioned.

As per coding guidelines "**/*.java: Group Java class members by kind (static vs. instance) and visibility, sorted alphabetically. When adding new methods or fields, insert them in the correct alphabetical position among existing members of the same kind."

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@core/src/main/java/io/questdb/griffin/engine/table/FilteredRecordCursorFactory.java`
around lines 55 - 58, The public instance method getFilter() in class
FilteredRecordCursorFactory is out of alphabetical order (it currently precedes
getCursor()); move the entire getFilter() method so that getCursor() appears
before getFilter() among the class's public instance members, preserving the
method body and signature exactly and ensuring the class's public instance
members remain alphabetically ordered.
core/src/test/java/io/questdb/test/griffin/engine/join/JoinTest.java (1)

2866-2935: Reorder these new test methods to keep class-member alphabetical ordering consistent.

testJoinInnerPostJoinAndConstFilter should be placed with the testJoinInnerPostJoin... methods in strict alphabetical order (it currently appears after testJoinInnerPostJoinFilterFF).

As per coding guidelines: Group Java class members by kind (static vs. instance) and visibility, sorted alphabetically. When adding new methods or fields, insert them in the correct alphabetical position among existing members of the same kind.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@core/src/test/java/io/questdb/test/griffin/engine/join/JoinTest.java` around
lines 2866 - 2935, The two new test methods are out of alphabetical order; move
testJoinInnerPostJoinAndConstFilter so it appears in strict alphabetical order
among the other testJoinInnerPostJoin* methods (i.e., place it before
testJoinInnerPostJoinFilterFF) inside the JoinTest class; ensure you only
reorder the methods (preserve their bodies, annotations, and imports) so
class-member grouping and alphabetical sorting rules are maintained.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@core/src/main/java/io/questdb/griffin/SqlCodeGenerator.java`:
- Around line 9008-9009: The call to existingFiltered.halfClose() currently
happens before constructing the wrapper, which can leak ownership if new
FilteredRecordCursorFactory(base, filter) throws; move or defer the halfClose()
call until after the FilteredRecordCursorFactory has been successfully created
(i.e., construct new FilteredRecordCursorFactory(base, filter) first, then call
existingFiltered.halfClose()), ensuring that ownership of base/filter is only
released after the wrapper exists; if necessary, create the wrapper into a local
variable first and then invoke existingFiltered.halfClose() before returning it
so exceptions do not leave resources unowned.

---

Nitpick comments:
In
`@core/src/main/java/io/questdb/griffin/engine/table/FilteredRecordCursorFactory.java`:
- Around line 55-58: The public instance method getFilter() in class
FilteredRecordCursorFactory is out of alphabetical order (it currently precedes
getCursor()); move the entire getFilter() method so that getCursor() appears
before getFilter() among the class's public instance members, preserving the
method body and signature exactly and ensuring the class's public instance
members remain alphabetically ordered.

In `@core/src/test/java/io/questdb/test/griffin/engine/join/JoinTest.java`:
- Around line 2866-2935: The two new test methods are out of alphabetical order;
move testJoinInnerPostJoinAndConstFilter so it appears in strict alphabetical
order among the other testJoinInnerPostJoin* methods (i.e., place it before
testJoinInnerPostJoinFilterFF) inside the JoinTest class; ensure you only
reorder the methods (preserve their bodies, annotations, and imports) so
class-member grouping and alphabetical sorting rules are maintained.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 7cdcfff2-9bad-48bd-8858-6ddf27824e37

📥 Commits

Reviewing files that changed from the base of the PR and between db3ec2c and 0a0ab62.

📒 Files selected for processing (3)
  • core/src/main/java/io/questdb/griffin/SqlCodeGenerator.java
  • core/src/main/java/io/questdb/griffin/engine/table/FilteredRecordCursorFactory.java
  • core/src/test/java/io/questdb/test/griffin/engine/join/JoinTest.java

glasstiger and others added 2 commits March 5, 2026 16:14
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
@bluestreak01 bluestreak01 changed the title Fix AssertionError on JOIN with post-join and const WHERE filters fix(sql): fix AssertionError that could be triggered by JOIN SQL Mar 6, 2026
@bluestreak01 bluestreak01 merged commit 2263b2a into questdb:master Mar 6, 2026
31 checks passed
maciulis pushed a commit to maciulis/questdb that referenced this pull request Mar 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AssertionError at FilteredRecordCursorFactory.java

3 participants