Skip to content

fix(sql): fix bugs, resource leaks and NPE in SQL engine#6874

Merged
bluestreak01 merged 3 commits intomasterfrom
vi_misc_fix
Mar 16, 2026
Merged

fix(sql): fix bugs, resource leaks and NPE in SQL engine#6874
bluestreak01 merged 3 commits intomasterfrom
vi_misc_fix

Conversation

@bluestreak01
Copy link
Copy Markdown
Member

@bluestreak01 bluestreak01 commented Mar 14, 2026

Summary

Correctness fixes

  • DECIMAL fall-through to BINARY in UNION casts: generateCastFunctions lacked a break after the DECIMAL inner switch, causing fall-through to case BINARY: that added a spurious BinColumn to the cast function list. In multi-column UNION queries, this shifted subsequent column indices, causing UnsupportedOperationException at runtime.

  • moveClauses never incrementing position counter: when swapJoinOrder0 needed to steal multiple join clauses, only the first was moved. The rest stayed in the wrong join context, creating circular dependencies that broke topological sort and caused query compilation failures on 3+ table cross joins with multi-column WHERE conditions.

  • hasGroupByFunc / hasOrderedGroupByFunc skipping children of non-aggregate functions: the FUNCTION case did break instead of falling through to the default traversal logic, so nested aggregates like abs(sum(x)) were invisible. This caused PIVOT to reject valid aggregate expressions with "expected aggregate function".

Resource leak fixes

  • Const filter leaked in generateJoins: the filter function leaked when it was non-boolean (threw without closing), constant true (abandoned), or constant false (returned without closing).

  • generateSampleBy function leaks: timezoneNameFunc, offsetFunc, sampleFromFunc, and sampleToFunc were created outside the outer try block and never freed on error paths.

Crash fix

  • NPE in CREATE MATERIALIZED VIEW: when WITH BASE specifies a nonexistent table not referenced in the query, getTableTokenIfExists() returned null and .isView() threw NullPointerException instead of a descriptive SqlException.

Latent bug fixes (currently unreachable, fixed for correctness)

  • generateFill always reading first fill value: getQuick(0) instead of getQuick(i) in a loop. Currently gated by the optimizer restricting multi-value FILL to a different code path.

  • GeoHash-to-VARCHAR cast using wrong column type and parameter: all four GEO→VARCHAR casts used toTag instead of fromType. Currently unreachable because UNION type resolution never produces VARCHAR as target for GEO sources.

Known bug documented (fix deferred)

  • subQueryMode nesting corruption: nested parseAsSubQuery calls reset the boolean flag prematurely, causing valid 3-level nested queries to be rejected. Both counter-based and save/restore fixes break 39 other parser tests — a proper fix requires a deeper refactor. Test added to document the issue.

Test plan

  • UnionAllCastTest#testIntDecimalMultiColumn — multi-column INT→DECIMAL UNION ALL; fails with UnsupportedOperationException before fix
  • SqlOptimiserTest#testMoveClausesIncrementsPositionCounter — 3-table CROSS JOIN with multi-column WHERE; fails to compile before fix
  • PivotTest#testPivotWithWrappedAggregateabs(SUM(x)) in PIVOT; rejected before fix
  • PivotTest#testPivotWithCoalesceWrappedAggregatecoalesce(0, SUM(x)) in PIVOT; rejected before fix
  • PivotTest#testPivotWithCoalesceVarArgsWrappedAggregatecoalesce(NULL, NULL, SUM(x)) in PIVOT; rejected before fix
  • CreateMatViewTest#testCreateMatViewBaseTableNoReferenceAndDoesNotExist — NPE before fix
  • SqlParserTest#testSubQueryModeNestedCorruption — documents known bug
  • SampleByTest full suite (283 tests) — verifies generateSampleBy leak fix has no regressions

🤖 Generated with Claude Code

SqlCodeGenerator: add missing break after DECIMAL cases in
generateCastFunctions, preventing fall-through to BINARY that
corrupted the cast function list in multi-column UNION queries.

SqlCodeGenerator: fix generateFill to use getQuick(i) instead of
getQuick(0), so each fill column reads its own fill value expression
rather than always reading the first one.

SqlCodeGenerator: fix GeoHash-to-VARCHAR cast functions to use
GeoByteColumn (not GeoShortColumn) and fromType (not toTag),
matching the correct pattern used by GeoHash-to-STRING casts.

SqlOptimiser: fix moveClauses to increment the position counter
after each matched clause, so all clauses marked for stealing are
actually moved during join reordering. Without this, multi-column
equi-joins on cross-joined tables created circular dependencies
that broke topological sort.

SqlOptimiser: fix hasGroupByFunc and hasOrderedGroupByFunc to
traverse rhs and args of non-aggregate FUNCTION nodes. Previously
these methods only followed lhs, missing nested aggregates like
abs(sum(x)) and causing PIVOT to reject valid expressions.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@bluestreak01 bluestreak01 added Bug Incorrect or unexpected behavior SQL Issues or changes relating to SQL execution labels Mar 14, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 14, 2026

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: a9c65154-28c3-47b4-b812-3ce9c3dd0f7a

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch vi_misc_fix
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Tip

CodeRabbit can suggest fixes for GitHub Check annotations.

Configure the reviews.tools.github-checks setting to adjust the time to wait for GitHub Checks to complete.

RaphDal
RaphDal previously approved these changes Mar 16, 2026
Copy link
Copy Markdown
Contributor

@RaphDal RaphDal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

SqlCodeGenerator: close filter function on all error paths in
generateJoins const-filter handling. The filter leaked when it was
non-boolean (threw without closing), constant true (abandoned), or
constant false (returned without closing).

SqlCodeGenerator: wrap generateSampleBy function allocations in try-catch.
timezoneNameFunc, offsetFunc, sampleFromFunc, and sampleToFunc were
created outside the outer try block and never freed on error paths
from coerceRuntimeConstantType or subsequent factory construction.

SqlParser: add null check for getTableTokenIfExists return value in
parseCreateMatView. When WITH BASE specifies a nonexistent table not
referenced in the query, the method returned null and the subsequent
isView() call threw NullPointerException instead of SqlException.

SqlParserTest: add test documenting known subQueryMode nesting
corruption bug where nested parseAsSubQuery calls reset the flag
prematurely, causing valid queries to be rejected.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@bluestreak01 bluestreak01 changed the title fix(sql): fix five bugs in SQL parser, optimiser and code generator fix(sql): fix bugs, resource leaks and NPE in SQL engine Mar 16, 2026
Fix generateSampleBy indentation after moving the try block up
to wrap function parsing. Remove commented-out capacity overflow
check in DirectLongList. Move shared block comment in PivotTest
into each test method to avoid orphaned section headings.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@glasstiger
Copy link
Copy Markdown
Contributor

[PR Coverage check]

😍 pass : 75 / 80 (93.75%)

file detail

path covered line new line coverage
🔵 io/questdb/griffin/SqlCodeGenerator.java 69 74 93.24%
🔵 io/questdb/griffin/SqlParser.java 2 2 100.00%
🔵 io/questdb/griffin/SqlOptimiser.java 4 4 100.00%

@RaphDal RaphDal self-requested a review March 16, 2026 15:00
@bluestreak01 bluestreak01 merged commit 434f077 into master Mar 16, 2026
51 checks passed
@bluestreak01 bluestreak01 deleted the vi_misc_fix branch March 16, 2026 15:21
@ctkqiang
Copy link
Copy Markdown

Well done

maciulis pushed a commit to maciulis/questdb that referenced this pull request Mar 16, 2026
@onlybugs05
Copy link
Copy Markdown

@bluestreak01 Hey Bro Are You eligible for Swag

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug Incorrect or unexpected behavior SQL Issues or changes relating to SQL execution

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants