You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit be40091
Browse filesBrowse the repository at this point in the historyBrowse files
[fix](nereids) Fix deep copy of scans with virtual columns
### What problem does this PR solve?
Issue Number: None
Related PR: #69292
Problem Summary:
A CTE fails to plan when the optimizer inlines it after one of its scans got
a virtual column:
WITH c AS (
SELECT id, score() AS s FROM t WHERE c1 MATCH 'apple'
ORDER BY s DESC LIMIT 2
)
SELECT * FROM c;
-- A expression contains slot not from children
A score or ANN TopN pushed into a scan, and a MATCH in the select list, add a
virtual column to the scan. After rewrite, the optimizer inlines a CTE that
has one consumer left (Optimizer.normalizeCtePlan) by deep copying its
rewritten body, so the copy includes these scans:
- score() and l2_distance_approximate() are nondeterministic, so the rewrite
does not inline a CTE that calls them, even with one consumer. Once the
TopN pushdown has replaced them with a virtual column of the scan, the
optimizer inlines the CTE.
- A CTE with MATCH in its select list reaches the optimizer when the rewrite
removes all but one of its consumers, e.g. a UNION ALL branch with
WHERE 1 = 0.
LogicalPlanDeepCopier copied a scan with virtual columns wrongly:
- it gave each virtual column two new ids: the slots above the scan were
mapped to the first one, and the copied scan output the second one;
- the copied scan still output the original virtual column, with its
original id, next to the new one;
- the score and ANN order keys of the copied scan still used the original
slots.
So the plan above the copied scan read a slot that the scan does not output.
The copier now gives each virtual column one new id, maps the original id to
it and maps the score and ANN order keys to the new slots. Relations without
virtual columns are copied as before.
LogicalOlapScan.withVirtualColumns replaces the virtual columns of the scan,
but kept their old slots in the output next to the new ones. It now replaces
them there too and keeps the other output slots, which the copier has
already mapped. Computing the output of the copy again would not work: it
gives VARIANT subpath slots new ids. AdjustNullable, the other caller that
replaces existing virtual columns, no longer keeps two slots for a virtual
column whose nullability changes.
### Release note
Fix "A expression contains slot not from children" for a CTE that orders by
score() or by an ANN distance with a LIMIT, or that has MATCH in its select
list and is inlined after rewrite.
### Check List (For Author)
- Test: Unit Test and Regression test
- Unit tests: LogicalPlanDeepCopierTest.testDeepCopyOlapScanWithVirtualColumn
and CTEInlineTest.inlineCteWhoseScanHasVirtualColumns (CTEs with a score
TopN, an ANN TopN, a MATCH projection, and a VARIANT subpath with a MATCH
projection, with one consumer and with a second consumer removed by
WHERE 1 = 0). Both fail on master (4 output slots instead of 3; "A
expression contains slot not from children") and pass with this change.
OrExpansionTest, SplitMultiDistinctTest,
DecomposeRepeatWithPreAggregationTest, LogicalOlapScanTest,
IvmDeltaRewriteHelperTest and PushDownVirtualColumnsIntoOlapScanTest
pass. Checkstyle passes.
- Regression test: nereids_rules_p0/cte/test_cte_inline_virtual_column,
with its .out generated by the framework, on an FE built from this
branch and a BE built from 41a58af plus #69292's SegmentIterator
change (this change is FE only). The VARIANT case is only in the unit
test, because that BE reads v['x'] as NULL even without a CTE.
- Behavior changed: No
- Does this need documentation: No
Co-Authored-By: Claude Opus 5.5 <[email protected]>
0 commit comments