Finish window function query planning stage. #15151
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR finalizes the window function query planning stage by introducing comprehensive support for window function analysis, planning, and execution.
- Added new analysis methods and extraction utilities for window functions and expressions.
- Extended the planner, execution operators, and window frame implementations to support window function operations.
Reviewed Changes
Copilot reviewed 52 out of 52 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| StatementAnalyzer.java | Added imports and new methods to analyze window definitions and resolve window functions. |
| ExpressionTreeUtils.java | Introduced static methods to extract window functions and nested window expressions. |
| ExpressionAnalyzer.java | Updated logic to process window function calls and integrated window analysis into expression processing. |
| ExpressionAnalysis.java | Extended analysis result to include window functions. |
| Analysis.java | Added maps and methods to record window definitions, window functions, and order-by window functions. |
| PlanVisitor.java, PlanNodeType.java, PlanGraphPrinter.java | Added new visit methods and node types to support window function planning and visualization. |
| TableOperatorGenerator.java | Integrated the new window function operators into the query execution plan. |
| Various window frame and function files | Updated window frame implementations and window function implementations (NthValue, Lead, Lag, NTile, etc.) to support new window query planning stage. |
| FilterAndProjectOperator.java | Triggered evaluation in column transformers as part of the projection process. |
Comments suppressed due to low confidence (1)
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/process/window/partition/frame/RangeFrame.java:78
- [nitpick] Consider adding parentheses to clarify the operator precedence in this condition. For example: if ((frameInfo.getStartType() == UNBOUNDED_PRECEDING && frameInfo.getEndType() == UNBOUNDED_FOLLOWING) || noOrderBy) {
if (frameInfo.getStartType() == UNBOUNDED_PRECEDING && frameInfo.getEndType() == UNBOUNDED_FOLLOWING || noOrderBy) {
| if (!hasNonMappableUDF) { | ||
| // get result of calculated common sub expressions | ||
| for (ColumnTransformer columnTransformer : commonTransformerList) { | ||
| columnTransformer.tryEvaluate(); |
There was a problem hiding this comment.
Why add this new line? Are there any bugs before? If so, you may need to add IT about it.
There was a problem hiding this comment.
if there is no .tryEvaluate, getColumn may return null.
| builder.writeBoolean(partition.getBoolean(defaultValChannel, index)); | ||
| return; | ||
| case TEXT: | ||
| case STRING: |
| builder.writeBoolean(partition.getBoolean(defaultValChannel, index)); | ||
| return; | ||
| case TEXT: | ||
| case STRING: |
| } | ||
| break; | ||
| case INT64: | ||
| case TIMESTAMP: |
There was a problem hiding this comment.
What about DATE and BLOB
| } | ||
| break; | ||
| case INT64: | ||
| case TIMESTAMP: |
There was a problem hiding this comment.
What about DATE and BLOB
| // ignore | ||
| } | ||
|
|
||
| // now we only support scalar or agg functions |
| if (isAggregation) { | ||
| functionKind = FunctionKind.AGGREGATE; | ||
| } else { | ||
| boolean isWindow = metadata.isWindowFunction(session, functionName, accessControl); |
There was a problem hiding this comment.
what if agg function used in window clause? It's FunctionKind.AGGREGATE or FunctionKind.WINDOW
There was a problem hiding this comment.
If this function is aggregation function, we will still warp it into a window function instance.
|
|
||
| import static java.util.Objects.requireNonNull; | ||
|
|
||
| public class DataOrganizationSpecification { |
There was a problem hiding this comment.
deduplicated with org.apache.iotdb.db.queryengine.plan.relational.planner.DataOrganizationSpecification
| public class QueryPlanner { | ||
| private final Analysis analysis; | ||
| private final SymbolAllocator symbolAllocator; | ||
| private final QueryId idAllocator; |
There was a problem hiding this comment.
why not using queryIdAllocator?
| .map(argument -> coercions.get(argument).toSymbolReference()) | ||
| .collect(toImmutableList()), | ||
| frame, | ||
| // TODO: remove ignore null |
There was a problem hiding this comment.
we should never have todo, fix it
There was a problem hiding this comment.
Looks like we already support IGNORE NULLS keyword. So this TODO is meanningless.
|
PTAL~ |
2 participants
As per title.