Skip to content

Fix 'WEEK' transform function in multi-stage query engine #13483

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
Jackie-Jiang merged 1 commit into from
Jun 26, 2024
Merged

Fix 'WEEK' transform function in multi-stage query engine #13483

Jackie-Jiang merged 1 commit into from
Jun 26, 2024

Conversation

@yashmayya
Copy link
Contributor

@yashmayya yashmayya commented Jun 26, 2024

  • In the v2 multi-stage query engine, numerous date time functions are rewritten to a call to EXTRACT by Calcite.
  • Of all the functions that get re-written to an EXTRACT call, the only one that is not supported by Pinot's extract transformation function is WEEK. This leads to simple queries like SELECT WEEK(ts) FROM airlineStats failing with:
Received error query execution result block: {200=QueryExecutionError:
org.apache.pinot.spi.exception.BadQueryRequestException: Caught exception while initializing transform function: extract
	at org.apache.pinot.core.operator.transform.function.TransformFunctionFactory.get(TransformFunctionFactory.java:332)
	at org.apache.pinot.core.operator.transform.TransformOperator.<init>(TransformOperator.java:56)
	at org.apache.pinot.core.plan.ProjectPlanNode.run(ProjectPlanNode.java:85)
	at org.apache.pinot.core.plan.StreamingSelectionPlanNode.run(StreamingSelectionPlanNode.java:54)
...
Caused by: java.lang.IllegalArgumentException: No enum constant org.apache.pinot.core.operator.transform.function.ExtractTransformFunction.Field.WEEK
	at java.base/java.lang.Enum.valueOf(Enum.java:273)
	at org.apache.pinot.core.operator.transform.function.ExtractTransformFunction$Field.valueOf(ExtractTransformFunction.java:38)
	at org.apache.pinot.core.operator.transform.function.ExtractTransformFunction.init(ExtractTransformFunction.java:54)
	at org.apache.pinot.core.operator.transform.function.BaseTransformFunction.init(BaseTransformFunction.java:120)}
org.apache.pinot.query.service.dispatch.QueryDispatcher.runReducer(QueryDispatcher.java:306)
org.apache.pinot.query.service.dispatch.QueryDispatcher.submitAndReduce(QueryDispatcher.java:96)
org.apache.pinot.broker.requesthandler.MultiStageBrokerRequestHandler.handleRequest(MultiStageBrokerRequestHandler.java:217)
org.apache.pinot.broker.requesthandler.BaseBrokerRequestHandler.handleRequest(BaseBrokerRequestHandler.java:133)

@yashmayya yashmayya added bugfix multi-stage Related to the multi-stage query engine labels Jun 26, 2024
@yashmayya yashmayya requested review from Jackie-Jiang and gortiz June 26, 2024 10:36
@codecov-commenter
Copy link

codecov-commenter commented Jun 26, 2024 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 62.00%. Comparing base (59551e4) to head (6ddb781).
Report is 679 commits behind head on master.

Additional details and impacted files 
@@             Coverage Diff              @@
##             master   #13483      +/-   ##
============================================
+ Coverage     61.75%   62.00%   +0.25%     
+ Complexity      207      198       -9     
============================================
  Files          2436     2560     +124     
  Lines        133233   141361    +8128     
  Branches      20636    21929    +1293     
============================================
+ Hits          82274    87654    +5380     
- Misses        44911    47051    +2140     
- Partials       6048     6656     +608
Flag Coverage Δ
custom-integration1 <0.01% <0.00%> (-0.01%) ⬇️
integration <0.01% <0.00%> (-0.01%) ⬇️
integration1 <0.01% <0.00%> (-0.01%) ⬇️
integration2 0.00% <0.00%> (ø)
java-11 35.37% <100.00%> (-26.34%) ⬇️
java-21 61.88% <100.00%> (+0.25%) ⬆️
skip-bytebuffers-false 61.98% <100.00%> (+0.23%) ⬆️
skip-bytebuffers-true 61.87% <100.00%> (+34.14%) ⬆️
temurin 62.00% <100.00%> (+0.25%) ⬆️
unittests 62.00% <100.00%> (+0.25%) ⬆️
unittests1 46.65% <100.00%> (-0.24%) ⬇️
unittests2 27.53% <0.00%> (-0.20%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Jackie-Jiang Jackie-Jiang merged commit 465a290 into apache:master Jun 26, 2024
@yashmayya yashmayya mentioned this pull request Jun 28, 2024
Merged
suyashpatel98 pushed a commit to suyashpatel98/pinot that referenced this pull request Jul 6, 2024
@yashmayya @suyashpatel98
Fix 'WEEK' transform function in multi-stage query engine (apache#13483)
b7e1ac8
@npawar npawar added the v1v2 label Aug 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gortiz gortiz gortiz approved these changes

@Jackie-Jiang Jackie-Jiang Jackie-Jiang approved these changes

Assignees

No one assigned

Labels

bugfix multi-stage Related to the multi-stage query engine v1v2

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@yashmayya @codecov-commenter @gortiz @Jackie-Jiang @npawar