Skip to content

Conversation

@yashmayya
Copy link
Contributor

@yashmayya yashmayya commented Jun 23, 2024

@yashmayya yashmayya added the multi-stage Related to the multi-stage query engine label Jun 23, 2024
@codecov-commenter
Copy link

codecov-commenter commented Jun 23, 2024

Codecov Report

❌ Patch coverage is 80.00000% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 62.06%. Comparing base (59551e4) to head (bf1120c).
⚠️ Report is 2926 commits behind head on master.

Files with missing lines Patch % Lines
...rg/apache/pinot/common/function/DateTimeUtils.java 86.66% 1 Missing and 1 partial ⚠️
...ot/calcite/rel/rules/PinotEvaluateLiteralRule.java 0.00% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #13463      +/-   ##
============================================
+ Coverage     61.75%   62.06%   +0.31%     
+ Complexity      207      198       -9     
============================================
  Files          2436     2559     +123     
  Lines        133233   141341    +8108     
  Branches      20636    21928    +1292     
============================================
+ Hits          82274    87727    +5453     
- Misses        44911    46950    +2039     
- Partials       6048     6664     +616     
Flag Coverage Δ
custom-integration1 <0.01% <0.00%> (-0.01%) ⬇️
integration <0.01% <0.00%> (-0.01%) ⬇️
integration1 <0.01% <0.00%> (-0.01%) ⬇️
integration2 0.00% <0.00%> (ø)
java-11 61.99% <80.00%> (+0.28%) ⬆️
java-21 61.94% <80.00%> (+0.32%) ⬆️
skip-bytebuffers-false 62.05% <80.00%> (+0.30%) ⬆️
skip-bytebuffers-true 61.87% <80.00%> (+34.14%) ⬆️
temurin 62.06% <80.00%> (+0.31%) ⬆️
unittests 62.06% <80.00%> (+0.31%) ⬆️
unittests1 46.67% <80.00%> (-0.22%) ⬇️
unittests2 27.58% <0.00%> (-0.15%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@yashmayya
Copy link
Contributor Author

CalciteSqlCompilerTest.testExtract is failing with:

Caught exception while invoking method: public static int org.apache.pinot.common.function.scalar.DateTimeFunctions.extract(java.lang.String,long) with arguments: [YEAR, 2017-06-15]

However, the test itself seems a little odd because even before the changes from this PR, while the test passes (i.e., the query compiles to a v1 Pinot query), trying to run an actual query like SELECT EXTRACT(YEAR FROM '2017-06-15') FROM table; results in a similar type related error:

Error Code: 200

QueryExecutionError:
java.lang.IllegalArgumentException: Invalid long value: 2017-06-15
	at org.apache.pinot.common.request.context.LiteralContext.getLongValue(LiteralContext.java:258)
	at org.apache.pinot.core.operator.transform.function.LiteralTransformFunction.getLongLiteral(LiteralTransformFunction.java:65)
	at org.apache.pinot.core.operator.transform.function.LiteralTransformFunction.transformToLongValuesSV(LiteralTransformFunction.java:141)
	at org.apache.pinot.core.operator.transform.function.ExtractTransformFunction.transformToIntValuesSV(ExtractTransformFunction.java:61)

@Jackie-Jiang can the test be updated to use timestamps in the form of milliseconds since epoch or is there some other missing context here?

@yashmayya
Copy link
Contributor Author

The only reason the test wasn't failing earlier was that since there was no scalar function for EXTRACT, the compile time function invoker query rewriter skips evaluation here when the query rewrite rules are being invoked here during the Pinot query compilation. So, the test didn't make much sense since the EXTRACT function never supported timestamp strings (that aren't milliseconds since epoch) in the first place. I've updated the test and also rebased this on top of changes from #13483.

@yashmayya yashmayya marked this pull request as ready for review June 28, 2024 11:28
@yashmayya yashmayya requested a review from Jackie-Jiang June 28, 2024 11:28
@Jackie-Jiang Jackie-Jiang merged commit a0e861e into apache:master Jun 28, 2024
suyashpatel98 pushed a commit to suyashpatel98/pinot that referenced this pull request Jul 6, 2024
@npawar npawar added the v1v2 label Aug 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

multi-stage Related to the multi-stage query engine v1v2

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support EXTRACT as scalar function

4 participants