Skip to content

Conversation

@Jackie-Jiang
Copy link
Contributor

  • Separate server query log and metric emitting into a separate class: ServerQueryLogger
  • Allow logging query and emitting metric for GRPC request and multi-stage leaf stage
  • Bugfixes:
    • Add numResizes into prometheus config
    • Change resizeTimeMs to be a meter instead of gauge
  • Configuration change:
    • pinot.server.query.log.maxRatePerSecond to replace the deprecated pinot.query.scheduler.query.log.maxRatePerSecond (backward compatible)
    • pinot.server.query.log.droppedReportMaxRatePerSecond to configure the dropped log report frequency

Release Notes

New config added for server:

  • pinot.server.query.log.maxRatePerSecond: query log max rate (QPS, default 10K)
  • pinot.server.query.log.droppedReportMaxRatePerSecond: dropped query log report max rate (QPS, default 1)

@Jackie-Jiang Jackie-Jiang added enhancement release-notes Referenced by PRs that need attention when compiling the next release notes Configuration Config changes (addition/deletion/change in behavior) bugfix refactor observability multi-stage Related to the multi-stage query engine labels Oct 20, 2023
@codecov-commenter
Copy link

codecov-commenter commented Oct 20, 2023

Codecov Report

❌ Patch coverage is 9.60000% with 113 lines in your changes missing coverage. Please review.
✅ Project coverage is 66.29%. Comparing base (86d3c44) to head (4b05e5a).
⚠️ Report is 3278 commits behind head on master.

Files with missing lines Patch % Lines
...che/pinot/core/query/logger/ServerQueryLogger.java 2.85% 102 Missing ⚠️
...e/operator/LeafStageTransferableBlockOperator.java 20.00% 2 Missing and 2 partials ⚠️
...che/pinot/core/query/scheduler/QueryScheduler.java 50.00% 1 Missing and 2 partials ⚠️
...che/pinot/core/transport/grpc/GrpcQueryServer.java 0.00% 3 Missing ⚠️
.../java/org/apache/pinot/server/conf/ServerConf.java 75.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #11838      +/-   ##
============================================
- Coverage     66.35%   66.29%   -0.07%     
  Complexity      207      207              
============================================
  Files          2350     2351       +1     
  Lines        127281   127310      +29     
  Branches      19603    19606       +3     
============================================
- Hits          84463    84397      -66     
- Misses        36924    37031     +107     
+ Partials       5894     5882      -12     
Flag Coverage Δ
custom-integration1 <0.01% <0.00%> (ø)
integration <0.01% <0.00%> (ø)
integration1 <0.01% <0.00%> (ø)
integration2 0.00% <0.00%> (ø)
java-11 66.25% <9.60%> (-0.08%) ⬇️
java-21 66.16% <9.60%> (-0.06%) ⬇️
skip-bytebuffers-false 66.27% <9.60%> (-0.08%) ⬇️
skip-bytebuffers-true 66.13% <9.60%> (+48.32%) ⬆️
temurin 66.29% <9.60%> (-0.07%) ⬇️
unittests 66.29% <9.60%> (-0.07%) ⬇️
unittests1 67.05% <6.66%> (-0.07%) ⬇️
unittests2 17.81% <3.20%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Jackie-Jiang Jackie-Jiang merged commit 996f060 into apache:master Oct 21, 2023
@Jackie-Jiang Jackie-Jiang deleted the grpc_query_log branch October 21, 2023 00:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Configuration Config changes (addition/deletion/change in behavior) enhancement multi-stage Related to the multi-stage query engine observability refactor release-notes Referenced by PRs that need attention when compiling the next release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants