Skip to content

RUM-8450: Add metrics for internal benchmarking#2581

Merged
jonathanmos merged 6 commits into
developfrom
jmoskovich/rum-8450/benchmark-metrics
Apr 10, 2025
Merged

RUM-8450: Add metrics for internal benchmarking#2581
jonathanmos merged 6 commits into
developfrom
jmoskovich/rum-8450/benchmark-metrics

Conversation

@jonathanmos

@jonathanmos jonathanmos commented Apr 3, 2025 •
edited
Loading

Copy link
Copy Markdown
Member

What does this PR do?

Adds metrics to benchmarking to align with ios.

The metrics that are being added (all metrics sampled every 10 seconds and collected per track):
• bytes_written - number of bytes written whenever a batch is written to disk.
• bytes_uploaded - number of bytes uploaded when a batch is sent.
• bytes_deleted - number of bytes deleted when a batch is removed.
• batch_count - number of batch files in the directory.
• upload_count - number of uploads - this is actual uploads, not upload triggers.
• response_latency - time for upload network requests to execute.

Motivation

Align metrics with ios as part of the effort to improve upload reliability.

Additional Notes

Anything else we should know when reviewing?

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Make sure you discussed the feature or bugfix with the maintaining team in an Issue
  • Make sure each commit and the PR mention the Issue number (cf the CONTRIBUTING doc)

@jonathanmos
jonathanmos force-pushed the jmoskovich/rum-8450/benchmark-metrics branch from b4d090f to 9208d89 Compare April 3, 2025 12:10
@jonathanmos jonathanmos changed the title RUM-8450: Add metrics for benchmarking RUM-8450: Add metrics for internal benchmarking Apr 3, 2025
@codecov-commenter

codecov-commenter commented Apr 3, 2025 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 68.42105% with 36 lines in your changes missing coverage. Please review.

Project coverage is 70.16%. Comparing base (dfb2ae3) to head (e38dee6).
Report is 195 commits behind head on develop.

Files with missing lines Patch % Lines
...adog/android/internal/profiler/DDExecutionTimer.kt 0.00% 16 Missing ⚠️
...tadog/android/internal/profiler/GlobalBenchmark.kt 0.00% 8 Missing ⚠️
.../android/core/internal/metrics/BenchmarkUploads.kt 83.33% 4 Missing ⚠️
...id/core/internal/data/upload/DataOkHttpUploader.kt 88.89% 3 Missing ⚠️
...og/android/internal/profiler/NoOpExecutionTimer.kt 0.00% 2 Missing ⚠️
...in/com/datadog/android/core/internal/SdkFeature.kt 92.86% 1 Missing ⚠️
...d/core/internal/persistence/ConsentAwareStorage.kt 90.91% 0 Missing and 1 partial ⚠️
...adog/android/internal/profiler/BenchmarkSpanExt.kt 0.00% 1 Missing ⚠️
Additional details and impacted files 
@@             Coverage Diff             @@
##           develop    #2581      +/-   ##
===========================================
- Coverage    70.18%   70.16%   -0.01%     
===========================================
  Files          809      812       +3     
  Lines        30317    30399      +82     
  Branches      5081     5086       +5     
===========================================
+ Hits         21275    21329      +54     
- Misses        7623     7649      +26     
- Partials      1419     1421       +2
Files with missing lines Coverage Δ
...id/core/internal/data/upload/DataUploadRunnable.kt 100.00% <100.00%> (+3.51%) ⬆️
.../core/internal/persistence/FileEventBatchWriter.kt 97.62% <100.00%> (+0.12%) ⬆️
...in/com/datadog/android/core/internal/SdkFeature.kt 91.13% <92.86%> (+0.03%) ⬆️
...d/core/internal/persistence/ConsentAwareStorage.kt 96.04% <90.91%> (-0.70%) ⬇️
...adog/android/internal/profiler/BenchmarkSpanExt.kt 0.00% <0.00%> (ø)
...og/android/internal/profiler/NoOpExecutionTimer.kt 0.00% <0.00%> (ø)
...id/core/internal/data/upload/DataOkHttpUploader.kt 90.09% <88.89%> (-0.65%) ⬇️
.../android/core/internal/metrics/BenchmarkUploads.kt 83.33% <83.33%> (ø)
...tadog/android/internal/profiler/GlobalBenchmark.kt 0.00% <0.00%> (ø)
...adog/android/internal/profiler/DDExecutionTimer.kt 0.00% <0.00%> (ø)

... and 26 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jonathanmos
jonathanmos marked this pull request as ready for review April 7, 2025 07:14
@jonathanmos
jonathanmos requested review from a team as code owners April 7, 2025 07:14
ambushwork
ambushwork reviewed Apr 7, 2025
View reviewed changes
Comment thread ...oid-core/src/main/kotlin/com/datadog/android/core/internal/data/upload/DataUploadRunnable.kt Outdated
Comment thread ...id-core/src/main/kotlin/com/datadog/android/core/internal/persistence/ConsentAwareStorage.kt Outdated
Comment thread tools/benchmark/src/main/java/com/datadog/benchmark/profiler/DDBenchmarkMeter.kt
Comment thread tools/benchmark/src/main/java/com/datadog/benchmark/DatadogSdkMeter.kt
Comment thread tools/benchmark/src/main/java/com/datadog/benchmark/DatadogSdkMeter.kt Outdated
Comment thread sample/benchmark/src/main/java/com/datadog/benchmark/sample/benchmark/SyntheticsRun.kt Outdated
Comment thread dd-sdk-android-internal/src/main/java/com/datadog/android/internal/profiler/DDExecutionTimer.kt Outdated
0xnm
0xnm reviewed Apr 7, 2025
View reviewed changes

@0xnm 0xnm left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall looks good, I've added few questions

Comment thread dd-sdk-android-core/src/main/kotlin/com/datadog/android/core/internal/SdkFeature.kt
Comment thread ...oid-core/src/main/kotlin/com/datadog/android/core/internal/data/upload/DataUploadRunnable.kt Outdated
Comment thread ...id-core/src/main/kotlin/com/datadog/android/core/internal/persistence/ConsentAwareStorage.kt
Comment thread ...ore/src/test/kotlin/com/datadog/android/core/internal/persistence/ConsentAwareStorageTest.kt Outdated
Comment thread dd-sdk-android-internal/src/main/java/com/datadog/android/internal/profiler/DDExecutionTimer.kt Outdated
@jonathanmos
jonathanmos requested review from 0xnm and ambushwork April 8, 2025 14:12
@jonathanmos
jonathanmos force-pushed the jmoskovich/rum-8450/benchmark-metrics branch from a7e3c1f to 27265c1 Compare April 9, 2025 13:26
@jonathanmos
jonathanmos requested a review from ambushwork April 9, 2025 14:22
0xnm
0xnm previously approved these changes Apr 10, 2025
View reviewed changes
Comment thread dd-sdk-android-core/src/main/kotlin/com/datadog/android/core/internal/SdkFeature.kt Outdated
Comment thread ...-android-internal/src/main/java/com/datadog/android/internal/profiler/BenchmarkSdkUploads.kt Outdated
Comment thread dd-sdk-android-internal/src/main/java/com/datadog/android/internal/profiler/DDExecutionTimer.kt Outdated
Comment thread tools/benchmark/src/main/java/com/datadog/benchmark/profiler/DDBenchmarkMeter.kt Outdated
Comment thread tools/benchmark/src/test/java/internal/MetricRequestBodyBuilderTest.kt Outdated
ambushwork
ambushwork previously approved these changes Apr 10, 2025
View reviewed changes
@jonathanmos
jonathanmos dismissed stale reviews from ambushwork and 0xnm via 91ae133 April 10, 2025 10:50
@jonathanmos
jonathanmos force-pushed the jmoskovich/rum-8450/benchmark-metrics branch from 91ae133 to e38dee6 Compare April 10, 2025 11:20
@jonathanmos
jonathanmos merged commit 22f6b22 into develop Apr 10, 2025
@jonathanmos
jonathanmos deleted the jmoskovich/rum-8450/benchmark-metrics branch April 10, 2025 12:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@0xnm 0xnm 0xnm approved these changes
@ambushwork ambushwork ambushwork left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@jonathanmos @codecov-commenter @0xnm @ambushwork