Skip to content

cut down shimmer-startup benchmark so it can run in benchmarking platform#5613

Merged
BridgeAR merged 3 commits into
masterfrom
bengl/shimmer-startup-cut
Apr 27, 2025
Merged

cut down shimmer-startup benchmark so it can run in benchmarking platform#5613
BridgeAR merged 3 commits into
masterfrom
bengl/shimmer-startup-cut

Conversation

@bengl

@bengl bengl commented Apr 25, 2025

Copy link
Copy Markdown
Collaborator

These benchmarks were taking too long and causing builds to be flaky.

@bengl
bengl requested a review from a team as a code owner April 25, 2025 12:48
@github-actions

github-actions Bot commented Apr 25, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

Overall package size

Self size: 9.31 MB
Deduped: 102.54 MB
No deduping: 103.06 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.5.1 | 29.73 MB | 29.73 MB | | @datadog/native-appsec | 8.5.2 | 19.33 MB | 19.34 MB | | @datadog/native-iast-taint-tracking | 3.3.1 | 13.99 MB | 13.99 MB | | @datadog/pprof | 5.7.1 | 9.51 MB | 9.88 MB | | @opentelemetry/core | 1.30.1 | 908.66 kB | 7.16 MB | | protobufjs | 7.4.0 | 2.77 MB | 5.42 MB | | @datadog/wasm-js-rewriter | 4.0.1 | 2.85 MB | 3.58 MB | | @datadog/native-metrics | 3.1.1 | 1.02 MB | 1.43 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | import-in-the-middle | 1.13.1 | 117.64 kB | 839.26 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.1 | 109.9 kB | 109.9 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.2 | 53.63 kB | 53.63 kB | | istanbul-lib-coverage | 3.2.0 | 29.34 kB | 29.34 kB | | rfdc | 1.4.1 | 27.15 kB | 27.15 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | dc-polyfill | 0.1.8 | 25.08 kB | 25.08 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | shell-quote | 1.8.2 | 23.54 kB | 23.54 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | semifies | 1.0.0 | 15.84 kB | 15.84 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.3 | 4.47 kB | 4.47 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@codecov

codecov Bot commented Apr 25, 2025 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.25%. Comparing base (c1c496d) to head (100dd18).
Report is 1 commits behind head on master.

Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #5613      +/-   ##
==========================================
- Coverage   79.26%   79.25%   -0.01%     
==========================================
  Files         513      513              
  Lines       23412    23412              
==========================================
- Hits        18557    18556       -1     
- Misses       4855     4856       +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Apr 25, 2025 •
edited
Loading

Copy link
Copy Markdown

Datadog Report

Branch report: bengl/shimmer-startup-cut
Commit report: 6445e5a
Test service: dd-trace-js-integration-tests

✅ 0 Failed, 928 Passed, 0 Skipped, 15m 39.11s Total Time

@pr-commenter

pr-commenter Bot commented Apr 25, 2025 •
edited
Loading

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2025-04-25 18:57:48

Comparing candidate commit ba41756 in PR branch bengl/shimmer-startup-cut with baseline commit c1c496d in branch master.

Found 87 performance improvements and 0 performance regressions! Performance is the same for 1180 metrics, 56 unstable metrics.

scenario:shimmer-startup-arrow-wrap-18

  • 🟩 cpu_user_time [-69.550s; -68.410s] or [-95.850%; -94.279%]
  • 🟩 execution_time [-70.387s; -69.260s] or [-95.787%; -94.253%]
  • 🟩 instructions [-366.1G instructions; -365.3G instructions] or [-95.111%; -94.911%]
  • 🟩 max_rss_usage [-1.107GB; -1.077GB] or [-90.265%; -87.809%]

scenario:shimmer-startup-arrow-wrap-20

  • 🟩 cpu_user_time [-62.342s; -61.541s] or [-95.574%; -94.346%]
  • 🟩 execution_time [-63.202s; -62.395s] or [-95.528%; -94.308%]
  • 🟩 instructions [-321.4G instructions; -320.6G instructions] or [-95.045%; -94.832%]
  • 🟩 max_rss_usage [-1.130GB; -1.084GB] or [-91.098%; -87.372%]

scenario:shimmer-startup-arrow-wrap-22

  • 🟩 cpu_user_time [-58.982s; -58.505s] or [-95.428%; -94.655%]
  • 🟩 execution_time [-59.811s; -59.335s] or [-95.361%; -94.602%]
  • 🟩 instructions [-316.4G instructions; -315.5G instructions] or [-95.174%; -94.891%]
  • 🟩 max_rss_usage [-1.098GB; -1.061GB] or [-90.126%; -87.077%]

scenario:shimmer-startup-arrow-wrapfn-18

  • 🟩 cpu_user_time [-6.042s; -5.988s] or [-94.695%; -93.859%]
  • 🟩 execution_time [-6.085s; -6.032s] or [-94.459%; -93.633%]
  • 🟩 instructions [-33.8G instructions; -33.7G instructions] or [-94.731%; -94.345%]

scenario:shimmer-startup-arrow-wrapfn-20

  • 🟩 cpu_user_time [-5.607s; -5.537s] or [-94.945%; -93.772%]
  • 🟩 execution_time [-5.608s; -5.540s] or [-94.703%; -93.567%]
  • 🟩 instructions [-33.7G instructions; -33.5G instructions] or [-94.947%; -94.398%]

scenario:shimmer-startup-arrow-wrapfn-22

  • 🟩 cpu_user_time [-5.225s; -5.134s] or [-95.139%; -93.490%]
  • 🟩 execution_time [-5.227s; -5.138s] or [-94.864%; -93.255%]
  • 🟩 instructions [-31.5G instructions; -31.3G instructions] or [-94.924%; -94.355%]

scenario:shimmer-startup-asyncarrow-wrap-18

  • 🟩 cpu_user_time [-69.494s; -68.859s] or [-95.458%; -94.585%]
  • 🟩 execution_time [-70.310s; -69.679s] or [-95.396%; -94.541%]
  • 🟩 instructions [-369.2G instructions; -368.3G instructions] or [-95.117%; -94.874%]
  • 🟩 max_rss_usage [-1.094GB; -1.067GB] or [-90.078%; -87.865%]

scenario:shimmer-startup-asyncarrow-wrap-20

  • 🟩 cpu_user_time [-62.675s; -61.840s] or [-95.643%; -94.369%]
  • 🟩 execution_time [-63.493s; -62.647s] or [-95.584%; -94.309%]
  • 🟩 instructions [-322.1G instructions; -321.5G instructions] or [-95.035%; -94.866%]
  • 🟩 max_rss_usage [-1.139GB; -1.086GB] or [-91.389%; -87.204%]

scenario:shimmer-startup-asyncarrow-wrap-22

  • 🟩 cpu_user_time [-58.226s; -57.738s] or [-95.447%; -94.647%]
  • 🟩 execution_time [-59.043s; -58.548s] or [-95.390%; -94.589%]
  • 🟩 instructions [-314.3G instructions; -313.4G instructions] or [-95.150%; -94.876%]
  • 🟩 max_rss_usage [-1.109GB; -1.063GB] or [-90.497%; -86.741%]

scenario:shimmer-startup-asyncarrow-wrapfn-18

  • 🟩 cpu_user_time [-6.525s; -6.455s] or [-94.822%; -93.810%]
  • 🟩 execution_time [-6.566s; -6.495s] or [-94.633%; -93.604%]
  • 🟩 instructions [-35.7G instructions; -35.5G instructions] or [-94.815%; -94.323%]

scenario:shimmer-startup-asyncarrow-wrapfn-20

  • 🟩 cpu_user_time [-5.898s; -5.804s] or [-95.095%; -93.582%]
  • 🟩 execution_time [-5.903s; -5.808s] or [-94.925%; -93.395%]
  • 🟩 instructions [-35.4G instructions; -35.3G instructions] or [-94.923%; -94.436%]

scenario:shimmer-startup-asyncarrow-wrapfn-22

  • 🟩 cpu_user_time [-5.500s; -5.460s] or [-94.732%; -94.032%]
  • 🟩 execution_time [-5.502s; -5.463s] or [-94.463%; -93.802%]
  • 🟩 instructions [-33.2G instructions; -33.1G instructions] or [-94.822%; -94.484%]

scenario:shimmer-startup-asyncdeclared-wrap-18

  • 🟩 cpu_user_time [-69.655s; -68.594s] or [-95.764%; -94.305%]
  • 🟩 execution_time [-70.486s; -69.429s] or [-95.698%; -94.262%]
  • 🟩 instructions [-369.2G instructions; -368.6G instructions] or [-95.083%; -94.931%]
  • 🟩 max_rss_usage [-1.124GB; -1.081GB] or [-90.833%; -87.415%]

scenario:shimmer-startup-asyncdeclared-wrap-20

  • 🟩 cpu_user_time [-62.914s; -61.931s] or [-95.771%; -94.275%]
  • 🟩 execution_time [-63.783s; -62.774s] or [-95.735%; -94.221%]
  • 🟩 instructions [-321.6G instructions; -321.2G instructions] or [-95.001%; -94.884%]
  • 🟩 max_rss_usage [-1.127GB; -1.076GB] or [-91.217%; -87.114%]

scenario:shimmer-startup-asyncdeclared-wrap-22

  • 🟩 cpu_user_time [-58.421s; -57.856s] or [-95.496%; -94.572%]
  • 🟩 execution_time [-59.233s; -58.651s] or [-95.449%; -94.512%]
  • 🟩 instructions [-313.9G instructions; -313.3G instructions] or [-95.094%; -94.926%]
  • 🟩 max_rss_usage [-1.067GB; -1.059GB] or [-88.717%; -88.079%]

scenario:shimmer-startup-asyncdeclared-wrapfn-18

  • 🟩 cpu_user_time [-6.491s; -6.435s] or [-94.751%; -93.936%]
  • 🟩 execution_time [-6.536s; -6.481s] or [-94.543%; -93.751%]
  • 🟩 instructions [-35.6G instructions; -35.5G instructions] or [-94.772%; -94.403%]

scenario:shimmer-startup-asyncdeclared-wrapfn-20

  • 🟩 cpu_user_time [-5.915s; -5.827s] or [-95.121%; -93.715%]
  • 🟩 execution_time [-5.915s; -5.825s] or [-94.892%; -93.444%]
  • 🟩 instructions [-35.4G instructions; -35.2G instructions] or [-94.935%; -94.424%]

scenario:shimmer-startup-asyncdeclared-wrapfn-22

  • 🟩 cpu_user_time [-5.509s; -5.435s] or [-95.002%; -93.726%]
  • 🟩 execution_time [-5.513s; -5.437s] or [-94.767%; -93.463%]
  • 🟩 instructions [-33.2G instructions; -33.0G instructions] or [-94.850%; -94.450%]

scenario:shimmer-startup-declared-wrap-18

  • 🟩 cpu_user_time [-68.820s; -68.451s] or [-95.279%; -94.769%]
  • 🟩 execution_time [-69.667s; -69.299s] or [-95.227%; -94.723%]
  • 🟩 instructions [-366.2G instructions; -365.5G instructions] or [-95.089%; -94.917%]
  • 🟩 max_rss_usage [-1.112GB; -1.075GB] or [-90.537%; -87.499%]

scenario:shimmer-startup-declared-wrap-20

  • 🟩 cpu_user_time [-62.256s; -61.241s] or [-95.781%; -94.220%]
  • 🟩 execution_time [-63.095s; -62.067s] or [-95.720%; -94.160%]
  • 🟩 instructions [-321.5G instructions; -320.6G instructions] or [-95.067%; -94.799%]
  • 🟩 max_rss_usage [-1.131GB; -1.078GB] or [-91.343%; -87.125%]

scenario:shimmer-startup-declared-wrap-22

  • 🟩 cpu_user_time [-59.364s; -58.168s] or [-96.072%; -94.136%]
  • 🟩 execution_time [-60.176s; -58.965s] or [-95.996%; -94.063%]
  • 🟩 instructions [-316.2G instructions; -315.4G instructions] or [-95.152%; -94.887%]
  • 🟩 max_rss_usage [-1.108GB; -1.068GB] or [-90.289%; -87.028%]

scenario:shimmer-startup-declared-wrapfn-18

  • 🟩 cpu_user_time [-53.540s; -53.133s] or [-95.346%; -94.621%]
  • 🟩 execution_time [-53.684s; -53.270s] or [-95.319%; -94.584%]
  • 🟩 instructions [-295.6G instructions; -295.1G instructions] or [-95.115%; -94.979%]
  • 🟩 max_rss_usage [-5.949MB; -5.217MB] or [-9.817%; -8.608%]

scenario:shimmer-startup-declared-wrapfn-20

  • 🟩 cpu_user_time [-49.249s; -48.844s] or [-95.342%; -94.558%]
  • 🟩 execution_time [-49.340s; -48.935s] or [-95.311%; -94.528%]
  • 🟩 instructions [-261.1G instructions; -260.5G instructions] or [-95.094%; -94.871%]
  • 🟩 max_rss_usage [-5.344MB; -5.211MB] or [-9.176%; -8.947%]

scenario:shimmer-startup-declared-wrapfn-22

  • 🟩 cpu_user_time [-45.349s; -45.160s] or [-95.195%; -94.797%]
  • 🟩 execution_time [-45.375s; -45.186s] or [-95.155%; -94.759%]
  • 🟩 instructions [-255.7G instructions; -255.1G instructions] or [-95.132%; -94.930%]
  • 🟩 max_rss_usage [-10.118MB; -7.994MB] or [-15.104%; -11.933%]

watson
watson previously approved these changes Apr 25, 2025
View reviewed changes
BridgeAR
BridgeAR reviewed Apr 25, 2025
View reviewed changes
Comment thread benchmark/sirun/shimmer-startup/index.js Outdated
Comment thread benchmark/sirun/shimmer-startup/meta.json Outdated
@bengl @BridgeAR
1e6->1e5 in index.js
ba41756 
Co-authored-by: Ruben Bridgewater <[email protected]>
@bengl @BridgeAR
5->20 in meta.json
100dd18 
Co-authored-by: Ruben Bridgewater <[email protected]>
@BridgeAR
BridgeAR merged commit 26ffd9d into master Apr 27, 2025
@BridgeAR
BridgeAR deleted the bengl/shimmer-startup-cut branch April 27, 2025 01:57
ghost pushed a commit that referenced this pull request Apr 28, 2025
@bengl @watson
cut down shimmer-startup benchmark so it can run in benchmarking plat…
0ac661c 
…form (#5613)
@ghost ghost mentioned this pull request Apr 28, 2025
Merged
watson pushed a commit that referenced this pull request Apr 29, 2025
@bengl @watson
cut down shimmer-startup benchmark so it can run in benchmarking plat…
2225d59 
…form (#5613)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ddyurchenko ddyurchenko ddyurchenko left review comments
@watson watson watson approved these changes
@BridgeAR BridgeAR BridgeAR approved these changes

Assignees

No one assigned

Labels

semver-patch

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@bengl @watson @BridgeAR @ddyurchenko