Skip to content

benchmarks: Use 1 Warmup Interation and ensure opcache is used#2531

Merged
PROFeNoM merged 1 commit intomasterfrom
alex/experiments/web-bench-warmups
Feb 23, 2024
Merged

benchmarks: Use 1 Warmup Interation and ensure opcache is used#2531
PROFeNoM merged 1 commit intomasterfrom
alex/experiments/web-bench-warmups

Conversation

@PROFeNoM
Copy link
Copy Markdown
Contributor

@PROFeNoM PROFeNoM commented Feb 21, 2024 •
edited
Loading

Description

Native profiles when:

These profiles assess that opcache is indeed used in web requests. What's more, the warmups seemingly responsible for improvements in both kinds of benchmarks, because the first run will be longer since DD_AUTOLOAD_NO_COMPILE is used.

Reviewer checklist

  • Test coverage seems ok.
  • Appropriate labels assigned.

@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Feb 22, 2024 •
edited
Loading

Benchmarks

Benchmark execution time: 2024-02-23 11:22:48

Comparing candidate commit 4257a5e in PR branch alex/experiments/web-bench-warmups with baseline commit ab2dd7d in branch master.

Found 35 performance improvements and 2 performance regressions! Performance is the same for 144 metrics, 1 unstable metrics.

scenario:ContextPropagationBench/benchExtractHeaders128Bit

  • 🟩 execution_time [-56.335ns; -28.998ns] or [-6.273%; -3.229%]

scenario:ContextPropagationBench/benchExtractHeaders128Bit-opcache

  • 🟩 execution_time [-84.318ns; -41.015ns] or [-6.008%; -2.923%]

scenario:ContextPropagationBench/benchExtractHeaders64Bit

  • 🟩 execution_time [-130.436ns; -91.564ns] or [-9.927%; -6.968%]

scenario:ContextPropagationBench/benchExtractHeaders64Bit-opcache

  • 🟩 execution_time [-157.791ns; -112.209ns] or [-11.846%; -8.424%]

scenario:ContextPropagationBench/benchExtractTraceContext128Bit

  • 🟩 execution_time [-136.200ns; -93.800ns] or [-5.950%; -4.098%]

scenario:ContextPropagationBench/benchExtractTraceContext128Bit-opcache

  • 🟩 execution_time [-168.737ns; -77.263ns] or [-7.311%; -3.348%]

scenario:ContextPropagationBench/benchExtractTraceContext64Bit

  • 🟩 execution_time [-70.410ns; -47.590ns] or [-7.992%; -5.402%]

scenario:ContextPropagationBench/benchExtractTraceContext64Bit-opcache

  • 🟩 execution_time [-67.493ns; -50.507ns] or [-7.670%; -5.739%]

scenario:ContextPropagationBench/benchInject128Bit-opcache

  • 🟩 execution_time [-188.861ns; -117.139ns] or [-6.735%; -4.178%]

scenario:EmptyFileBench/benchEmptyFileBaseline

  • 🟩 execution_time [-1.524ms; -1.320ms] or [-40.479%; -35.048%]

scenario:EmptyFileBench/benchEmptyFileBaseline-opcache

  • 🟩 execution_time [-1.593ms; -1.395ms] or [-40.197%; -35.212%]

scenario:EmptyFileBench/benchEmptyFileOverhead

  • 🟩 execution_time [-6.341ms; -6.108ms] or [-63.166%; -60.850%]

scenario:EmptyFileBench/benchEmptyFileOverhead-opcache

  • 🟩 execution_time [-6.257ms; -6.058ms] or [-61.968%; -59.998%]

scenario:LaravelBench/benchLaravelBaseline

  • 🟩 execution_time [-1.629ms; -1.395ms] or [-40.607%; -34.780%]

scenario:LaravelBench/benchLaravelBaseline-opcache

  • 🟩 execution_time [-1.634ms; -1.419ms] or [-39.245%; -34.084%]

scenario:LaravelBench/benchLaravelOverhead

  • 🟩 execution_time [-6.536ms; -6.285ms] or [-62.872%; -60.458%]

scenario:LaravelBench/benchLaravelOverhead-opcache

  • 🟩 execution_time [-6.504ms; -6.315ms] or [-62.104%; -60.292%]

scenario:LogsInjectionBench/benchLogsInfoBaseline

  • 🟩 execution_time [-140.569ns; -108.431ns] or [-9.954%; -7.678%]

scenario:LogsInjectionBench/benchLogsInfoBaseline-opcache

  • 🟩 execution_time [-214.875ns; -144.725ns] or [-11.675%; -7.863%]

scenario:LogsInjectionBench/benchLogsInfoInjection-opcache

  • 🟩 execution_time [-517.093ns; -201.507ns] or [-5.137%; -2.002%]

scenario:LogsInjectionBench/benchLogsNullBaseline-opcache

  • 🟩 execution_time [-1.199µs; -1.091µs] or [-5.071%; -4.613%]

scenario:LogsInjectionBench/benchLogsNullInjection

  • 🟩 execution_time [-2.035µs; -1.253µs] or [-3.811%; -2.346%]

scenario:LogsInjectionBench/benchLogsNullInjection-opcache

  • 🟩 execution_time [-2.389µs; -2.295µs] or [-9.604%; -9.224%]

scenario:MessagePackSerializationBench/benchMessagePackSerialization

  • 🟩 execution_time [-7.699µs; -5.461µs] or [-4.962%; -3.519%]

scenario:MessagePackSerializationBench/benchMessagePackSerialization-opcache

  • 🟩 execution_time [-7.822µs; -6.458µs] or [-5.014%; -4.140%]

scenario:SamplingRuleMatchingBench/benchRegexMatching3-opcache

  • 🟥 execution_time [+1.024µs; +1.276µs] or [+16.684%; +20.792%]

scenario:SamplingRuleMatchingBench/benchRegexMatching4-opcache

  • 🟥 execution_time [+0.988µs; +1.241µs] or [+16.184%; +20.321%]

scenario:SymfonyBench/benchSymfonyBaseline

  • 🟩 execution_time [-14.518ms; -14.278ms] or [-58.987%; -58.009%]

scenario:SymfonyBench/benchSymfonyBaseline-opcache

  • 🟩 execution_time [-14.450ms; -14.285ms] or [-58.601%; -57.932%]

scenario:SymfonyBench/benchSymfonyOverhead

  • 🟩 execution_time [-19.177ms; -18.964ms] or [-60.970%; -60.295%]

scenario:SymfonyBench/benchSymfonyOverhead-opcache

  • 🟩 execution_time [-19.159ms; -19.014ms] or [-60.762%; -60.303%]

scenario:WordPressBench/benchEnhancedWordPressOverhead

  • 🟩 execution_time [-55.715ms; -53.784ms] or [-21.179%; -20.445%]

scenario:WordPressBench/benchEnhancedWordPressOverhead-opcache

  • 🟩 execution_time [-54.781ms; -53.148ms] or [-20.856%; -20.234%]

scenario:WordPressBench/benchWordPressBaseline

  • 🟩 execution_time [-42.885ms; -42.317ms] or [-66.072%; -65.197%]

scenario:WordPressBench/benchWordPressBaseline-opcache

  • 🟩 execution_time [-42.479ms; -41.995ms] or [-65.680%; -64.931%]

scenario:WordPressBench/benchWordPressOverhead

  • 🟩 execution_time [-56.222ms; -53.934ms] or [-21.406%; -20.535%]

scenario:WordPressBench/benchWordPressOverhead-opcache

  • 🟩 execution_time [-56.863ms; -54.138ms] or [-21.619%; -20.583%]

@PROFeNoM PROFeNoM changed the title Use 1 Warmup Interation + apache2handler (+ opcache) benchmarks: Use 1 Warmup Interation and ensure opcache is used Feb 22, 2024
@PROFeNoM PROFeNoM force-pushed the alex/experiments/web-bench-warmups branch from db770e4 to a47228d Compare February 22, 2024 14:11
@PROFeNoM PROFeNoM marked this pull request as ready for review February 22, 2024 14:45
@PROFeNoM PROFeNoM requested a review from a team as a code owner February 22, 2024 14:45
bwoebi
bwoebi approved these changes Feb 22, 2024
View reviewed changes
Copy link
Copy Markdown
Collaborator

@bwoebi bwoebi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see 20%-60% perf improvement, I approve!

@PROFeNoM PROFeNoM force-pushed the alex/experiments/web-bench-warmups branch from a47228d to f027eeb Compare February 23, 2024 08:18
@PROFeNoM PROFeNoM marked this pull request as draft February 23, 2024 10:03
@PROFeNoM PROFeNoM force-pushed the alex/experiments/web-bench-warmups branch from f027eeb to 4257a5e Compare February 23, 2024 10:51
@PROFeNoM PROFeNoM self-assigned this Feb 23, 2024
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Feb 23, 2024

Codecov Report

Merging #2531 (4257a5e) into master (ab2dd7d) will increase coverage by 2.19%.
Report is 1 commits behind head on master.
The diff coverage is n/a.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #2531      +/-   ##
============================================
+ Coverage     76.55%   78.74%   +2.19%     
  Complexity      267      267              
============================================
  Files           138      112      -26     
  Lines         17459    13485    -3974     
  Branches        976        0     -976     
============================================
- Hits          13365    10619    -2746     
+ Misses         3574     2866     -708     
+ Partials        520        0     -520
Flag Coverage Δ
appsec-extension ?
tracer-extension 78.70% <ø> (ø)
tracer-integrations 79.49% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

see 26 files with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ab2dd7d...4257a5e. Read the comment docs.

@PROFeNoM PROFeNoM marked this pull request as ready for review February 23, 2024 12:39
@PROFeNoM PROFeNoM requested a review from bwoebi February 23, 2024 12:39
@PROFeNoM PROFeNoM merged commit 527e4d9 into master Feb 23, 2024
@PROFeNoM PROFeNoM deleted the alex/experiments/web-bench-warmups branch February 23, 2024 15:05
@github-actions github-actions Bot added this to the 0.99.0 milestone Feb 23, 2024
@PROFeNoM PROFeNoM mentioned this pull request Feb 26, 2024
Merged
2 tasks
bwoebi pushed a commit that referenced this pull request Mar 20, 2024
@PROFeNoM @bwoebi
Add warmups everywhere applicable (#2531)
35d2b5d 
(cherry picked from commit 527e4d9)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bwoebi bwoebi Awaiting requested review from bwoebi

Assignees

@PROFeNoM PROFeNoM

Labels

cat:benchmarks

Projects

None yet

Milestone

0.99.0

Development

Successfully merging this pull request may close these issues.

3 participants

@PROFeNoM @codecov-commenter @bwoebi