Skip to content

Have JFR setting event for native-image recording#6348

Merged
jbachorik merged 4 commits into
masterfrom
jb/native_image_jfr_setting
Dec 13, 2023
Merged

Have JFR setting event for native-image recording#6348
jbachorik merged 4 commits into
masterfrom
jb/native_image_jfr_setting

Conversation

@jbachorik

@jbachorik jbachorik commented Dec 12, 2023 •
edited by atlassian Bot
Loading

Copy link
Copy Markdown
Contributor

What Does This Do

It provides a convenient way for the recording processors to tell the originating JFR implementation.

Motivation

To make it easier for the recording processors to decide how to handle a recording.
Eg. the native-image generated JFRs are using itimer for the ExecutionSample events and as such don't need upscaling samples to CPU time via thread CPU load.

Additional Notes

Jira ticket: PROF-8784

@pr-commenter

pr-commenter Bot commented Dec 12, 2023 •
edited
Loading

Copy link
Copy Markdown

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jb/native_image_jfr_setting
git_commit_date 1702461697 1702462973
git_commit_sha e26f1a4 725e1b3
release_version 1.26.0-SNAPSHOT~e26f1a4abf 1.26.0-SNAPSHOT~725e1b31bd
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1702466399 1702466399
ci_job_id 388641944 388641944
ci_pipeline_id 25006498 25006498
cpu_model Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 2 performance regressions! Performance is the same for 44 metrics, 8 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:petclinic:profiling:ProfilingAgent worse
[+10.681ms; +22.161ms] or [+7.851%; +16.289%]		 152.472ms 136.051ms
scenario:startup:petclinic:profiling:Profiling worse
[+10.686ms; +22.183ms] or [+7.851%; +16.298%]		 152.543ms 136.109ms
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.26.0-SNAPSHOT~725e1b31bd, baseline=1.26.0-SNAPSHOT~e26f1a4abf

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.567 s) : 0, 1566752
Total [baseline] (12.2 s) : 0, 12199766
Agent [candidate] (1.567 s) : 0, 1567473
Total [candidate] (12.243 s) : 0, 12243244
section iast
Agent [baseline] (1.721 s) : 0, 1720745
Total [baseline] (12.93 s) : 0, 12930000
Agent [candidate] (1.719 s) : 0, 1718831
Total [candidate] (12.925 s) : 0, 12924565
section iast_TELEMETRY_OFF
Agent [baseline] (1.715 s) : 0, 1715327
Total [baseline] (12.938 s) : 0, 12938097
Agent [candidate] (1.716 s) : 0, 1715793
Total [candidate] (12.949 s) : 0, 12949129
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.567 s -
Agent iast 1.721 s 153.993 ms (9.8%)
Agent iast_TELEMETRY_OFF 1.715 s 148.575 ms (9.5%)
Total tracing 12.2 s -
Total iast 12.93 s 730.234 ms (6.0%)
Total iast_TELEMETRY_OFF 12.938 s 738.331 ms (6.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.567 s -
Agent iast 1.719 s 151.358 ms (9.7%)
Agent iast_TELEMETRY_OFF 1.716 s 148.32 ms (9.5%)
Total tracing 12.243 s -
Total iast 12.925 s 681.321 ms (5.6%)
Total iast_TELEMETRY_OFF 12.949 s 705.885 ms (5.8%) 
gantt
    title insecure-bank - break down per module: candidate=1.26.0-SNAPSHOT~725e1b31bd, baseline=1.26.0-SNAPSHOT~e26f1a4abf

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (968.927 ms) : 0, 968927
BytebuddyAgent [candidate] (970.754 ms) : 0, 970754
GlobalTracer [baseline] (458.361 ms) : 0, 458361
GlobalTracer [candidate] (456.627 ms) : 0, 456627
AppSec [baseline] (68.864 ms) : 0, 68864
AppSec [candidate] (69.416 ms) : 0, 69416
Remote Config [baseline] (845.886 µs) : 0, 846
Remote Config [candidate] (849.089 µs) : 0, 849
Telemetry [baseline] (9.419 ms) : 0, 9419
Telemetry [candidate] (9.409 ms) : 0, 9409
section iast
BytebuddyAgent [baseline] (1.139 s) : 0, 1139265
BytebuddyAgent [candidate] (1.138 s) : 0, 1137990
GlobalTracer [baseline] (421.266 ms) : 0, 421266
GlobalTracer [candidate] (420.78 ms) : 0, 420780
AppSec [baseline] (65.963 ms) : 0, 65963
AppSec [candidate] (66.62 ms) : 0, 66620
IAST [baseline] (22.241 ms) : 0, 22241
IAST [candidate] (22.977 ms) : 0, 22977
Remote Config [baseline] (763.048 µs) : 0, 763
Remote Config [candidate] (765.89 µs) : 0, 766
Telemetry [baseline] (10.778 ms) : 0, 10778
Telemetry [candidate] (9.313 ms) : 0, 9313
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (1.13 s) : 0, 1130042
BytebuddyAgent [candidate] (1.132 s) : 0, 1131716
GlobalTracer [baseline] (422.307 ms) : 0, 422307
GlobalTracer [candidate] (422.246 ms) : 0, 422246
AppSec [baseline] (67.252 ms) : 0, 67252
AppSec [candidate] (66.366 ms) : 0, 66366
IAST [baseline] (21.962 ms) : 0, 21962
IAST [candidate] (23.271 ms) : 0, 23271
Remote Config [baseline] (789.835 µs) : 0, 790
Remote Config [candidate] (809.773 µs) : 0, 810
Telemetry [baseline] (12.322 ms) : 0, 12322
Telemetry [candidate] (10.724 ms) : 0, 10724
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.26.0-SNAPSHOT~725e1b31bd, baseline=1.26.0-SNAPSHOT~e26f1a4abf

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.562 s) : 0, 1561567
Total [baseline] (12.848 s) : 0, 12848250
Agent [candidate] (1.572 s) : 0, 1572277
Total [candidate] (12.911 s) : 0, 12910893
section appsec
Agent [baseline] (1.701 s) : 0, 1701148
Total [baseline] (12.96 s) : 0, 12959886
Agent [candidate] (1.709 s) : 0, 1709448
Total [candidate] (13.054 s) : 0, 13054467
section iast
Agent [baseline] (1.72 s) : 0, 1719850
Total [baseline] (13.326 s) : 0, 13326117
Agent [candidate] (1.728 s) : 0, 1728117
Total [candidate] (13.285 s) : 0, 13285272
section profiling
Agent [baseline] (1.868 s) : 0, 1867550
Total [baseline] (13.228 s) : 0, 13228447
Agent [candidate] (1.891 s) : 0, 1890936
Total [candidate] (13.258 s) : 0, 13257952
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.562 s -
Agent appsec 1.701 s 139.581 ms (8.9%)
Agent iast 1.72 s 158.283 ms (10.1%)
Agent profiling 1.868 s 305.983 ms (19.6%)
Total tracing 12.848 s -
Total appsec 12.96 s 111.636 ms (0.9%)
Total iast 13.326 s 477.866 ms (3.7%)
Total profiling 13.228 s 380.197 ms (3.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.572 s -
Agent appsec 1.709 s 137.171 ms (8.7%)
Agent iast 1.728 s 155.84 ms (9.9%)
Agent profiling 1.891 s 318.658 ms (20.3%)
Total tracing 12.911 s -
Total appsec 13.054 s 143.574 ms (1.1%)
Total iast 13.285 s 374.379 ms (2.9%)
Total profiling 13.258 s 347.059 ms (2.7%) 
gantt
    title petclinic - break down per module: candidate=1.26.0-SNAPSHOT~725e1b31bd, baseline=1.26.0-SNAPSHOT~e26f1a4abf

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (967.114 ms) : 0, 967114
BytebuddyAgent [candidate] (972.271 ms) : 0, 972271
GlobalTracer [baseline] (455.468 ms) : 0, 455468
GlobalTracer [candidate] (460.288 ms) : 0, 460288
AppSec [baseline] (68.885 ms) : 0, 68885
AppSec [candidate] (68.974 ms) : 0, 68974
Remote Config [baseline] (842.574 µs) : 0, 843
Remote Config [candidate] (848.04 µs) : 0, 848
Telemetry [baseline] (9.29 ms) : 0, 9290
Telemetry [candidate] (9.386 ms) : 0, 9386
section appsec
BytebuddyAgent [baseline] (968.786 ms) : 0, 968786
BytebuddyAgent [candidate] (975.948 ms) : 0, 975948
GlobalTracer [baseline] (457.81 ms) : 0, 457810
GlobalTracer [candidate] (456.968 ms) : 0, 456968
AppSec [baseline] (203.828 ms) : 0, 203828
AppSec [candidate] (205.178 ms) : 0, 205178
Remote Config [baseline] (796.184 µs) : 0, 796
Remote Config [candidate] (802.271 µs) : 0, 802
Telemetry [baseline] (9.916 ms) : 0, 9916
Telemetry [candidate] (9.993 ms) : 0, 9993
section iast
BytebuddyAgent [baseline] (1.139 s) : 0, 1138566
BytebuddyAgent [candidate] (1.144 s) : 0, 1144474
GlobalTracer [baseline] (420.91 ms) : 0, 420910
GlobalTracer [candidate] (422.178 ms) : 0, 422178
AppSec [baseline] (66.47 ms) : 0, 66470
AppSec [candidate] (66.418 ms) : 0, 66418
Remote Config [baseline] (759.473 µs) : 0, 759
Remote Config [candidate] (766.576 µs) : 0, 767
Telemetry [baseline] (10.751 ms) : 0, 10751
Telemetry [candidate] (9.999 ms) : 0, 9999
IAST [baseline] (22.238 ms) : 0, 22238
IAST [candidate] (23.673 ms) : 0, 23673
section profiling
BytebuddyAgent [baseline] (989.803 ms) : 0, 989803
BytebuddyAgent [candidate] (995.453 ms) : 0, 995453
GlobalTracer [baseline] (571.047 ms) : 0, 571047
GlobalTracer [candidate] (572.079 ms) : 0, 572079
AppSec [baseline] (70.022 ms) : 0, 70022
AppSec [candidate] (70.024 ms) : 0, 70024
Remote Config [baseline] (860.591 µs) : 0, 861
Remote Config [candidate] (849.849 µs) : 0, 850
Telemetry [baseline] (9.754 ms) : 0, 9754
Telemetry [candidate] (9.769 ms) : 0, 9769
ProfilingAgent [baseline] (136.051 ms) : 0, 136051
ProfilingAgent [candidate] (152.472 ms) : 0, 152472
Profiling [baseline] (136.109 ms) : 0, 136109
Profiling [candidate] (152.543 ms) : 0, 152543
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2023-12-13T10:51:48 2023-12-13T11:14:26
git_branch master jb/native_image_jfr_setting
git_commit_date 1702461697 1702462973
git_commit_sha e26f1a4 725e1b3
release_version 1.26.0-SNAPSHOT~e26f1a4abf 1.26.0-SNAPSHOT~725e1b31bd
start_time 2023-12-13T10:51:32 2023-12-13T11:14:10
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1702466399 1702466399
ci_job_id 388641944 388641944
ci_pipeline_id 25006498 25006498
cpu_model Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 5 metrics, 17 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.26.0-SNAPSHOT~725e1b31bd, baseline=1.26.0-SNAPSHOT~e26f1a4abf
    dateFormat X
    axisFormat %s
section baseline
no_agent (442.3 µs) : 413, 471
.   : milestone, 442,
iast (577.907 µs) : 547, 609
.   : milestone, 578,
iast_FULL (662.774 µs) : 633, 693
.   : milestone, 663,
iast_INACTIVE (543.793 µs) : 513, 575
.   : milestone, 544,
iast_TELEMETRY_OFF (568.392 µs) : 538, 599
.   : milestone, 568,
tracing (535.747 µs) : 505, 567
.   : milestone, 536,
section candidate
no_agent (461.944 µs) : 433, 491
.   : milestone, 462,
iast (575.463 µs) : 544, 607
.   : milestone, 575,
iast_FULL (665.692 µs) : 634, 697
.   : milestone, 666,
iast_INACTIVE (541.295 µs) : 511, 572
.   : milestone, 541,
iast_TELEMETRY_OFF (577.223 µs) : 546, 609
.   : milestone, 577,
tracing (532.415 µs) : 503, 562
.   : milestone, 532,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 442.3 µs [413.375 µs, 471.225 µs] -
iast 577.907 µs [546.832 µs, 608.982 µs] 135.607 µs (30.7%)
iast_FULL 662.774 µs [632.721 µs, 692.826 µs] 220.473 µs (49.8%)
iast_INACTIVE 543.793 µs [512.611 µs, 574.975 µs] 101.493 µs (22.9%)
iast_TELEMETRY_OFF 568.392 µs [537.936 µs, 598.847 µs] 126.092 µs (28.5%)
tracing 535.747 µs [504.988 µs, 566.507 µs] 93.447 µs (21.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 461.944 µs [432.901 µs, 490.987 µs] -
iast 575.463 µs [544.366 µs, 606.561 µs] 113.519 µs (24.6%)
iast_FULL 665.692 µs [633.941 µs, 697.442 µs] 203.748 µs (44.1%)
iast_INACTIVE 541.295 µs [510.535 µs, 572.055 µs] 79.351 µs (17.2%)
iast_TELEMETRY_OFF 577.223 µs [545.797 µs, 608.65 µs] 115.279 µs (25.0%)
tracing 532.415 µs [502.953 µs, 561.877 µs] 70.471 µs (15.3%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.26.0-SNAPSHOT~725e1b31bd, baseline=1.26.0-SNAPSHOT~e26f1a4abf
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.714 ms) : 1688, 1739
.   : milestone, 1714,
appsec (2.22 ms) : 2187, 2253
.   : milestone, 2220,
iast (1.915 ms) : 1881, 1948
.   : milestone, 1915,
profiling (1.907 ms) : 1873, 1942
.   : milestone, 1907,
tracing (1.912 ms) : 1879, 1945
.   : milestone, 1912,
section candidate
no_agent (1.73 ms) : 1705, 1755
.   : milestone, 1730,
appsec (2.205 ms) : 2172, 2238
.   : milestone, 2205,
iast (1.925 ms) : 1892, 1957
.   : milestone, 1925,
profiling (1.924 ms) : 1889, 1958
.   : milestone, 1924,
tracing (1.904 ms) : 1871, 1938
.   : milestone, 1904,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.714 ms [1.688 ms, 1.739 ms] -
appsec 2.22 ms [2.187 ms, 2.253 ms] 506.136 µs (29.5%)
iast 1.915 ms [1.881 ms, 1.948 ms] 200.76 µs (11.7%)
profiling 1.907 ms [1.873 ms, 1.942 ms] 193.219 µs (11.3%)
tracing 1.912 ms [1.879 ms, 1.945 ms] 198.359 µs (11.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.73 ms [1.705 ms, 1.755 ms] -
appsec 2.205 ms [2.172 ms, 2.238 ms] 474.782 µs (27.4%)
iast 1.925 ms [1.892 ms, 1.957 ms] 194.197 µs (11.2%)
profiling 1.924 ms [1.889 ms, 1.958 ms] 193.137 µs (11.2%)
tracing 1.904 ms [1.871 ms, 1.938 ms] 174.075 µs (10.1%)

@jbachorik
jbachorik marked this pull request as ready for review December 12, 2023 12:03
@jbachorik
jbachorik requested a review from a team as a code owner December 12, 2023 12:03
@jbachorik
jbachorik merged commit e200ff1 into master Dec 13, 2023
@jbachorik
jbachorik deleted the jb/native_image_jfr_setting branch December 13, 2023 11:41
@github-actions github-actions Bot added this to the 1.26.0 milestone Dec 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@richardstartin richardstartin richardstartin approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

comp: native-image GraalVM native-image comp: profiling Profiling

Projects

None yet

Milestone

1.26.0

Development

Successfully merging this pull request may close these issues.

3 participants

@jbachorik @richardstartin @nikita-tkachenko-datadog