Skip to content

Fix instrumentation when bytecode generation fails#5767

Merged
jpbempel merged 2 commits into
masterfrom
jpbempel/invalid-instrumentation-null
Aug 28, 2023
Merged

Fix instrumentation when bytecode generation fails#5767
jpbempel merged 2 commits into
masterfrom
jpbempel/invalid-instrumentation-null

Conversation

@jpbempel

@jpbempel jpbempel commented Aug 23, 2023

Copy link
Copy Markdown
Member

If bytecode generation fails for any reason we should returns null for cancelling the current class transformation

What Does This Do

Motivation

Additional Notes

@jpbempel
Fix instrumentation when bytecode generation fails
cd05558 
If bytecode generation fails for any reason we should returns null for
cancelling the current class transformation
@jpbempel
jpbempel requested a review from a team as a code owner August 23, 2023 13:46
@jpbempel
jpbempel requested review from ojung and removed request for a team August 23, 2023 13:46
@jpbempel jpbempel added the comp: debugger Dynamic Instrumentation label Aug 23, 2023
@pr-commenter

pr-commenter Bot commented Aug 23, 2023 •
edited
Loading

Copy link
Copy Markdown

Benchmarks

Startup

Parameters

Baseline Candidate
commit 1.20.0-SNAPSHOT~65849d8869 1.20.0-SNAPSHOT~29e598b1e7
config baseline candidate
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
module Agent Agent
parent None None
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 62 cases.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.20.0-SNAPSHOT~29e598b1e7, baseline=1.20.0-SNAPSHOT~65849d8869

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (986.42 ms) : 0, 986420
Total [baseline] (8.666 s) : 0, 8665969
Agent [candidate] (988.224 ms) : 0, 988224
Total [candidate] (8.64 s) : 0, 8640148
section appsec
Agent [baseline] (1.071 s) : 0, 1070643
Total [baseline] (8.751 s) : 0, 8751082
Agent [candidate] (1.074 s) : 0, 1074360
Total [candidate] (8.731 s) : 0, 8731160
section iast
Agent [baseline] (1.09 s) : 0, 1090180
Total [baseline] (9.171 s) : 0, 9171392
Agent [candidate] (1.098 s) : 0, 1097598
Total [candidate] (9.184 s) : 0, 9183788
section profiling
Agent [baseline] (1.168 s) : 0, 1168108
Total [baseline] (8.937 s) : 0, 8936689
Agent [candidate] (1.162 s) : 0, 1161607
Total [candidate] (8.879 s) : 0, 8878700
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 986.42 ms -
Agent appsec 1.071 s 84.223 ms (8.5%)
Agent iast 1.09 s 103.76 ms (10.5%)
Agent profiling 1.168 s 181.688 ms (18.4%)
Total tracing 8.666 s -
Total appsec 8.751 s 85.113 ms (1.0%)
Total iast 9.171 s 505.422 ms (5.8%)
Total profiling 8.937 s 270.72 ms (3.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 988.224 ms -
Agent appsec 1.074 s 86.136 ms (8.7%)
Agent iast 1.098 s 109.374 ms (11.1%)
Agent profiling 1.162 s 173.383 ms (17.5%)
Total tracing 8.64 s -
Total appsec 8.731 s 91.011 ms (1.1%)
Total iast 9.184 s 543.64 ms (6.3%)
Total profiling 8.879 s 238.551 ms (2.8%) 
gantt
    title insecure-bank - break down per module: candidate=1.20.0-SNAPSHOT~29e598b1e7, baseline=1.20.0-SNAPSHOT~65849d8869

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (612.174 ms) : 0, 612174
BytebuddyAgent [candidate] (612.888 ms) : 0, 612888
GlobalTracer [baseline] (283.282 ms) : 0, 283282
GlobalTracer [candidate] (284.625 ms) : 0, 284625
AppSec [baseline] (50.033 ms) : 0, 50033
AppSec [candidate] (49.739 ms) : 0, 49739
Remote Config [baseline] (664.203 µs) : 0, 664
Remote Config [candidate] (665.873 µs) : 0, 666
Telemetry [baseline] (5.992 ms) : 0, 5992
Telemetry [candidate] (5.971 ms) : 0, 5971
section appsec
BytebuddyAgent [baseline] (611.94 ms) : 0, 611940
BytebuddyAgent [candidate] (613.59 ms) : 0, 613590
GlobalTracer [baseline] (283.456 ms) : 0, 283456
GlobalTracer [candidate] (285.418 ms) : 0, 285418
AppSec [baseline] (134.567 ms) : 0, 134567
AppSec [candidate] (134.588 ms) : 0, 134588
Remote Config [baseline] (719.151 µs) : 0, 719
Remote Config [candidate] (721.776 µs) : 0, 722
Telemetry [baseline] (5.743 ms) : 0, 5743
Telemetry [candidate] (5.749 ms) : 0, 5749
section iast
BytebuddyAgent [baseline] (718.219 ms) : 0, 718219
BytebuddyAgent [candidate] (723.057 ms) : 0, 723057
GlobalTracer [baseline] (270.314 ms) : 0, 270314
GlobalTracer [candidate] (272.526 ms) : 0, 272526
AppSec [baseline] (47.171 ms) : 0, 47171
AppSec [candidate] (47.407 ms) : 0, 47407
Remote Config [baseline] (557.853 µs) : 0, 558
Remote Config [candidate] (548.721 µs) : 0, 549
Telemetry [baseline] (5.786 ms) : 0, 5786
Telemetry [candidate] (5.716 ms) : 0, 5716
IAST [baseline] (13.856 ms) : 0, 13856
IAST [candidate] (13.916 ms) : 0, 13916
section profiling
BytebuddyAgent [baseline] (630.619 ms) : 0, 630619
BytebuddyAgent [candidate] (627.223 ms) : 0, 627223
GlobalTracer [baseline] (350.207 ms) : 0, 350207
GlobalTracer [candidate] (347.422 ms) : 0, 347422
AppSec [baseline] (49.808 ms) : 0, 49808
AppSec [candidate] (49.206 ms) : 0, 49206
Remote Config [baseline] (658.911 µs) : 0, 659
Remote Config [candidate] (645.934 µs) : 0, 646
Telemetry [baseline] (6.092 ms) : 0, 6092
Telemetry [candidate] (5.956 ms) : 0, 5956
ProfilingAgent [baseline] (77.447 ms) : 0, 77447
ProfilingAgent [candidate] (78.267 ms) : 0, 78267
Profiling [baseline] (77.471 ms) : 0, 77471
Profiling [candidate] (78.291 ms) : 0, 78291
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.20.0-SNAPSHOT~29e598b1e7, baseline=1.20.0-SNAPSHOT~65849d8869

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (986.903 ms) : 0, 986903
Total [baseline] (9.189 s) : 0, 9189220
Agent [candidate] (987.124 ms) : 0, 987124
Total [candidate] (9.214 s) : 0, 9213776
section appsec
Agent [baseline] (1.07 s) : 0, 1070110
Total [baseline] (9.292 s) : 0, 9291995
Agent [candidate] (1.076 s) : 0, 1076280
Total [candidate] (9.33 s) : 0, 9330150
section iast
Agent [baseline] (1.092 s) : 0, 1091560
Total [baseline] (9.433 s) : 0, 9433176
Agent [candidate] (1.09 s) : 0, 1090244
Total [candidate] (9.433 s) : 0, 9432922
section profiling
Agent [baseline] (1.166 s) : 0, 1165787
Total [baseline] (9.523 s) : 0, 9522808
Agent [candidate] (1.16 s) : 0, 1159649
Total [candidate] (9.48 s) : 0, 9480128
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 986.903 ms -
Agent appsec 1.07 s 83.207 ms (8.4%)
Agent iast 1.092 s 104.657 ms (10.6%)
Agent profiling 1.166 s 178.884 ms (18.1%)
Total tracing 9.189 s -
Total appsec 9.292 s 102.775 ms (1.1%)
Total iast 9.433 s 243.956 ms (2.7%)
Total profiling 9.523 s 333.588 ms (3.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 987.124 ms -
Agent appsec 1.076 s 89.156 ms (9.0%)
Agent iast 1.09 s 103.12 ms (10.4%)
Agent profiling 1.16 s 172.525 ms (17.5%)
Total tracing 9.214 s -
Total appsec 9.33 s 116.375 ms (1.3%)
Total iast 9.433 s 219.146 ms (2.4%)
Total profiling 9.48 s 266.352 ms (2.9%) 
gantt
    title petclinic - break down per module: candidate=1.20.0-SNAPSHOT~29e598b1e7, baseline=1.20.0-SNAPSHOT~65849d8869

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (612.699 ms) : 0, 612699
BytebuddyAgent [candidate] (611.973 ms) : 0, 611973
GlobalTracer [baseline] (283.412 ms) : 0, 283412
GlobalTracer [candidate] (284.357 ms) : 0, 284357
AppSec [baseline] (49.986 ms) : 0, 49986
AppSec [candidate] (49.897 ms) : 0, 49897
Remote Config [baseline] (656.073 µs) : 0, 656
Remote Config [candidate] (675.393 µs) : 0, 675
Telemetry [baseline] (5.951 ms) : 0, 5951
Telemetry [candidate] (6.024 ms) : 0, 6024
section appsec
BytebuddyAgent [baseline] (610.989 ms) : 0, 610989
BytebuddyAgent [candidate] (613.647 ms) : 0, 613647
GlobalTracer [baseline] (283.792 ms) : 0, 283792
GlobalTracer [candidate] (286.458 ms) : 0, 286458
AppSec [baseline] (134.681 ms) : 0, 134681
AppSec [candidate] (135.487 ms) : 0, 135487
Remote Config [baseline] (721.502 µs) : 0, 722
Remote Config [candidate] (726.522 µs) : 0, 727
Telemetry [baseline] (5.71 ms) : 0, 5710
Telemetry [candidate] (5.76 ms) : 0, 5760
section iast
BytebuddyAgent [baseline] (719.083 ms) : 0, 719083
BytebuddyAgent [candidate] (717.905 ms) : 0, 717905
GlobalTracer [baseline] (270.748 ms) : 0, 270748
GlobalTracer [candidate] (271.089 ms) : 0, 271089
AppSec [baseline] (47.242 ms) : 0, 47242
AppSec [candidate] (47.046 ms) : 0, 47046
Remote Config [baseline] (560.273 µs) : 0, 560
Remote Config [candidate] (533.866 µs) : 0, 534
Telemetry [baseline] (5.762 ms) : 0, 5762
Telemetry [candidate] (5.623 ms) : 0, 5623
IAST [baseline] (13.861 ms) : 0, 13861
IAST [candidate] (13.748 ms) : 0, 13748
section profiling
BytebuddyAgent [baseline] (630.035 ms) : 0, 630035
BytebuddyAgent [candidate] (625.272 ms) : 0, 625272
GlobalTracer [baseline] (347.981 ms) : 0, 347981
GlobalTracer [candidate] (346.856 ms) : 0, 346856
AppSec [baseline] (49.462 ms) : 0, 49462
AppSec [candidate] (49.23 ms) : 0, 49230
Remote Config [baseline] (657.306 µs) : 0, 657
Remote Config [candidate] (643.768 µs) : 0, 644
Telemetry [baseline] (6.097 ms) : 0, 6097
Telemetry [candidate] (5.949 ms) : 0, 5949
ProfilingAgent [baseline] (78.151 ms) : 0, 78151
ProfilingAgent [candidate] (78.777 ms) : 0, 78777
Profiling [baseline] (78.176 ms) : 0, 78176
Profiling [candidate] (78.801 ms) : 0, 78801
Loading

Load

Parameters

Baseline Candidate
commit 1.20.0-SNAPSHOT~65849d8869 1.20.0-SNAPSHOT~29e598b1e7
config baseline candidate
end_time 2023-08-24T10:29:45 2023-08-24T10:47:52
start_time 2023-08-24T10:29:28 2023-08-24T10:47:35
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 24 cases.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.20.0-SNAPSHOT~29e598b1e7, baseline=1.20.0-SNAPSHOT~65849d8869
    dateFormat X
    axisFormat %s
section baseline
no_agent (365.356 µs) : 346, 385
.   : milestone, 365,
appsec (667.193 µs) : 647, 688
.   : milestone, 667,
iast (460.024 µs) : 439, 481
.   : milestone, 460,
iast_FULL (522.714 µs) : 502, 543
.   : milestone, 523,
iast_INACTIVE (435.212 µs) : 414, 456
.   : milestone, 435,
profiling (437.234 µs) : 416, 458
.   : milestone, 437,
tracing (432.209 µs) : 412, 453
.   : milestone, 432,
section candidate
no_agent (365.5 µs) : 345, 386
.   : milestone, 365,
appsec (666.737 µs) : 646, 688
.   : milestone, 667,
iast (465.081 µs) : 444, 486
.   : milestone, 465,
iast_FULL (528.812 µs) : 508, 550
.   : milestone, 529,
iast_INACTIVE (441.193 µs) : 421, 462
.   : milestone, 441,
profiling (445.723 µs) : 424, 467
.   : milestone, 446,
tracing (435.921 µs) : 414, 457
.   : milestone, 436,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 365.356 µs [345.643 µs, 385.068 µs] -
appsec 667.193 µs [646.58 µs, 687.805 µs] 301.837 µs (82.6%)
iast 460.024 µs [439.488 µs, 480.56 µs] 94.669 µs (25.9%)
iast_FULL 522.714 µs [502.125 µs, 543.304 µs] 157.359 µs (43.1%)
iast_INACTIVE 435.212 µs [414.152 µs, 456.273 µs] 69.857 µs (19.1%)
profiling 437.234 µs [416.221 µs, 458.248 µs] 71.879 µs (19.7%)
tracing 432.209 µs [411.672 µs, 452.747 µs] 66.854 µs (18.3%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 365.5 µs [345.126 µs, 385.874 µs] -
appsec 666.737 µs [645.581 µs, 687.894 µs] 301.238 µs (82.4%)
iast 465.081 µs [443.824 µs, 486.337 µs] 99.581 µs (27.2%)
iast_FULL 528.812 µs [508.008 µs, 549.617 µs] 163.312 µs (44.7%)
iast_INACTIVE 441.193 µs [420.649 µs, 461.737 µs] 75.693 µs (20.7%)
profiling 445.723 µs [424.451 µs, 466.995 µs] 80.223 µs (21.9%)
tracing 435.921 µs [414.471 µs, 457.372 µs] 70.421 µs (19.3%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.20.0-SNAPSHOT~29e598b1e7, baseline=1.20.0-SNAPSHOT~65849d8869
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.333 ms) : 1315, 1352
.   : milestone, 1333,
appsec (1.697 ms) : 1673, 1722
.   : milestone, 1697,
iast (1.473 ms) : 1449, 1497
.   : milestone, 1473,
profiling (1.47 ms) : 1445, 1495
.   : milestone, 1470,
tracing (1.464 ms) : 1440, 1488
.   : milestone, 1464,
section candidate
no_agent (1.342 ms) : 1323, 1362
.   : milestone, 1342,
appsec (1.663 ms) : 1639, 1687
.   : milestone, 1663,
iast (1.457 ms) : 1432, 1482
.   : milestone, 1457,
profiling (1.467 ms) : 1439, 1494
.   : milestone, 1467,
tracing (1.443 ms) : 1419, 1467
.   : milestone, 1443,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.333 ms [1.315 ms, 1.352 ms] -
appsec 1.697 ms [1.673 ms, 1.722 ms] 364.103 µs (27.3%)
iast 1.473 ms [1.449 ms, 1.497 ms] 139.738 µs (10.5%)
profiling 1.47 ms [1.445 ms, 1.495 ms] 136.756 µs (10.3%)
tracing 1.464 ms [1.44 ms, 1.488 ms] 130.506 µs (9.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.342 ms [1.323 ms, 1.362 ms] -
appsec 1.663 ms [1.639 ms, 1.687 ms] 320.629 µs (23.9%)
iast 1.457 ms [1.432 ms, 1.482 ms] 114.845 µs (8.6%)
profiling 1.467 ms [1.439 ms, 1.494 ms] 124.303 µs (9.3%)
tracing 1.443 ms [1.419 ms, 1.467 ms] 100.491 µs (7.5%)

shatzi
shatzi approved these changes Aug 27, 2023
View reviewed changes

@shatzi shatzi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome fix.

@jpbempel
jpbempel merged commit b9eecfb into master Aug 28, 2023
@jpbempel
jpbempel deleted the jpbempel/invalid-instrumentation-null branch August 28, 2023 06:28
@github-actions github-actions Bot added this to the 1.20.0 milestone Aug 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ojung ojung Awaiting requested review from ojung ojung is a code owner automatically assigned from DataDog/debugger-java
1 more reviewer
@shatzi shatzi shatzi approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

comp: debugger Dynamic Instrumentation type: bug fix Bug fix

Projects

None yet

Milestone

1.20.0

Development

Successfully merging this pull request may close these issues.

3 participants

@jpbempel @shatzi @ValentinZakharov