Skip to content

Fail smoke tests if received traces cannot be decoded#7520

Merged
smola merged 1 commit intomasterfrom
smola/smoke-fail-on-decoding-exception
Aug 28, 2024
Merged

Fail smoke tests if received traces cannot be decoded#7520
smola merged 1 commit intomasterfrom
smola/smoke-fail-on-decoding-exception

Conversation

@smola
Copy link
Copy Markdown
Member

@smola smola commented Aug 27, 2024 •
edited
Loading

What Does This Do

If smoke tests "mock agent" fails to decode a trace, bubble the exception up to waitForTraces (or eventually to cleanup). This way we get a clearer indication of the issue, rather than just a timeout on receiving the traces.

Motivation

Faster troubleshooting.

Additional Notes

Before this change:

Condition not satisfied after 30.00 seconds and 69 attempts
	at spock.util.concurrent.PollingConditions.within(PollingConditions.java:185)
	at spock.util.concurrent.PollingConditions.eventually(PollingConditions.java:140)
	at datadog.smoketest.AbstractSmokeTest.waitForTraceCount(AbstractSmokeTest.groovy:231)
	at datadog.smoketest.AbstractSmokeTest.waitForTraceCount(AbstractSmokeTest.groovy:227)
	at datadog.smoketest.appsec.SpringBootSmokeTest.rasp reports stacktrace on sql injection(SpringBootSmokeTest.groovy:282)
Caused by: Condition not satisfied:

traceCount.get() >= count
|          |     |  |
0          0     |  1
                 false

	at datadog.smoketest.AbstractSmokeTest.waitForTraceCount_closure1(AbstractSmokeTest.groovy:232)

After this change:

Condition not satisfied after 30.00 seconds and 80 attempts
	at spock.util.concurrent.PollingConditions.within(PollingConditions.java:185)
	at spock.util.concurrent.PollingConditions.eventually(PollingConditions.java:140)
	at datadog.smoketest.AbstractSmokeTest.waitForTraceCount(AbstractSmokeTest.groovy:240)
	at datadog.smoketest.AbstractSmokeTest.waitForTraceCount(AbstractSmokeTest.groovy:236)
	at datadog.smoketest.appsec.SpringBootSmokeTest.rasp reports stacktrace on sql injection(SpringBootSmokeTest.groovy:282)
	Suppressed: java.lang.IllegalArgumentException: Wrong span element map size 13. Expected 12.
		at datadog.trace.test.agent.decoder.v04.raw.SpanV04.unpack(SpanV04.java:37)
		at datadog.trace.test.agent.decoder.v04.raw.SpanV04.unpackSpans(SpanV04.java:21)
[...]

Contributor Checklist

@smola smola added comp: testing Testing tag: no release notes Changes to exclude from release notes labels Aug 27, 2024
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Aug 27, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/smoke-fail-on-decoding-exception
git_commit_date 1724769397 1724777604
git_commit_sha 115998f b26e812
release_version 1.39.0-SNAPSHOT~115998f9f0 1.39.0-SNAPSHOT~b26e81274c
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1724780068 1724780068
ci_job_id 618856185 618856185
ci_pipeline_id 42954457 42954457
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 15 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.39.0-SNAPSHOT~b26e81274c, baseline=1.39.0-SNAPSHOT~115998f9f0

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.047 s) : 0, 1047450
Total [baseline] (10.381 s) : 0, 10380624
Agent [candidate] (1.046 s) : 0, 1046227
Total [candidate] (10.427 s) : 0, 10426984
section appsec
Agent [baseline] (1.176 s) : 0, 1175857
Total [baseline] (10.513 s) : 0, 10513145
Agent [candidate] (1.178 s) : 0, 1177530
Total [candidate] (10.444 s) : 0, 10444399
section iast
Agent [baseline] (1.174 s) : 0, 1174294
Total [baseline] (10.89 s) : 0, 10890302
Agent [candidate] (1.172 s) : 0, 1172313
Total [candidate] (10.808 s) : 0, 10807574
section profiling
Agent [baseline] (1.255 s) : 0, 1255172
Total [baseline] (10.584 s) : 0, 10584093
Agent [candidate] (1.263 s) : 0, 1263437
Total [candidate] (10.637 s) : 0, 10637209
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.047 s -
Agent appsec 1.176 s 128.406 ms (12.3%)
Agent iast 1.174 s 126.844 ms (12.1%)
Agent profiling 1.255 s 207.721 ms (19.8%)
Total tracing 10.381 s -
Total appsec 10.513 s 132.521 ms (1.3%)
Total iast 10.89 s 509.678 ms (4.9%)
Total profiling 10.584 s 203.469 ms (2.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.046 s -
Agent appsec 1.178 s 131.303 ms (12.6%)
Agent iast 1.172 s 126.086 ms (12.1%)
Agent profiling 1.263 s 217.21 ms (20.8%)
Total tracing 10.427 s -
Total appsec 10.444 s 17.415 ms (0.2%)
Total iast 10.808 s 380.59 ms (3.7%)
Total profiling 10.637 s 210.225 ms (2.0%) 
gantt
    title petclinic - break down per module: candidate=1.39.0-SNAPSHOT~b26e81274c, baseline=1.39.0-SNAPSHOT~115998f9f0

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (668.603 ms) : 0, 668603
BytebuddyAgent [candidate] (667.514 ms) : 0, 667514
GlobalTracer [baseline] (305.848 ms) : 0, 305848
GlobalTracer [candidate] (305.836 ms) : 0, 305836
AppSec [baseline] (51.298 ms) : 0, 51298
AppSec [candidate] (51.206 ms) : 0, 51206
Remote Config [baseline] (668.13 µs) : 0, 668
Remote Config [candidate] (662.13 µs) : 0, 662
Telemetry [baseline] (7.48 ms) : 0, 7480
Telemetry [candidate] (7.449 ms) : 0, 7449
section appsec
BytebuddyAgent [baseline] (681.982 ms) : 0, 681982
BytebuddyAgent [candidate] (682.993 ms) : 0, 682993
GlobalTracer [baseline] (302.624 ms) : 0, 302624
GlobalTracer [candidate] (301.964 ms) : 0, 301964
AppSec [baseline] (160.998 ms) : 0, 160998
AppSec [candidate] (161.026 ms) : 0, 161026
IAST [baseline] (18.371 ms) : 0, 18371
IAST [candidate] (18.885 ms) : 0, 18885
Remote Config [baseline] (637.75 µs) : 0, 638
Remote Config [candidate] (641.215 µs) : 0, 641
Telemetry [baseline] (6.834 ms) : 0, 6834
Telemetry [candidate] (7.812 ms) : 0, 7812
section iast
BytebuddyAgent [baseline] (779.409 ms) : 0, 779409
BytebuddyAgent [candidate] (777.807 ms) : 0, 777807
GlobalTracer [baseline] (296.569 ms) : 0, 296569
GlobalTracer [candidate] (296.096 ms) : 0, 296096
AppSec [baseline] (50.449 ms) : 0, 50449
AppSec [candidate] (50.487 ms) : 0, 50487
IAST [baseline] (25.006 ms) : 0, 25006
IAST [candidate] (24.243 ms) : 0, 24243
Remote Config [baseline] (1.281 ms) : 0, 1281
Remote Config [candidate] (597.894 µs) : 0, 598
Telemetry [baseline] (7.977 ms) : 0, 7977
Telemetry [candidate] (9.486 ms) : 0, 9486
section profiling
BytebuddyAgent [baseline] (669.309 ms) : 0, 669309
BytebuddyAgent [candidate] (673.712 ms) : 0, 673712
GlobalTracer [baseline] (391.281 ms) : 0, 391281
GlobalTracer [candidate] (393.433 ms) : 0, 393433
AppSec [baseline] (52.438 ms) : 0, 52438
AppSec [candidate] (53.051 ms) : 0, 53051
Remote Config [baseline] (684.518 µs) : 0, 685
Remote Config [candidate] (709.546 µs) : 0, 710
Telemetry [baseline] (7.396 ms) : 0, 7396
Telemetry [candidate] (7.527 ms) : 0, 7527
ProfilingAgent [baseline] (96.381 ms) : 0, 96381
ProfilingAgent [candidate] (97.16 ms) : 0, 97160
Profiling [baseline] (96.405 ms) : 0, 96405
Profiling [candidate] (97.184 ms) : 0, 97184
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.39.0-SNAPSHOT~b26e81274c, baseline=1.39.0-SNAPSHOT~115998f9f0

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.059 s) : 0, 1059351
Total [baseline] (8.508 s) : 0, 8508094
Agent [candidate] (1.048 s) : 0, 1048212
Total [candidate] (8.467 s) : 0, 8467476
section iast
Agent [baseline] (1.171 s) : 0, 1171027
Total [baseline] (8.954 s) : 0, 8953660
Agent [candidate] (1.173 s) : 0, 1172842
Total [candidate] (8.966 s) : 0, 8965972
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.171 s) : 0, 1170977
Total [baseline] (8.941 s) : 0, 8941458
Agent [candidate] (1.179 s) : 0, 1179035
Total [candidate] (8.927 s) : 0, 8927347
section iast_TELEMETRY_OFF
Agent [baseline] (1.186 s) : 0, 1186320
Total [baseline] (8.998 s) : 0, 8998391
Agent [candidate] (1.168 s) : 0, 1167903
Total [candidate] (8.947 s) : 0, 8946595
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.059 s -
Agent iast 1.171 s 111.676 ms (10.5%)
Agent iast_HARDCODED_SECRET_DISABLED 1.171 s 111.626 ms (10.5%)
Agent iast_TELEMETRY_OFF 1.186 s 126.969 ms (12.0%)
Total tracing 8.508 s -
Total iast 8.954 s 445.566 ms (5.2%)
Total iast_HARDCODED_SECRET_DISABLED 8.941 s 433.364 ms (5.1%)
Total iast_TELEMETRY_OFF 8.998 s 490.297 ms (5.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.048 s -
Agent iast 1.173 s 124.63 ms (11.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.179 s 130.823 ms (12.5%)
Agent iast_TELEMETRY_OFF 1.168 s 119.69 ms (11.4%)
Total tracing 8.467 s -
Total iast 8.966 s 498.495 ms (5.9%)
Total iast_HARDCODED_SECRET_DISABLED 8.927 s 459.871 ms (5.4%)
Total iast_TELEMETRY_OFF 8.947 s 479.118 ms (5.7%) 
gantt
    title insecure-bank - break down per module: candidate=1.39.0-SNAPSHOT~b26e81274c, baseline=1.39.0-SNAPSHOT~115998f9f0

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (677.079 ms) : 0, 677079
BytebuddyAgent [candidate] (668.855 ms) : 0, 668855
GlobalTracer [baseline] (308.582 ms) : 0, 308582
GlobalTracer [candidate] (306.185 ms) : 0, 306185
AppSec [baseline] (51.701 ms) : 0, 51701
AppSec [candidate] (51.456 ms) : 0, 51456
Remote Config [baseline] (677.48 µs) : 0, 677
Remote Config [candidate] (672.327 µs) : 0, 672
Telemetry [baseline] (7.559 ms) : 0, 7559
Telemetry [candidate] (7.482 ms) : 0, 7482
section iast
BytebuddyAgent [baseline] (777.054 ms) : 0, 777054
BytebuddyAgent [candidate] (779.149 ms) : 0, 779149
GlobalTracer [baseline] (295.737 ms) : 0, 295737
GlobalTracer [candidate] (296.368 ms) : 0, 296368
AppSec [baseline] (48.349 ms) : 0, 48349
AppSec [candidate] (51.914 ms) : 0, 51914
IAST [baseline] (27.792 ms) : 0, 27792
IAST [candidate] (24.05 ms) : 0, 24050
Remote Config [baseline] (586.388 µs) : 0, 586
Remote Config [candidate] (580.982 µs) : 0, 581
Telemetry [baseline] (7.96 ms) : 0, 7960
Telemetry [candidate] (7.18 ms) : 0, 7180
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (777.765 ms) : 0, 777765
BytebuddyAgent [candidate] (782.949 ms) : 0, 782949
GlobalTracer [baseline] (296.325 ms) : 0, 296325
GlobalTracer [candidate] (298.379 ms) : 0, 298379
AppSec [baseline] (49.03 ms) : 0, 49030
AppSec [candidate] (50.761 ms) : 0, 50761
IAST [baseline] (24.97 ms) : 0, 24970
IAST [candidate] (23.965 ms) : 0, 23965
Remote Config [baseline] (590.857 µs) : 0, 591
Remote Config [candidate] (577.316 µs) : 0, 577
Telemetry [baseline] (8.681 ms) : 0, 8681
Telemetry [candidate] (8.728 ms) : 0, 8728
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (786.463 ms) : 0, 786463
BytebuddyAgent [candidate] (774.618 ms) : 0, 774618
GlobalTracer [baseline] (300.096 ms) : 0, 300096
GlobalTracer [candidate] (296.149 ms) : 0, 296149
AppSec [baseline] (52.018 ms) : 0, 52018
AppSec [candidate] (51.188 ms) : 0, 51188
IAST [baseline] (24.617 ms) : 0, 24617
IAST [candidate] (23.208 ms) : 0, 23208
Remote Config [baseline] (593.78 µs) : 0, 594
Remote Config [candidate] (582.275 µs) : 0, 582
Telemetry [baseline] (8.731 ms) : 0, 8731
Telemetry [candidate] (8.54 ms) : 0, 8540
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-08-27T17:05:10 2024-08-27T17:11:57
git_branch master smola/smoke-fail-on-decoding-exception
git_commit_date 1724769397 1724777604
git_commit_sha 115998f b26e812
release_version 1.39.0-SNAPSHOT~115998f9f0 1.39.0-SNAPSHOT~b26e81274c
start_time 2024-08-27T17:04:57 2024-08-27T17:11:44
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1724779060 1724779060
ci_job_id 618856186 618856186
ci_pipeline_id 42954457 42954457
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 18 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.39.0-SNAPSHOT~b26e81274c, baseline=1.39.0-SNAPSHOT~115998f9f0
    dateFormat X
    axisFormat %s
section baseline
no_agent (364.344 µs) : 345, 384
.   : milestone, 364,
iast (482.041 µs) : 459, 505
.   : milestone, 482,
iast_FULL (544.634 µs) : 523, 566
.   : milestone, 545,
iast_GLOBAL (503.817 µs) : 482, 525
.   : milestone, 504,
iast_HARDCODED_SECRET_DISABLED (481.679 µs) : 460, 504
.   : milestone, 482,
iast_INACTIVE (437.761 µs) : 417, 458
.   : milestone, 438,
iast_TELEMETRY_OFF (470.082 µs) : 448, 492
.   : milestone, 470,
tracing (433.17 µs) : 413, 453
.   : milestone, 433,
section candidate
no_agent (365.43 µs) : 346, 385
.   : milestone, 365,
iast (485.161 µs) : 462, 508
.   : milestone, 485,
iast_FULL (549.019 µs) : 528, 570
.   : milestone, 549,
iast_GLOBAL (501.879 µs) : 480, 524
.   : milestone, 502,
iast_HARDCODED_SECRET_DISABLED (481.12 µs) : 459, 504
.   : milestone, 481,
iast_INACTIVE (446.37 µs) : 425, 467
.   : milestone, 446,
iast_TELEMETRY_OFF (472.289 µs) : 450, 495
.   : milestone, 472,
tracing (438.142 µs) : 418, 458
.   : milestone, 438,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 364.344 µs [344.704 µs, 383.984 µs] -
iast 482.041 µs [459.375 µs, 504.706 µs] 117.696 µs (32.3%)
iast_FULL 544.634 µs [523.455 µs, 565.812 µs] 180.289 µs (49.5%)
iast_GLOBAL 503.817 µs [482.489 µs, 525.145 µs] 139.473 µs (38.3%)
iast_HARDCODED_SECRET_DISABLED 481.679 µs [459.593 µs, 503.765 µs] 117.335 µs (32.2%)
iast_INACTIVE 437.761 µs [417.43 µs, 458.092 µs] 73.416 µs (20.2%)
iast_TELEMETRY_OFF 470.082 µs [447.713 µs, 492.452 µs] 105.738 µs (29.0%)
tracing 433.17 µs [413.174 µs, 453.165 µs] 68.825 µs (18.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 365.43 µs [346.143 µs, 384.717 µs] -
iast 485.161 µs [462.467 µs, 507.854 µs] 119.731 µs (32.8%)
iast_FULL 549.019 µs [527.691 µs, 570.348 µs] 183.589 µs (50.2%)
iast_GLOBAL 501.879 µs [480.062 µs, 523.697 µs] 136.449 µs (37.3%)
iast_HARDCODED_SECRET_DISABLED 481.12 µs [458.506 µs, 503.735 µs] 115.69 µs (31.7%)
iast_INACTIVE 446.37 µs [425.421 µs, 467.32 µs] 80.94 µs (22.1%)
iast_TELEMETRY_OFF 472.289 µs [449.766 µs, 494.812 µs] 106.859 µs (29.2%)
tracing 438.142 µs [418.1 µs, 458.183 µs] 72.712 µs (19.9%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.39.0-SNAPSHOT~b26e81274c, baseline=1.39.0-SNAPSHOT~115998f9f0
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.348 ms) : 1328, 1367
.   : milestone, 1348,
appsec (1.744 ms) : 1721, 1767
.   : milestone, 1744,
appsec_no_iast (1.718 ms) : 1694, 1742
.   : milestone, 1718,
iast (1.464 ms) : 1441, 1487
.   : milestone, 1464,
profiling (1.477 ms) : 1454, 1499
.   : milestone, 1477,
tracing (1.464 ms) : 1439, 1489
.   : milestone, 1464,
section candidate
no_agent (1.363 ms) : 1343, 1382
.   : milestone, 1363,
appsec (1.696 ms) : 1674, 1718
.   : milestone, 1696,
appsec_no_iast (1.7 ms) : 1676, 1724
.   : milestone, 1700,
iast (1.468 ms) : 1446, 1491
.   : milestone, 1468,
profiling (1.478 ms) : 1455, 1502
.   : milestone, 1478,
tracing (1.464 ms) : 1440, 1489
.   : milestone, 1464,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.348 ms [1.328 ms, 1.367 ms] -
appsec 1.744 ms [1.721 ms, 1.767 ms] 396.282 µs (29.4%)
appsec_no_iast 1.718 ms [1.694 ms, 1.742 ms] 369.818 µs (27.4%)
iast 1.464 ms [1.441 ms, 1.487 ms] 116.348 µs (8.6%)
profiling 1.477 ms [1.454 ms, 1.499 ms] 128.932 µs (9.6%)
tracing 1.464 ms [1.439 ms, 1.489 ms] 115.919 µs (8.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.363 ms [1.343 ms, 1.382 ms] -
appsec 1.696 ms [1.674 ms, 1.718 ms] 333.307 µs (24.5%)
appsec_no_iast 1.7 ms [1.676 ms, 1.724 ms] 337.281 µs (24.8%)
iast 1.468 ms [1.446 ms, 1.491 ms] 105.609 µs (7.8%)
profiling 1.478 ms [1.455 ms, 1.502 ms] 115.94 µs (8.5%)
tracing 1.464 ms [1.44 ms, 1.489 ms] 101.802 µs (7.5%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/smoke-fail-on-decoding-exception
git_commit_date 1724769397 1724777604
git_commit_sha 115998f b26e812
release_version 1.39.0-SNAPSHOT~115998f9f0 1.39.0-SNAPSHOT~b26e81274c
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1724779584 1724779584
ci_job_id 618856187 618856187
ci_pipeline_id 42954457 42954457
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.39.0-SNAPSHOT~b26e81274c, baseline=1.39.0-SNAPSHOT~115998f9f0
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.397 s) : 15397000, 15397000
.   : milestone, 15397000,
appsec (15.095 s) : 15095000, 15095000
.   : milestone, 15095000,
iast (18.818 s) : 18818000, 18818000
.   : milestone, 18818000,
iast_GLOBAL (17.99 s) : 17990000, 17990000
.   : milestone, 17990000,
profiling (15.046 s) : 15046000, 15046000
.   : milestone, 15046000,
tracing (15.156 s) : 15156000, 15156000
.   : milestone, 15156000,
section candidate
no_agent (14.973 s) : 14973000, 14973000
.   : milestone, 14973000,
appsec (15.315 s) : 15315000, 15315000
.   : milestone, 15315000,
iast (18.876 s) : 18876000, 18876000
.   : milestone, 18876000,
iast_GLOBAL (17.734 s) : 17734000, 17734000
.   : milestone, 17734000,
profiling (14.744 s) : 14744000, 14744000
.   : milestone, 14744000,
tracing (15.156 s) : 15156000, 15156000
.   : milestone, 15156000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.397 s [15.397 s, 15.397 s] -
appsec 15.095 s [15.095 s, 15.095 s] -302.0 ms (-2.0%)
iast 18.818 s [18.818 s, 18.818 s] 3.421 s (22.2%)
iast_GLOBAL 17.99 s [17.99 s, 17.99 s] 2.593 s (16.8%)
profiling 15.046 s [15.046 s, 15.046 s] -351.0 ms (-2.3%)
tracing 15.156 s [15.156 s, 15.156 s] -241.0 ms (-1.6%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.973 s [14.973 s, 14.973 s] -
appsec 15.315 s [15.315 s, 15.315 s] 342.0 ms (2.3%)
iast 18.876 s [18.876 s, 18.876 s] 3.903 s (26.1%)
iast_GLOBAL 17.734 s [17.734 s, 17.734 s] 2.761 s (18.4%)
profiling 14.744 s [14.744 s, 14.744 s] -229.0 ms (-1.5%)
tracing 15.156 s [15.156 s, 15.156 s] 183.0 ms (1.2%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.39.0-SNAPSHOT~b26e81274c, baseline=1.39.0-SNAPSHOT~115998f9f0
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.462 ms) : 1451, 1473
.   : milestone, 1462,
appsec (2.224 ms) : 2189, 2258
.   : milestone, 2224,
iast (1.967 ms) : 1925, 2008
.   : milestone, 1967,
iast_GLOBAL (2.024 ms) : 1981, 2067
.   : milestone, 2024,
profiling (1.871 ms) : 1836, 1905
.   : milestone, 1871,
tracing (1.85 ms) : 1817, 1884
.   : milestone, 1850,
section candidate
no_agent (1.467 ms) : 1455, 1478
.   : milestone, 1467,
appsec (2.233 ms) : 2198, 2268
.   : milestone, 2233,
iast (1.986 ms) : 1943, 2029
.   : milestone, 1986,
iast_GLOBAL (2.026 ms) : 1982, 2070
.   : milestone, 2026,
profiling (2.305 ms) : 2066, 2544
.   : milestone, 2305,
tracing (1.85 ms) : 1817, 1883
.   : milestone, 1850,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.462 ms [1.451 ms, 1.473 ms] -
appsec 2.224 ms [2.189 ms, 2.258 ms] 761.475 µs (52.1%)
iast 1.967 ms [1.925 ms, 2.008 ms] 504.626 µs (34.5%)
iast_GLOBAL 2.024 ms [1.981 ms, 2.067 ms] 561.809 µs (38.4%)
profiling 1.871 ms [1.836 ms, 1.905 ms] 408.821 µs (28.0%)
tracing 1.85 ms [1.817 ms, 1.884 ms] 388.404 µs (26.6%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.467 ms [1.455 ms, 1.478 ms] -
appsec 2.233 ms [2.198 ms, 2.268 ms] 766.527 µs (52.3%)
iast 1.986 ms [1.943 ms, 2.029 ms] 519.189 µs (35.4%)
iast_GLOBAL 2.026 ms [1.982 ms, 2.07 ms] 559.119 µs (38.1%)
profiling 2.305 ms [2.066 ms, 2.544 ms] 837.81 µs (57.1%)
tracing 1.85 ms [1.817 ms, 1.883 ms] 382.771 µs (26.1%)

@smola smola marked this pull request as ready for review August 28, 2024 07:52
@smola smola requested a review from a team as a code owner August 28, 2024 07:52
PerfectSlayer
PerfectSlayer approved these changes Aug 28, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@PerfectSlayer PerfectSlayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice addition 👍

@smola smola merged commit 546f797 into master Aug 28, 2024
@smola smola deleted the smola/smoke-fail-on-decoding-exception branch August 28, 2024 08:16
@github-actions github-actions Bot added this to the 1.39.0 milestone Aug 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@PerfectSlayer PerfectSlayer PerfectSlayer approved these changes

@manuel-alvarez-alvarez manuel-alvarez-alvarez Awaiting requested review from manuel-alvarez-alvarez manuel-alvarez-alvarez is a code owner automatically assigned from DataDog/apm-java

Assignees

No one assigned

Labels

comp: testing Testing tag: no release notes Changes to exclude from release notes

Projects

None yet

Milestone

1.39.0

Development

Successfully merging this pull request may close these issues.

2 participants

@smola @PerfectSlayer