Skip to content

Optimize in-progress tests and suites lookup#6728

Merged
nikita-tkachenko-datadog merged 3 commits into
masterfrom
nikita-tkachenko/test-lookup-optimization
Mar 8, 2024
Merged

Optimize in-progress tests and suites lookup#6728
nikita-tkachenko-datadog merged 3 commits into
masterfrom
nikita-tkachenko/test-lookup-optimization

Conversation

@nikita-tkachenko-datadog

@nikita-tkachenko-datadog nikita-tkachenko-datadog commented Feb 23, 2024

Copy link
Copy Markdown
Contributor

What Does This Do

Optimizes lookup of in-progress test and test suites for JUnit 5-based frameworks.

ConcurrentHashMap used for storing in-progress tests and suites is being replaced with ContextStore that is backed by a field that the tracer injects into JUnit's TestDescriptor class (so a test or a test suite corresponding to a specific descriptor is obtained directly from the field).

Jira ticket: CIVIS-9262

@nikita-tkachenko-datadog nikita-tkachenko-datadog added the comp: ci visibility Continuous Integration Visibility label Feb 23, 2024
@nikita-tkachenko-datadog
nikita-tkachenko-datadog force-pushed the nikita-tkachenko/early-flakiness-detection branch from a7b55a4 to b1243c5 Compare February 23, 2024 20:40
@pr-commenter

pr-commenter Bot commented Feb 23, 2024

Copy link
Copy Markdown

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/test-lookup-optimization
git_commit_date 1709043155 1709047373
git_commit_sha 2f0211b ec35a31
release_version 1.31.0-SNAPSHOT~2f0211b9b3 1.31.0-SNAPSHOT~ec35a3182f
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1709050196 1709050196
ci_job_id 444650499 444650499
ci_pipeline_id 29025460 29025460
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 42 metrics, 12 unstable metrics.

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-02-27T15:46:18 2024-02-27T16:04:56
git_branch master nikita-tkachenko/test-lookup-optimization
git_commit_date 1709043155 1709047373
git_commit_sha 2f0211b ec35a31
release_version 1.31.0-SNAPSHOT~2f0211b9b3 1.31.0-SNAPSHOT~ec35a3182f
start_time 2024-02-27T15:46:04 2024-02-27T16:04:43
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1709050196 1709050196
ci_job_id 444650499 444650499
ci_pipeline_id 29025460 29025460
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 15 unstable metrics.

Request duration reports for insecure-bank
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.31.0-SNAPSHOT~ec35a3182f, baseline=1.31.0-SNAPSHOT~2f0211b9b3
    dateFormat X
    axisFormat %s
section baseline
no_agent (360.134 µs) : 340, 380
.   : milestone, 360,
iast (479.956 µs) : 459, 501
.   : milestone, 480,
iast_FULL (534.337 µs) : 514, 555
.   : milestone, 534,
iast_GLOBAL (502.345 µs) : 481, 524
.   : milestone, 502,
iast_HARDCODED_SECRET_DISABLED (472.089 µs) : 452, 492
.   : milestone, 472,
iast_INACTIVE (453.457 µs) : 433, 474
.   : milestone, 453,
iast_TELEMETRY_OFF (475.084 µs) : 454, 496
.   : milestone, 475,
tracing (438.553 µs) : 418, 459
.   : milestone, 439,
section candidate
no_agent (363.778 µs) : 344, 383
.   : milestone, 364,
iast (474.287 µs) : 453, 495
.   : milestone, 474,
iast_FULL (534.986 µs) : 515, 555
.   : milestone, 535,
iast_GLOBAL (496.988 µs) : 476, 518
.   : milestone, 497,
iast_HARDCODED_SECRET_DISABLED (471.714 µs) : 451, 492
.   : milestone, 472,
iast_INACTIVE (453.879 µs) : 433, 475
.   : milestone, 454,
iast_TELEMETRY_OFF (472.405 µs) : 452, 493
.   : milestone, 472,
tracing (440.537 µs) : 420, 461
.   : milestone, 441,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 360.134 µs [340.152 µs, 380.116 µs] -
iast 479.956 µs [459.045 µs, 500.867 µs] 119.822 µs (33.3%)
iast_FULL 534.337 µs [513.999 µs, 554.675 µs] 174.203 µs (48.4%)
iast_GLOBAL 502.345 µs [481.074 µs, 523.615 µs] 142.211 µs (39.5%)
iast_HARDCODED_SECRET_DISABLED 472.089 µs [451.804 µs, 492.375 µs] 111.955 µs (31.1%)
iast_INACTIVE 453.457 µs [432.605 µs, 474.308 µs] 93.323 µs (25.9%)
iast_TELEMETRY_OFF 475.084 µs [454.079 µs, 496.089 µs] 114.95 µs (31.9%)
tracing 438.553 µs [417.762 µs, 459.345 µs] 78.419 µs (21.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 363.778 µs [344.09 µs, 383.466 µs] -
iast 474.287 µs [453.275 µs, 495.299 µs] 110.509 µs (30.4%)
iast_FULL 534.986 µs [514.663 µs, 555.309 µs] 171.208 µs (47.1%)
iast_GLOBAL 496.988 µs [475.736 µs, 518.239 µs] 133.21 µs (36.6%)
iast_HARDCODED_SECRET_DISABLED 471.714 µs [451.247 µs, 492.18 µs] 107.936 µs (29.7%)
iast_INACTIVE 453.879 µs [432.793 µs, 474.966 µs] 90.101 µs (24.8%)
iast_TELEMETRY_OFF 472.405 µs [451.762 µs, 493.048 µs] 108.627 µs (29.9%)
tracing 440.537 µs [420.147 µs, 460.927 µs] 76.759 µs (21.1%)
Request duration reports for petclinic
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.31.0-SNAPSHOT~ec35a3182f, baseline=1.31.0-SNAPSHOT~2f0211b9b3
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.338 ms) : 1319, 1357
.   : milestone, 1338,
appsec (1.774 ms) : 1751, 1798
.   : milestone, 1774,
iast (1.536 ms) : 1513, 1559
.   : milestone, 1536,
profiling (1.576 ms) : 1551, 1600
.   : milestone, 1576,
tracing (1.524 ms) : 1500, 1549
.   : milestone, 1524,
section candidate
no_agent (1.35 ms) : 1331, 1370
.   : milestone, 1350,
appsec (1.755 ms) : 1731, 1778
.   : milestone, 1755,
iast (1.51 ms) : 1486, 1533
.   : milestone, 1510,
profiling (1.533 ms) : 1510, 1557
.   : milestone, 1533,
tracing (1.5 ms) : 1476, 1523
.   : milestone, 1500,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.338 ms [1.319 ms, 1.357 ms] -
appsec 1.774 ms [1.751 ms, 1.798 ms] 436.419 µs (32.6%)
iast 1.536 ms [1.513 ms, 1.559 ms] 198.077 µs (14.8%)
profiling 1.576 ms [1.551 ms, 1.6 ms] 237.667 µs (17.8%)
tracing 1.524 ms [1.5 ms, 1.549 ms] 186.552 µs (13.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.35 ms [1.331 ms, 1.37 ms] -
appsec 1.755 ms [1.731 ms, 1.778 ms] 404.14 µs (29.9%)
iast 1.51 ms [1.486 ms, 1.533 ms] 159.322 µs (11.8%)
profiling 1.533 ms [1.51 ms, 1.557 ms] 182.731 µs (13.5%)
tracing 1.5 ms [1.476 ms, 1.523 ms] 149.477 µs (11.1%)

Base automatically changed from nikita-tkachenko/early-flakiness-detection to master February 27, 2024 14:12
@nikita-tkachenko-datadog
nikita-tkachenko-datadog force-pushed the nikita-tkachenko/test-lookup-optimization branch from bed7dd7 to 05440f0 Compare February 27, 2024 14:13
@nikita-tkachenko-datadog
nikita-tkachenko-datadog marked this pull request as ready for review February 28, 2024 11:49
@nikita-tkachenko-datadog
nikita-tkachenko-datadog requested a review from a team as a code owner February 28, 2024 11:49

@tonyredondo tonyredondo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nikita-tkachenko-datadog
nikita-tkachenko-datadog merged commit 0c7e4ea into master Mar 8, 2024
@nikita-tkachenko-datadog
nikita-tkachenko-datadog deleted the nikita-tkachenko/test-lookup-optimization branch March 8, 2024 14:17
@github-actions github-actions Bot added this to the 1.32.0 milestone Mar 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: ci visibility Continuous Integration Visibility

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants