Add GHA workflow to create a release branch and pin system tests by sarahchen6 · Pull Request #9782 · DataDog/dd-trace-java

sarahchen6 · 2025-10-16T18:27:53Z

What Does This Do

This PR adds a GHA workflow that is triggered either manually or when a tag is pushed for minor releases. It also adds a corresponding script that updates the commit SHA that we use to run system tests and a dd-octo-sts trust policy for the workflow.

If the release branch does not already exist, this workflow creates a branch that corresponds to the git tag that triggered the workflow (e.g. v1.54.0 should create the branch release/v1.54.x). The workflow then updates the system-tests commit SHA in run-system-tests.yaml to the then-latest SHA available via the script and pushes this with the newly created release branch.

Motivation

We want our master branch to run with the latest head of system-tests. However, when making patch releases off of a release branch, the patch should only be tested against the head of system-tests at the time that the minor release was made. Thus, whenever a minor release is made (i.e. a git tag of the pattern vX.Y.0 is pushed), we can immediately create the release branch that pins system tests to the then-current head. Whenever we go back to this branch to make patch releases, we should now only need to test against this pinned version of system-tests instead of the latest head on master.

Additional Notes

Unfortunately, since we are using a dd-octo-sts trust policy that only works off master, I cannot test this fully until the policy and workflow are merged. The next minor release isn't planned until Nov 3rd, but in the meantime, I can test this workflow with the manual trigger.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any useful labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

…it sha after minor release

datadog-official · 2025-10-16T18:38:21Z

🎯 Code Coverage
• Patch Coverage: 100.00%
• Total Coverage: 59.70% (-0.01%)

View detailed report

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 8dd3d64 | Docs | Was this helpful? Give us feedback!}

github-actions · 2025-10-16T18:41:53Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

pr-commenter · 2025-10-16T19:10:40Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/add-release-system-tests-workflow
git_commit_date	1760719732	1760720531
git_commit_sha	`5d03203`	`8dd3d64`
release_version	1.55.0-SNAPSHOT~5d0320341a	1.55.0-SNAPSHOT~8dd3d64f72

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1760722316	1760722316
ci_job_id	1185534984	1185534984
ci_pipeline_id	79650025	79650025
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-fbyqkzty 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-fbyqkzty 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 6 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.55.0-SNAPSHOT~8dd3d64f72, baseline=1.55.0-SNAPSHOT~5d0320341a

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.019 s) : 0, 1018815
Total [baseline] (8.673 s) : 0, 8673173
Agent [candidate] (1.016 s) : 0, 1016239
Total [candidate] (8.668 s) : 0, 8667966
section iast
Agent [baseline] (1.15 s) : 0, 1150013
Total [baseline] (9.27 s) : 0, 9269697
Agent [candidate] (1.152 s) : 0, 1152280
Total [candidate] (9.276 s) : 0, 9276181

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.019 s	-
Agent	iast	1.15 s	131.198 ms (12.9%)
Total	tracing	8.673 s	-
Total	iast	9.27 s	596.524 ms (6.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.016 s	-
Agent	iast	1.152 s	136.04 ms (13.4%)
Total	tracing	8.668 s	-
Total	iast	9.276 s	608.214 ms (7.0%)

gantt
    title insecure-bank - break down per module: candidate=1.55.0-SNAPSHOT~8dd3d64f72, baseline=1.55.0-SNAPSHOT~5d0320341a

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.471 ms) : 0, 1471
crashtracking [candidate] (1.462 ms) : 0, 1462
BytebuddyAgent [baseline] (694.367 ms) : 0, 694367
BytebuddyAgent [candidate] (692.628 ms) : 0, 692628
GlobalTracer [baseline] (242.833 ms) : 0, 242833
GlobalTracer [candidate] (242.702 ms) : 0, 242702
AppSec [baseline] (32.324 ms) : 0, 32324
AppSec [candidate] (32.069 ms) : 0, 32069
Debugger [baseline] (6.308 ms) : 0, 6308
Debugger [candidate] (6.266 ms) : 0, 6266
Remote Config [baseline] (685.427 µs) : 0, 685
Remote Config [candidate] (684.165 µs) : 0, 684
Telemetry [baseline] (9.383 ms) : 0, 9383
Telemetry [candidate] (9.321 ms) : 0, 9321
Flare Poller [baseline] (10.162 ms) : 0, 10162
Flare Poller [candidate] (10.022 ms) : 0, 10022
section iast
crashtracking [baseline] (1.496 ms) : 0, 1496
crashtracking [candidate] (1.499 ms) : 0, 1499
BytebuddyAgent [baseline] (813.739 ms) : 0, 813739
BytebuddyAgent [candidate] (816.184 ms) : 0, 816184
GlobalTracer [baseline] (231.867 ms) : 0, 231867
GlobalTracer [candidate] (231.663 ms) : 0, 231663
IAST [baseline] (26.831 ms) : 0, 26831
IAST [candidate] (26.622 ms) : 0, 26622
AppSec [baseline] (34.921 ms) : 0, 34921
AppSec [candidate] (35.04 ms) : 0, 35040
Debugger [baseline] (6.174 ms) : 0, 6174
Debugger [candidate] (6.184 ms) : 0, 6184
Remote Config [baseline] (611.145 µs) : 0, 611
Remote Config [candidate] (616.186 µs) : 0, 616
Telemetry [baseline] (8.779 ms) : 0, 8779
Telemetry [candidate] (8.796 ms) : 0, 8796
Flare Poller [baseline] (4.267 ms) : 0, 4267
Flare Poller [candidate] (4.224 ms) : 0, 4224

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.55.0-SNAPSHOT~8dd3d64f72, baseline=1.55.0-SNAPSHOT~5d0320341a

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.021 s) : 0, 1021086
Total [baseline] (10.629 s) : 0, 10629406
Agent [candidate] (1.028 s) : 0, 1028073
Total [candidate] (10.813 s) : 0, 10812719
section appsec
Agent [baseline] (1.197 s) : 0, 1197348
Total [baseline] (10.785 s) : 0, 10785154
Agent [candidate] (1.218 s) : 0, 1217949
Total [candidate] (10.916 s) : 0, 10915520
section iast
Agent [baseline] (1.152 s) : 0, 1152101
Total [baseline] (11.027 s) : 0, 11027219
Agent [candidate] (1.151 s) : 0, 1150737
Total [candidate] (11.074 s) : 0, 11074470
section profiling
Agent [baseline] (1.17 s) : 0, 1170066
Total [baseline] (10.889 s) : 0, 10888954
Agent [candidate] (1.187 s) : 0, 1186650
Total [candidate] (11.02 s) : 0, 11019792

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.021 s	-
Agent	appsec	1.197 s	176.261 ms (17.3%)
Agent	iast	1.152 s	131.014 ms (12.8%)
Agent	profiling	1.17 s	148.98 ms (14.6%)
Total	tracing	10.629 s	-
Total	appsec	10.785 s	155.748 ms (1.5%)
Total	iast	11.027 s	397.813 ms (3.7%)
Total	profiling	10.889 s	259.548 ms (2.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.028 s	-
Agent	appsec	1.218 s	189.876 ms (18.5%)
Agent	iast	1.151 s	122.664 ms (11.9%)
Agent	profiling	1.187 s	158.577 ms (15.4%)
Total	tracing	10.813 s	-
Total	appsec	10.916 s	102.801 ms (1.0%)
Total	iast	11.074 s	261.752 ms (2.4%)
Total	profiling	11.02 s	207.073 ms (1.9%)

gantt
    title petclinic - break down per module: candidate=1.55.0-SNAPSHOT~8dd3d64f72, baseline=1.55.0-SNAPSHOT~5d0320341a

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.471 ms) : 0, 1471
crashtracking [candidate] (1.494 ms) : 0, 1494
BytebuddyAgent [baseline] (694.708 ms) : 0, 694708
BytebuddyAgent [candidate] (701.058 ms) : 0, 701058
GlobalTracer [baseline] (243.444 ms) : 0, 243444
GlobalTracer [candidate] (244.397 ms) : 0, 244397
AppSec [baseline] (32.112 ms) : 0, 32112
AppSec [candidate] (32.421 ms) : 0, 32421
Debugger [baseline] (6.294 ms) : 0, 6294
Debugger [candidate] (6.353 ms) : 0, 6353
Remote Config [baseline] (677.579 µs) : 0, 678
Remote Config [candidate] (688.516 µs) : 0, 689
Telemetry [baseline] (9.347 ms) : 0, 9347
Telemetry [candidate] (9.337 ms) : 0, 9337
Flare Poller [baseline] (11.745 ms) : 0, 11745
Flare Poller [candidate] (10.945 ms) : 0, 10945
section appsec
crashtracking [baseline] (1.472 ms) : 0, 1472
crashtracking [candidate] (1.499 ms) : 0, 1499
BytebuddyAgent [baseline] (719.607 ms) : 0, 719607
BytebuddyAgent [candidate] (733.284 ms) : 0, 733284
GlobalTracer [baseline] (235.6 ms) : 0, 235600
GlobalTracer [candidate] (239.335 ms) : 0, 239335
IAST [baseline] (24.911 ms) : 0, 24911
IAST [candidate] (25.591 ms) : 0, 25591
AppSec [baseline] (175.536 ms) : 0, 175536
AppSec [candidate] (177.102 ms) : 0, 177102
Debugger [baseline] (6.101 ms) : 0, 6101
Debugger [candidate] (6.18 ms) : 0, 6180
Remote Config [baseline] (622.854 µs) : 0, 623
Remote Config [candidate] (643.598 µs) : 0, 644
Telemetry [baseline] (8.425 ms) : 0, 8425
Telemetry [candidate] (8.777 ms) : 0, 8777
Flare Poller [baseline] (3.865 ms) : 0, 3865
Flare Poller [candidate] (4.054 ms) : 0, 4054
section iast
crashtracking [baseline] (1.484 ms) : 0, 1484
crashtracking [candidate] (1.463 ms) : 0, 1463
BytebuddyAgent [baseline] (815.09 ms) : 0, 815090
BytebuddyAgent [candidate] (814.98 ms) : 0, 814980
GlobalTracer [baseline] (231.972 ms) : 0, 231972
GlobalTracer [candidate] (231.664 ms) : 0, 231664
IAST [baseline] (26.934 ms) : 0, 26934
IAST [candidate] (26.489 ms) : 0, 26489
AppSec [baseline] (35.177 ms) : 0, 35177
AppSec [candidate] (35.073 ms) : 0, 35073
Debugger [baseline] (6.167 ms) : 0, 6167
Debugger [candidate] (6.106 ms) : 0, 6106
Remote Config [baseline] (619.568 µs) : 0, 620
Remote Config [candidate] (592.818 µs) : 0, 593
Telemetry [baseline] (8.892 ms) : 0, 8892
Telemetry [candidate] (8.691 ms) : 0, 8691
Flare Poller [baseline] (4.234 ms) : 0, 4234
Flare Poller [candidate] (4.236 ms) : 0, 4236
section profiling
crashtracking [baseline] (1.471 ms) : 0, 1471
crashtracking [candidate] (1.488 ms) : 0, 1488
BytebuddyAgent [baseline] (723.688 ms) : 0, 723688
BytebuddyAgent [candidate] (734.08 ms) : 0, 734080
GlobalTracer [baseline] (219.994 ms) : 0, 219994
GlobalTracer [candidate] (222.47 ms) : 0, 222470
AppSec [baseline] (32.607 ms) : 0, 32607
AppSec [candidate] (32.986 ms) : 0, 32986
Debugger [baseline] (7.486 ms) : 0, 7486
Debugger [candidate] (7.678 ms) : 0, 7678
Remote Config [baseline] (720.365 µs) : 0, 720
Remote Config [candidate] (1.485 ms) : 0, 1485
Telemetry [baseline] (15.249 ms) : 0, 15249
Telemetry [candidate] (13.223 ms) : 0, 13223
Flare Poller [baseline] (4.157 ms) : 0, 4157
Flare Poller [candidate] (5.886 ms) : 0, 5886
ProfilingAgent [baseline] (110.565 ms) : 0, 110565
ProfilingAgent [candidate] (112.593 ms) : 0, 112593
Profiling [baseline] (111.212 ms) : 0, 111212
Profiling [candidate] (113.22 ms) : 0, 113220

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/add-release-system-tests-workflow
git_commit_date	1760719732	1760720531
git_commit_sha	`5d03203`	`8dd3d64`
release_version	1.55.0-SNAPSHOT~5d0320341a	1.55.0-SNAPSHOT~8dd3d64f72

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1760721980	1760721980
ci_job_id	1185534986	1185534986
ci_pipeline_id	79650025	79650025
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-3-ptpx5vvs 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-3-ptpx5vvs 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 3 performance improvements and 2 performance regressions! Performance is the same for 7 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:insecure-bank:profiling:high_load	worse [+286.818µs; +612.619µs] or [+3.235%; +6.910%]	unstable [-98.389op/s; +48.139op/s] or [-18.801%; +9.199%]	9.315ms	498.188op/s	8.866ms	523.312op/s
scenario:load:insecure-bank:iast_FULL:high_load	better [-1.911ms; -1.282ms] or [-12.419%; -8.333%]	unstable [+0.546op/s; +87.380op/s] or [+0.186%; +29.740%]	13.792ms	337.781op/s	15.389ms	293.818op/s
scenario:load:insecure-bank:iast:high_load	worse [+312.667µs; +672.072µs] or [+3.248%; +6.981%]	unstable [-86.469op/s; +40.719op/s] or [-17.927%; +8.442%]	10.119ms	459.469op/s	9.627ms	482.344op/s
scenario:load:insecure-bank:iast_GLOBAL:high_load	better [-1158.588µs; -754.271µs] or [-10.696%; -6.963%]	unstable [-10.346op/s; +118.669op/s] or [-2.486%; +28.514%]	9.876ms	470.344op/s	10.832ms	416.182op/s
scenario:load:petclinic:profiling:high_load	better [-3.108ms; -2.108ms] or [-6.252%; -4.239%]	unstable [-2.533op/s; +12.883op/s] or [-2.690%; +13.683%]	47.111ms	99.325op/s	49.719ms	94.150op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.55.0-SNAPSHOT~8dd3d64f72, baseline=1.55.0-SNAPSHOT~5d0320341a
    dateFormat X
    axisFormat %s
section baseline
no_agent (37.139 ms) : 36832, 37445
.   : milestone, 37139,
appsec (47.802 ms) : 47367, 48236
.   : milestone, 47802,
code_origins (43.307 ms) : 42927, 43687
.   : milestone, 43307,
iast (43.726 ms) : 43366, 44086
.   : milestone, 43726,
profiling (49.719 ms) : 49221, 50216
.   : milestone, 49719,
tracing (44.503 ms) : 44134, 44871
.   : milestone, 44503,
section candidate
no_agent (37.445 ms) : 37131, 37758
.   : milestone, 37445,
appsec (46.91 ms) : 46510, 47310
.   : milestone, 46910,
code_origins (43.406 ms) : 43032, 43780
.   : milestone, 43406,
iast (42.994 ms) : 42613, 43376
.   : milestone, 42994,
profiling (47.111 ms) : 46681, 47541
.   : milestone, 47111,
tracing (43.896 ms) : 43515, 44277
.   : milestone, 43896,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.139 ms [36.832 ms, 37.445 ms]	-
appsec	47.802 ms [47.367 ms, 48.236 ms]	10.663 ms (28.7%)
code_origins	43.307 ms [42.927 ms, 43.687 ms]	6.168 ms (16.6%)
iast	43.726 ms [43.366 ms, 44.086 ms]	6.587 ms (17.7%)
profiling	49.719 ms [49.221 ms, 50.216 ms]	12.58 ms (33.9%)
tracing	44.503 ms [44.134 ms, 44.871 ms]	7.364 ms (19.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.445 ms [37.131 ms, 37.758 ms]	-
appsec	46.91 ms [46.51 ms, 47.31 ms]	9.465 ms (25.3%)
code_origins	43.406 ms [43.032 ms, 43.78 ms]	5.962 ms (15.9%)
iast	42.994 ms [42.613 ms, 43.376 ms]	5.55 ms (14.8%)
profiling	47.111 ms [46.681 ms, 47.541 ms]	9.666 ms (25.8%)
tracing	43.896 ms [43.515 ms, 44.277 ms]	6.451 ms (17.2%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.55.0-SNAPSHOT~8dd3d64f72, baseline=1.55.0-SNAPSHOT~5d0320341a
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.249 ms) : 4192, 4306
.   : milestone, 4249,
iast (9.627 ms) : 9469, 9785
.   : milestone, 9627,
iast_FULL (15.389 ms) : 15080, 15698
.   : milestone, 15389,
iast_GLOBAL (10.832 ms) : 10630, 11035
.   : milestone, 10832,
profiling (8.866 ms) : 8711, 9020
.   : milestone, 8866,
tracing (7.829 ms) : 7710, 7947
.   : milestone, 7829,
section candidate
no_agent (4.367 ms) : 4317, 4417
.   : milestone, 4367,
iast (10.119 ms) : 9944, 10295
.   : milestone, 10119,
iast_FULL (13.792 ms) : 13518, 14066
.   : milestone, 13792,
iast_GLOBAL (9.876 ms) : 9704, 10048
.   : milestone, 9876,
profiling (9.315 ms) : 9167, 9464
.   : milestone, 9315,
tracing (7.83 ms) : 7708, 7952
.   : milestone, 7830,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.249 ms [4.192 ms, 4.306 ms]	-
iast	9.627 ms [9.469 ms, 9.785 ms]	5.378 ms (126.6%)
iast_FULL	15.389 ms [15.08 ms, 15.698 ms]	11.14 ms (262.2%)
iast_GLOBAL	10.832 ms [10.63 ms, 11.035 ms]	6.584 ms (155.0%)
profiling	8.866 ms [8.711 ms, 9.02 ms]	4.617 ms (108.7%)
tracing	7.829 ms [7.71 ms, 7.947 ms]	3.58 ms (84.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.367 ms [4.317 ms, 4.417 ms]	-
iast	10.119 ms [9.944 ms, 10.295 ms]	5.753 ms (131.7%)
iast_FULL	13.792 ms [13.518 ms, 14.066 ms]	9.425 ms (215.8%)
iast_GLOBAL	9.876 ms [9.704 ms, 10.048 ms]	5.509 ms (126.2%)
profiling	9.315 ms [9.167 ms, 9.464 ms]	4.948 ms (113.3%)
tracing	7.83 ms [7.708 ms, 7.952 ms]	3.463 ms (79.3%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/add-release-system-tests-workflow
git_commit_date	1760719732	1760720531
git_commit_sha	`5d03203`	`8dd3d64`
release_version	1.55.0-SNAPSHOT~5d0320341a	1.55.0-SNAPSHOT~8dd3d64f72

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1760722513	1760722513
ci_job_id	1185534988	1185534988
ci_pipeline_id	79650025	79650025
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-2-o7kecnrv 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-2-o7kecnrv 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.55.0-SNAPSHOT~8dd3d64f72, baseline=1.55.0-SNAPSHOT~5d0320341a
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.474 ms) : 1462, 1485
.   : milestone, 1474,
appsec (3.713 ms) : 3495, 3930
.   : milestone, 3713,
iast (2.207 ms) : 2143, 2270
.   : milestone, 2207,
iast_GLOBAL (2.24 ms) : 2176, 2304
.   : milestone, 2240,
profiling (2.053 ms) : 2001, 2105
.   : milestone, 2053,
tracing (2.017 ms) : 1967, 2066
.   : milestone, 2017,
section candidate
no_agent (1.472 ms) : 1460, 1483
.   : milestone, 1472,
appsec (3.75 ms) : 3526, 3973
.   : milestone, 3750,
iast (2.209 ms) : 2145, 2272
.   : milestone, 2209,
iast_GLOBAL (2.252 ms) : 2188, 2316
.   : milestone, 2252,
profiling (2.042 ms) : 1991, 2093
.   : milestone, 2042,
tracing (2.021 ms) : 1972, 2071
.   : milestone, 2021,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.474 ms [1.462 ms, 1.485 ms]	-
appsec	3.713 ms [3.495 ms, 3.93 ms]	2.239 ms (151.9%)
iast	2.207 ms [2.143 ms, 2.27 ms]	732.888 µs (49.7%)
iast_GLOBAL	2.24 ms [2.176 ms, 2.304 ms]	766.788 µs (52.0%)
profiling	2.053 ms [2.001 ms, 2.105 ms]	579.231 µs (39.3%)
tracing	2.017 ms [1.967 ms, 2.066 ms]	542.988 µs (36.8%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.472 ms [1.46 ms, 1.483 ms]	-
appsec	3.75 ms [3.526 ms, 3.973 ms]	2.278 ms (154.8%)
iast	2.209 ms [2.145 ms, 2.272 ms]	737.159 µs (50.1%)
iast_GLOBAL	2.252 ms [2.188 ms, 2.316 ms]	780.678 µs (53.0%)
profiling	2.042 ms [1.991 ms, 2.093 ms]	570.355 µs (38.8%)
tracing	2.021 ms [1.972 ms, 2.071 ms]	549.586 µs (37.3%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.55.0-SNAPSHOT~8dd3d64f72, baseline=1.55.0-SNAPSHOT~5d0320341a
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.893 s) : 14893000, 14893000
.   : milestone, 14893000,
appsec (15.046 s) : 15046000, 15046000
.   : milestone, 15046000,
iast (18.49 s) : 18490000, 18490000
.   : milestone, 18490000,
iast_GLOBAL (18.113 s) : 18113000, 18113000
.   : milestone, 18113000,
profiling (15.389 s) : 15389000, 15389000
.   : milestone, 15389000,
tracing (15.326 s) : 15326000, 15326000
.   : milestone, 15326000,
section candidate
no_agent (14.922 s) : 14922000, 14922000
.   : milestone, 14922000,
appsec (14.837 s) : 14837000, 14837000
.   : milestone, 14837000,
iast (18.855 s) : 18855000, 18855000
.   : milestone, 18855000,
iast_GLOBAL (17.725 s) : 17725000, 17725000
.   : milestone, 17725000,
profiling (15.074 s) : 15074000, 15074000
.   : milestone, 15074000,
tracing (15.358 s) : 15358000, 15358000
.   : milestone, 15358000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.893 s [14.893 s, 14.893 s]	-
appsec	15.046 s [15.046 s, 15.046 s]	153.0 ms (1.0%)
iast	18.49 s [18.49 s, 18.49 s]	3.597 s (24.2%)
iast_GLOBAL	18.113 s [18.113 s, 18.113 s]	3.22 s (21.6%)
profiling	15.389 s [15.389 s, 15.389 s]	496.0 ms (3.3%)
tracing	15.326 s [15.326 s, 15.326 s]	433.0 ms (2.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.922 s [14.922 s, 14.922 s]	-
appsec	14.837 s [14.837 s, 14.837 s]	-85.0 ms (-0.6%)
iast	18.855 s [18.855 s, 18.855 s]	3.933 s (26.4%)
iast_GLOBAL	17.725 s [17.725 s, 17.725 s]	2.803 s (18.8%)
profiling	15.074 s [15.074 s, 15.074 s]	152.0 ms (1.0%)
tracing	15.358 s [15.358 s, 15.358 s]	436.0 ms (2.9%)

PerfectSlayer · 2025-10-17T07:58:49Z

+        id: define-branch
+        run: |
+          TAG=${{ steps.determine-tag.outputs.tag }}
+          BRANCH=$(echo "$TAG" | sed -E 's/^(v[0-9]+\.[0-9]+)\.0$/release\/\1.x/')


💭 thought: ‏This \1.x is so confusing 😵

Yeah 😬 I believe it's taking the first matched argument and adding .x instead of .0, but let me see if this can be simplified.

💡 Since I'm already checking that the $TAG is in the correct format, I only need to replace the trailing 0 with x !

PerfectSlayer · 2025-10-17T07:59:45Z

+          echo "tag=${TAG}" >> "$GITHUB_OUTPUT"
+          echo "Processing release tag: ${TAG}"
+
+      - name: Validate tag format


🎯 suggestion: ‏I would merge determine tag and validate tag format for simplicity.

PerfectSlayer · 2025-10-17T08:16:12Z

+      - name: Checkout system-tests to get latest SHA
+        if: steps.check-branch.outputs.exists == 'false'
+        id: system-test-ref
+        uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # 5.0.0
+        with:
+          repository: "DataDog/system-tests"
+          path: system-tests
+          ref: main
+
+      - name: Update reference 1/2 in run-system-tests.yaml
+        if: steps.check-branch.outputs.exists == 'false'
+        run: .github/scripts/update_system_test_reference.sh
+        env:
+          TARGET: ".github/workflows/run-system-tests.yaml"
+          PATTERN: '(\s*system-tests\.yml@)(\S+)(\s+# system tests.*)'
+          REF: ${{ steps.system-test-ref.outputs.commit }}
+
+      - name: Update reference 2/2 in run-system-tests.yaml
+        if: steps.check-branch.outputs.exists == 'false'
+        run: .github/scripts/update_system_test_reference.sh
+        env:
+          TARGET: ".github/workflows/run-system-tests.yaml"
+          PATTERN: '(\s*ref: )(\S+)(\s+# system tests.*)'
+          REF: ${{ steps.system-test-ref.outputs.commit }}


🎯 suggestion: ‏I would simplify this part too with the following changes:

First I would specialize the script update_system_test_reference.sh to take the desired system test branch as parameter. It would make all the changes needed in the run-system-tests workflow (and even elsewhere if needed). This will also simplify the pattern cleaning thing -- you can implement all the str replace there directly.
We used to have a such script from Santi. It make sense to move it to /tooling too.

Then, you don't need to checkout the system test repo to get the SHA1 of a branch.
You can use git ls-remote https://github.com/DataDog/system-tests <branch-name> and then cut -f 1 like:

COMMIT_REF=$(git ls-remote https://github.com/DataDog/system-tests <branch-name> | cut -f 1)

Oh thanks, good idea. Will do!

Wow much cleaner: dae2904

PerfectSlayer

👏 praise: ‏Looks way simpler now :)

sarahchen6 added 2 commits October 16, 2025 14:21

Add GHA workflow to create a release branch and pin system tests comm…

805a42e

…it sha after minor release

Update policy name in workflow

c0a94e8

sarahchen6 marked this pull request as ready for review October 16, 2025 18:41

sarahchen6 requested a review from a team as a code owner October 16, 2025 18:41

sarahchen6 requested review from AlexeyKuznetsov-DD and removed request for a team October 16, 2025 18:41

sarahchen6 added type: enhancement Enhancements and improvements tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Oct 16, 2025

PerfectSlayer reviewed Oct 17, 2025

View reviewed changes

Clean workflow and script

dae2904

sarahchen6 requested a review from a team as a code owner October 17, 2025 15:29

sarahchen6 requested review from PerfectSlayer and amarziali October 17, 2025 15:29

Merge branch 'master' into sarahchen6/add-release-system-tests-workflow

8dd3d64

PerfectSlayer approved these changes Oct 20, 2025

View reviewed changes

sarahchen6 merged commit fb31324 into master Oct 20, 2025
537 checks passed

sarahchen6 deleted the sarahchen6/add-release-system-tests-workflow branch October 20, 2025 13:38

github-actions Bot added this to the 1.55.0 milestone Oct 20, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add GHA workflow to create a release branch and pin system tests#9782

Add GHA workflow to create a release branch and pin system tests#9782
sarahchen6 merged 4 commits intomasterfrom
sarahchen6/add-release-system-tests-workflow

sarahchen6 commented Oct 16, 2025 •

edited

Loading

Uh oh!

datadog-official Bot commented Oct 16, 2025 •

edited by datadog-datadog-prod-us1 Bot

Loading

Uh oh!

github-actions Bot commented Oct 16, 2025 •

edited

Loading

Uh oh!

pr-commenter Bot commented Oct 16, 2025 •

edited

Loading

Uh oh!

PerfectSlayer Oct 17, 2025

Uh oh!

sarahchen6 Oct 17, 2025

Uh oh!

sarahchen6 Oct 17, 2025

Uh oh!

PerfectSlayer Oct 17, 2025

Uh oh!

PerfectSlayer Oct 17, 2025

Uh oh!

sarahchen6 Oct 17, 2025 •

edited

Loading

Uh oh!

sarahchen6 Oct 17, 2025

Uh oh!

PerfectSlayer left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

sarahchen6 commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

datadog-official Bot commented Oct 16, 2025 • edited by datadog-datadog-prod-us1 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter Bot commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

PerfectSlayer Oct 17, 2025

Choose a reason for hiding this comment

Uh oh!

sarahchen6 Oct 17, 2025

Choose a reason for hiding this comment

Uh oh!

sarahchen6 Oct 17, 2025

Choose a reason for hiding this comment

Uh oh!

PerfectSlayer Oct 17, 2025

Choose a reason for hiding this comment

Uh oh!

PerfectSlayer Oct 17, 2025

Choose a reason for hiding this comment

Uh oh!

sarahchen6 Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sarahchen6 Oct 17, 2025

Choose a reason for hiding this comment

Uh oh!

PerfectSlayer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

sarahchen6 commented Oct 16, 2025 •

edited

Loading

datadog-official Bot commented Oct 16, 2025 •

edited by datadog-datadog-prod-us1 Bot

Loading

github-actions Bot commented Oct 16, 2025 •

edited

Loading

pr-commenter Bot commented Oct 16, 2025 •

edited

Loading

sarahchen6 Oct 17, 2025 •

edited

Loading