Skip to content

DataDog/rules_test_optimization

Repository files navigation

Datadog Test Optimization Bazel Module Extension

This repository provides Bazel integrations that fetch Datadog Test Optimization metadata during module/repository resolution and materialize JSON files for use in your build. It also generates public filegroups so consumers can depend on stable labels instead of wiring files manually.

Release status: module metadata currently uses 1.2.0, but Bazel Central Registry publication is still pending. Until BCR entries are published, install with git_override (Bzlmod) or commit-pinned git_repository / http_archive (WORKSPACE).

Onboarding paths

Pick the path that matches your repository:

  • Per-language onboarding guide: docs/Language_Onboarding.md
  • Bzlmod + core only (any language): sync + uploader integration without language-specific macros
  • Bzlmod + Go companion: dd_topt_go_test macro with importpath inference
  • Bzlmod + Python companion: dd_topt_py_test macro with analysis-time selection
  • Bzlmod + Java companion: dd_topt_java_test macro with analysis-time selection
  • Bzlmod + NodeJS companion: dd_topt_nodejs_test macro with analysis-time selection
  • Bzlmod + .NET companion: dd_topt_dotnet_test macro with analysis-time selection
  • Bzlmod + Ruby companion: dd_topt_ruby_test macro with analysis-time selection
  • Bzlmod + multi-service monorepo: one sync extension, per-service labels/exports
  • WORKSPACE mode: fully supported for v1 when Bzlmod is disabled, including Go, Python, and Java companion helpers
  • Other languages: use core sync/uploader now, or follow companion patterns for custom dd_topt_<lang>_test modules

Documentation map

Use this map to pick the right document instead of guessing from filenames. Repository behavior is source-of-truth in this repo's code/tests; external docs are informative and may lag temporarily.

Maintainer note on the vendored rules_go fork

The repository publishes the Go integration as complete base rules_go trees. rules_go_variant currently supports only:

  • base: generic Orchestrion-enabled support.
  • tools/tests/rules_go_variant_regressions: maintainer-only regression overlay copied into temporary smoke-test trees.

When multiple upstream rules_go versions are supported, rules_go_upstream selects the upstream support line. Omitting rules_go_upstream preserves the repository default. The default rules_go_upstream is currently v0_60_0, which preserves the existing third_party/rgo/v0_60_0/base path.

Maintainers track each supported upstream version with both patch series under third_party/rules_go_orchestrion/patches/<upstream>/ and materialized base trees under the registry-selected tree_path. The patch series is the maintainer source for rebasing. The materialized tree is the consumer artifact. CI verifies that they match.

Public consumer patch profiles live under third_party/rules_go_orchestrion/profiles/. They are generated and verified by tools/dev/generate_rules_go_consumer_patch.py and tools/dev/verify_rules_go_profiles.py so consumers that already own a rules_go patch stack can generate a sparse, profile-specific input and rebase or merge it locally instead of vendoring a second complete rules_go tree.

First-run checklist (all scenarios)

Use this checklist before your first CI rollout:

  1. Keep the generated repo name as test_optimization_data (or consistently replace it in labels/commands if you choose another name).
  2. Forward sync metadata environment variable names in .bazelrc under a named config, then use that config for test, doctor, and upload commands:
    • common:test-optimization --repo_env=DD_API_KEY
    • common:test-optimization --repo_env=DD_SITE
    • common:test-optimization --repo_env=DD_GIT_REPOSITORY_URL
    • common:test-optimization --repo_env=DD_GIT_BRANCH
    • common:test-optimization --repo_env=DD_GIT_TAG
    • common:test-optimization --repo_env=DD_GIT_COMMIT_SHA
    • common:test-optimization --repo_env=DD_PR_NUMBER
    • test:test-optimization --remote_download_minimal
    • test:test-optimization --remote_download_regex=.*test[.]outputs.*
    • test:test-optimization --zip_undeclared_test_outputs
    • DD_GIT_* must use --repo_env, never --test_env, so Git metadata does not become part of the test action cache key.
    • DD_TEST_OPTIMIZATION_AGENT_URL and DD_TEST_OPTIMIZATION_AGENTLESS_URL are not part of the Go/Orchestrion test sandbox. The uploader reads upload endpoints at bazel run time.
    • FETCH_SALT is not part of the normal flow. Use it only for a separate, explicit force-refresh sync when you intentionally want fresh backend metadata.
  3. Create exactly one logical doctor/uploader pair. Small repos may put those targets in the root package; large monorepos should prefer a lightweight package such as //tools/test_optimization.
  4. Use the named test config for BwoB, selective test.outputs materialization, and zipped undeclared outputs. Do not store --build_event_json_file in .bazelrc; CI wrappers should create a fresh BEP file path for each Bazel test invocation so parallel jobs cannot overwrite each other.
  5. Configure doctor, enrichment dry-run, and uploader with the matching BEP files and BEP artifact staging through wrapper arguments:
    • --bep-json=<path> once per Bazel test invocation
    • --freshness-source=bep
    • --freshness-mode=required
    • --artifact-source=bep
    • --artifact-staging-dir=<per-run-temp-dir> Keep --remote-artifacts=disabled unless BEP references remote-only artifacts that Bazel did not materialize locally. HTTP/HTTPS outputs.zip carriers can use native staging; bytestream/CAS/custom-auth providers need a downloader.

Published Go onboarding pins

When a repository consumes this project from GitHub, pin a commit that is already reachable from origin/main. Do not publish snippets that point at a feature-branch commit: those commits can disappear after squash merge and break consumer resolution.

Maintainers can generate the full WORKSPACE/archive tuple from this checkout:

./bazelw run //tools/dev:print_go_onboarding_pins -- \
  --commit "$(git rev-parse origin/main)" \
  --rules-go-upstream v0_60_0 \
  --variant base \
  --verify-main-reachable

The helper verifies the selected rules_go variant exists, verifies the commit is reachable from origin/main, downloads the real GitHub codeload archive, and prints RTO_COMMIT, RTO_ARCHIVE_URL, RTO_ARCHIVE_SHA256, RTO_ARCHIVE_PREFIX, the selected upstream support line, the selected variant, the default dd-trace-go version, and the default Orchestrion version. Published GitHub pins use tar.gz; other archive types must be mirrored and documented outside this helper.

Current v1.2.0 GitHub pins:

RTO_COMMIT="69953536d4ef1252c8181c267d16c61263f0aa4c"
RTO_REMOTE="https://github.com/DataDog/rules_test_optimization.git"
RTO_ARCHIVE_URL="https://codeload.github.com/DataDog/rules_test_optimization/tar.gz/69953536d4ef1252c8181c267d16c61263f0aa4c"
RTO_ARCHIVE_SHA256="fd54d1871fc01ff0bb3db190dfaadaa8256edd68a4f3bb85ecc08b315fbf5bd4"
RTO_ARCHIVE_PREFIX="rules_test_optimization-69953536d4ef1252c8181c267d16c61263f0aa4c"
RTO_ARCHIVE_TYPE="tar.gz"
RULES_GO_UPSTREAM="v0_60_0"
RULES_GO_VARIANT="base"
RULES_GO_STRIP_PREFIX="third_party/rgo/v0_60_0/base"
DD_TRACE_GO_VERSION="v2.9.0"
ORCHESTRION_VERSION="v1.9.0"

The archive URL, SHA256, and prefix are tied to the repository commit.

Consumers using the Go bootstrap can print the same tuple or write a repository-local summary:

bazel run @datadog-rules-test-optimization-go//:dd_topt_go_bootstrap -- \
  --print-published-pins \
  --rto-commit <published-origin-main-sha> \
  --rules-go-upstream v0_60_0 \
  --rules-go-variant base

bazel run @datadog-rules-test-optimization-go//:dd_topt_go_bootstrap -- \
  --write-onboarding-summary=TEST_OPTIMIZATION_GUIDE.md \
  --rto-commit <published-origin-main-sha> \
  --rules-go-upstream v0_60_0 \
  --rules-go-variant base

If the bootstrap runs from a checkout that has this repository's Git history, add --verify-main-reachable to enforce the same origin/main reachability check as the dev helper.

Quickstart by scenario

Bzlmod + core only (any language)

Use this when you do not need dd_topt_go_test:

# MODULE.bazel
bazel_dep(name = "datadog-rules-test-optimization", version = "1.2.0")
git_override(
    module_name = "datadog-rules-test-optimization",
    remote = "https://github.com/DataDog/rules_test_optimization.git",
    commit = "<commit-sha>",
)

test_optimization_sync = use_extension(
    "@datadog-rules-test-optimization//tools/core:test_optimization_sync.bzl",
    "test_optimization_sync_extension",
)
test_optimization_sync.test_optimization_sync(
    name = "test_optimization_data",
    service = "my-service",  # or set DD_SERVICE via --repo_env
)
use_repo(test_optimization_sync, "test_optimization_data")
# BUILD.bazel (workspace root)
load("@datadog-rules-test-optimization//tools/core:test_optimization_uploader.bzl", "dd_payload_uploader")
load("@datadog-rules-test-optimization//tools/core:test_optimization_doctor.bzl", "dd_test_optimization_doctor")

dd_test_optimization_doctor(
    name = "dd_test_optimization_doctor",
    data = ["@test_optimization_data//:test_optimization_context"],
)

dd_payload_uploader(
    name = "dd_upload_payloads",
    data = ["@test_optimization_data//:test_optimization_context"],
)

For first-pass troubleshooting after a test run, ask the customer for the doctor bundle:

bazel run //:dd_test_optimization_doctor -- \
  --support-bundle .topt/reports/dd-test-optimization-support.zip

When diagnosing BEP/BwoB CI behavior, pass the same --bep-json, --freshness-*, and --artifact-* flags you would pass to the doctor manually. The doctor bundle is doctor-only; it does not prove uploader dry-run, enrichment, or upload behavior. For the full escalation ladder and the order to inspect files inside the zip, see Collect diagnostic reports.

# Copy or vendor the full tools/test_optimization/ helper directory into the
# consumer repo. The support bundle option needs create_support_bundle.py beside
# the wrapper, or DD_TEST_OPTIMIZATION_SUPPORT_BUNDLE_COLLECTOR pointing to it.
tools/test_optimization/run_test_optimization_ci.sh \
  --report-dir .topt/reports \
  --support-bundle .topt/reports/dd-test-optimization-support.zip \
  //...

# Add --upload only when the real upload should run after doctor and dry-run pass.
DD_API_KEY="$DD_API_KEY" DD_SITE="$DD_SITE" \
  tools/test_optimization/run_test_optimization_ci.sh \
    --report-dir .topt/reports \
    --support-bundle .topt/reports/dd-test-optimization-support.zip \
    --upload \
    //...
# Copy or vendor the full tools/test_optimization/ helper directory into the
# consumer repo. The support bundle option needs create_support_bundle.py beside
# the wrapper, or DD_TEST_OPTIMIZATION_SUPPORT_BUNDLE_COLLECTOR pointing to it.
.\tools\test_optimization\run_test_optimization_ci.ps1 `
  -ReportDir .topt\reports `
  -SupportBundle .topt\reports\dd-test-optimization-support.zip `
  //...

# Add -Upload only when the real upload should run after doctor and dry-run pass.
$env:DD_API_KEY = "<your-api-key>"
$env:DD_SITE = "datadoghq.com"
.\tools\test_optimization\run_test_optimization_ci.ps1 `
  -ReportDir .topt\reports `
  -SupportBundle .topt\reports\dd-test-optimization-support.zip `
  -Upload `
  //...

Bzlmod + Go companion (dd_topt_go_test)

For a fresh single-service Go workspace, start with the small manual prerequisite block below, then let the guided bootstrap finish the Go-specific setup.

bazel_dep(name = "datadog-rules-test-optimization", version = "1.2.0")
git_override(
    module_name = "datadog-rules-test-optimization",
    remote = "https://github.com/DataDog/rules_test_optimization.git",
    commit = "<commit-sha>",
)

bazel_dep(name = "datadog-rules-test-optimization-go", version = "1.2.0")
git_override(
    module_name = "datadog-rules-test-optimization-go",
    remote = "https://github.com/DataDog/rules_test_optimization.git",
    commit = "<commit-sha>",
    strip_prefix = "modules/go",
)
bazel_dep(name = "rules_go", version = "0.60.0")

Then run the Datadog bootstrap helper once from the workspace that owns your Go module. --dd-trace-go-version is optional; if you omit it, the default is v2.9.0.

bazel run @datadog-rules-test-optimization-go//:dd_topt_go_bootstrap -- \
  --guided \
  --service go-service \
  --runtime-version 1.25.0 \
  --dd-trace-go-version v2.9.0 \
  --write-bazelrc

If the Go module lives below the workspace root:

bazel run @datadog-rules-test-optimization-go//:dd_topt_go_bootstrap -- \
  --guided \
  --service go-service \
  --runtime-version 1.25.0 \
  --dd-trace-go-version v2.9.0 \
  --go-module-dir path/to/go-module \
  --write-bazelrc

--dd-trace-go-version accepts a normal tag, a pseudo-version, a branch, or a commit SHA. Bootstrap resolves that input to exact Go module versions before it writes anything back to the workspace. If you rerun bootstrap without --dd-trace-go-version, it preserves the managed tracer config that is already in place.

The bootstrap helper:

  • updates MODULE.bazel with a Datadog-managed rules_go override back to this repository's selected clean vendored base module and the @rules_go//go:extensions.bzl Orchestrion wiring required for Bazel builds
  • adds the Datadog-managed single-service Go sync block (test_optimization_go_extension)
  • creates a root dd_test_optimization_doctor target when missing
  • creates a root dd_upload_payloads target when missing
  • can print or write the recommended .bazelrc block with --print-bazelrc-snippet or --write-bazelrc
  • creates //tools/build:dd_go_test.bzl for workspace-local Go tests
  • configures that wrapper with orchestrion_mode = "test_optimization" for the standard Go testing Test Optimization path
  • writes a deterministic orchestrion.tool.go that matches the Bazel-side Orchestrion wiring
  • repins dd-trace-go and the Orchestrion-managed Go helper packages to the resolved tracer versions
  • writes a starter orchestrion.yml when missing
  • writes either dd_trace_go_version or dd_trace_go_versions into the managed MODULE.bazel block, depending on whether the traced Go modules resolve to one shared version or different exact versions
  • keeps Bazel's injected tracer versions and the local Go module pins aligned, and the build fails fast if they drift apart

By default, bootstrap uses --go-mod-sync=targeted. That mode updates only the Orchestrion and dd-trace-go tool requirements, resolves the exact packages needed by orchestrion.tool.go, and verifies the result with go list -mod=readonly. It does not run go mod tidy, which keeps large repositories from rewriting unrelated module state. Use --go-mod-sync=tidy when you explicitly want the broad tidy behavior, or --go-mod-sync=off when you only want bootstrap to write managed Bazel/Orchestrion files.

Bootstrap runs module-sync commands with go by default. Pass --go-binary=/path/to/go when the workspace must use a pinned Go SDK, such as the same SDK Bazel uses in a monorepo. The value must be a single executable path named go or go.exe; do not include shell arguments.

By default, dd_topt_go_test also sets DD_SERVICE from the selected sync metadata service name. If you already set DD_SERVICE in the test target's env, the macro preserves your explicit value. If you pass env = select(...), the macro leaves that configurable env unchanged in this release.

The generated .bazelrc block is managed between # BEGIN Datadog Test Optimization Bazelrc and # END Datadog Test Optimization Bazelrc. It forwards sync metadata through common:test-optimization --repo_env=... and adds test:test-optimization --remote_download_minimal plus test:test-optimization --remote_download_regex=.*test[.]outputs.* and test:test-optimization --zip_undeclared_test_outputs. It deliberately does not generate a fixed --build_event_json_file; wrappers should pass a fresh BEP path for each Bazel test invocation. It deliberately does not generate --test_env=DD_GIT_*, --test_env=DD_TEST_OPTIMIZATION_AGENT_URL, or --test_env=DD_TEST_OPTIMIZATION_AGENTLESS_URL.

Then use the generated local wrapper in your package:

load("//tools/build:dd_go_test.bzl", "dd_go_test")

dd_go_test(
    name = "pkg_go_test",
    srcs = ["*_test.go"],
    embed = [":pkg_lib"],
)

Use the manual dd_topt_go_test(..., topt_data = ...) path only when the workspace already has custom sync wiring, mixed-language layout, or multi-service Go setup. Manual Go callsites should set orchestrion_mode = "test_optimization" for standard Go testing; the default general mode preserves broader generic Orchestrion behavior.

Bzlmod + Python companion (dd_topt_py_test)

bazel_dep(name = "datadog-rules-test-optimization-python", version = "1.2.0")
git_override(
    module_name = "datadog-rules-test-optimization-python",
    remote = "https://github.com/DataDog/rules_test_optimization.git",
    commit = "<commit-sha>",
    strip_prefix = "modules/python",
)
load("@python_deps//:requirements.bzl", "requirement")
load("@datadog-rules-test-optimization-python//:topt_py_test.bzl", "dd_topt_py_test")
load("@test_optimization_data//:export.bzl", "topt_data")

dd_topt_py_test(
    name = "pkg_py_test",
    srcs = glob(["test_*.py"]),
    deps = [
        ":pkg_lib",
        requirement("ddtrace"),
        requirement("pytest"),
    ],
    imports = ["example/python/pkg"],
    topt_data = topt_data,
)

dd_topt_py_test now defaults to @rules_python//python:py_test, so you only need py_test_rule when you intentionally override the underlying test rule. When you omit main, the macro uses the bundled pytest entry point, defaults args to the Bazel package path, and adds PYTEST_ADDOPTS=--ddtrace unless you already set it or opt out with --no-ddtrace. This default mode is runner_mode = "managed_pytest" and is the right choice when the repository does not already own a pytest runner.

Large repositories with their own Python wrapper can use runner_mode = "consumer_runner" to keep control of main, imports, and test execution policy:

dd_topt_py_test(
    name = "pkg_py_test",
    py_test_rule = my_repo_pytest_wrapper,
    runner_mode = "consumer_runner",
    module_identifier = "example.python.pkg",
    srcs = glob(["test_*.py"]),
    deps = [
        ":pkg_lib",
        requirement("ddtrace"),
        requirement("pytest"),
    ],
    topt_data = topt_data,
)

In consumer_runner mode, pass a repository-owned py_test_rule wrapper or an explicit main that executes pytest with the ddtrace plugin enabled. The base rules_python py_test without main is rejected because it does not prove pytest is actually running. Prefer module_identifier for payload selection in this mode so the Datadog macro does not need to synthesize Python imports. Replace @python_deps with the repository name generated by your rules_python pip_parse / pip.parse setup.

For Python monorepos, keep the doctor and uploader in a lightweight package instead of forcing the workspace root package to analyze. The convenience macro creates both targets with consistent context wiring:

# tools/test_optimization/BUILD.bazel
load("@datadog-rules-test-optimization//tools/core:test_optimization_targets.bzl", "dd_test_optimization_targets")

dd_test_optimization_targets(
    name = "test_optimization",
    sync_repo_name = "test_optimization_data",
    doctor_name = "dd_test_optimization_doctor",
    uploader_name = "dd_upload_payloads",
    expected_targets = [
        "//python/app:pkg_py_test",
    ],
)

Then run //tools/test_optimization:dd_test_optimization_doctor and //tools/test_optimization:dd_upload_payloads. Root labels remain valid for small repositories; package-local labels are the safer default for large monorepos because they avoid accidental analysis of unrelated root wiring.

The Python companion also publishes a read-only snippet generator:

bazel run @datadog-rules-test-optimization-python//tools/dd_topt_py_bootstrap:dd_topt_py_bootstrap -- \
  --mode=workspace \
  --service my-python-service \
  --runtime-version 3.12 \
  --runtime-module-path example.python.pkg \
  --rto-commit <commit-sha> \
  --private-repo-fetch ssh-git \
  --bazel-command bzl

By default it prints the normal onboarding snippets only. It never adds FETCH_SALT to .bazelrc, test, doctor, or uploader commands. To force a metadata refresh deliberately, ask for the separate refresh snippet:

bazel run @datadog-rules-test-optimization-python//tools/dd_topt_py_bootstrap:dd_topt_py_bootstrap -- \
  --mode=workspace \
  --service my-python-service \
  --runtime-version 3.12 \
  --runtime-module-path example.python.pkg \
  --rto-commit <commit-sha> \
  --print-refresh-snippet

For internal/private repositories, prefer SSH git fetch (ssh://[email protected]/DataDog/rules_test_optimization.git) unless your Bazel environment has authenticated archive access. A 404 from an unauthenticated codeload URL usually means authentication is missing, not that the commit is absent.

Bzlmod + Java companion (dd_topt_java_test)

bazel_dep(name = "datadog-rules-test-optimization-java", version = "1.2.0")
git_override(
    module_name = "datadog-rules-test-optimization-java",
    remote = "https://github.com/DataDog/rules_test_optimization.git",
    commit = "<commit-sha>",
    strip_prefix = "modules/java",
)
load("@datadog-rules-test-optimization-java//:topt_java_test.bzl", "dd_topt_java_test")
load("@test_optimization_data//:export.bzl", "topt_data")

dd_topt_java_test(
    name = "pkg_java_test",
    srcs = ["*Test.java"],
    deps = [":pkg_lib"],
    test_class = "com.example.pkg.SampleTest",
    topt_data = topt_data,
    agent_jar = "@dd_java_agent//file",
)

dd_topt_java_test defaults to @rules_java//java:defs.bzl%java_test, so most callers do not need to pass java_test_rule. Override it only when wrapping a custom test macro (e.g. a junit5 wrapper). The macro requires agent_jar so the dd-java-agent is wired into -javaagent and emits payloads; source the JAR with http_file/maven_install/a local filegroup. The macro injects the Java agent to take care of tracer instrumentation, so callers do not need to add their own -javaagent flag.

On Windows, add build --enable_runfiles to your .bazelrc. The macro injects -javaagent:$(rootpath <agent_jar>), which only resolves when runfiles are materialized as symlinks; Linux and macOS already do this by default.

CI Visibility is on by default — set ci_visibility_enabled = False only if your callsite owns DD_CIVISIBILITY_ENABLED.

If you want the tracer to populate test.source.file, test.source.start, and test.source.end tags, opt into source staging:

dd_topt_java_test(
    name = "pkg_java_test",
    srcs = ["SampleTest.java"],
    stage_sources = True,
    test_class = "com.example.pkg.SampleTest",
    topt_data = topt_data,
    agent_jar = "@dd_java_agent//file",
)

stage_sources adds the target's direct srcs to runfiles so the tracer's repo index can resolve them. Defaults to false because Bazel test runfiles do not include srcs by default and the extra files cost runfiles size.

Bzlmod + NodeJS companion (dd_topt_nodejs_test)

bazel_dep(name = "aspect_rules_js", version = "3.0.0-rc5")
bazel_dep(name = "rules_nodejs", version = "6.7.3")
bazel_dep(name = "datadog-rules-test-optimization-nodejs", version = "1.2.0")
git_override(
    module_name = "datadog-rules-test-optimization-nodejs",
    remote = "https://github.com/DataDog/rules_test_optimization.git",
    commit = "<commit-sha>",
    strip_prefix = "modules/nodejs",
)

node = use_extension("@rules_nodejs//nodejs:extensions.bzl", "node")
node.toolchain(node_version = "22.22.0")
use_repo(node, "nodejs", "nodejs_host", "nodejs_toolchains")
register_toolchains("@nodejs_toolchains//:all")
load("@aspect_rules_js//js:defs.bzl", "js_test")
load("@datadog-rules-test-optimization-nodejs//:topt_nodejs_test.bzl", "dd_topt_nodejs_test")
load("@test_optimization_data//:export.bzl", "topt_data")

dd_topt_nodejs_test(
    name = "pkg_nodejs_test",
    entry_point = "smoke_test.js",
    copy_data_to_bin = False,  # Datadog payload data comes from external repos.
    module_identifier = "apps/nodejs/pkg",
    topt_data = topt_data,
    nodejs_test_rule = js_test,
)

Bzlmod + .NET companion (dd_topt_dotnet_test)

bazel_dep(name = "rules_dotnet", version = "0.21.5")
bazel_dep(name = "datadog-rules-test-optimization-dotnet", version = "1.2.0")
git_override(
    module_name = "datadog-rules-test-optimization-dotnet",
    remote = "https://github.com/DataDog/rules_test_optimization.git",
    commit = "<commit-sha>",
    strip_prefix = "modules/dotnet",
)

dotnet = use_extension("@rules_dotnet//dotnet:extensions.bzl", "dotnet")
dotnet.toolchain(
    name = "dotnet",
    dotnet_version = "8.0.100",
)
use_repo(dotnet, "dotnet_toolchains")
register_toolchains("@dotnet_toolchains//:all")
load("@datadog-rules-test-optimization-dotnet//:topt_dotnet_test.bzl", "dd_topt_dotnet_test")
load("@test_optimization_data//:export.bzl", "topt_data")
load(":dotnet_test_adapter.bzl", "dotnet_csharp_test_adapter")

dd_topt_dotnet_test(
    name = "pkg_dotnet_test",
    srcs = ["smoke_test.cs"],
    target_frameworks = ["net8.0"],
    module_identifier = "Company.Product.Package",
    topt_data = topt_data,
    dotnet_test_rule = dotnet_csharp_test_adapter,
)

dotnet_test_adapter.bzl wraps @rules_dotnet//dotnet:defs.bzl csharp_test and maps the Datadog macro's env to csharp_test(envs = ...).

# dotnet_test_adapter.bzl
load("@rules_dotnet//dotnet:defs.bzl", "csharp_test")

def dotnet_csharp_test_adapter(name, data = None, env = None, **kwargs):
    envs = dict(kwargs.pop("envs", {}))
    if env:
        envs.update(env)

    csharp_test(
        name = name,
        data = [] if data == None else data,
        envs = envs,
        **kwargs
    )

Bzlmod + Ruby companion (dd_topt_ruby_test)

bazel_dep(name = "rules_ruby", version = "0.21.1")
bazel_dep(name = "datadog-rules-test-optimization-ruby", version = "1.2.0")
git_override(
    module_name = "datadog-rules-test-optimization-ruby",
    remote = "https://github.com/DataDog/rules_test_optimization.git",
    commit = "<commit-sha>",
    strip_prefix = "modules/ruby",
)

ruby = use_extension("@rules_ruby//ruby:extensions.bzl", "ruby")
ruby.toolchain(
    name = "ruby",
    version = "3.3.9",
)
use_repo(ruby, "ruby", "ruby_toolchains")
register_toolchains("@ruby_toolchains//:all")
load("@rules_ruby//ruby:defs.bzl", "rb_test")
load("@datadog-rules-test-optimization-ruby//:topt_ruby_test.bzl", "dd_topt_ruby_test")
load("@test_optimization_data//:export.bzl", "topt_data")

dd_topt_ruby_test(
    name = "pkg_ruby_test",
    srcs = ["smoke_test.rb"],
    main = "smoke_test.rb",
    module_identifier = "apps/ruby/pkg",
    topt_data = topt_data,
    ruby_test_rule = rb_test,
)

Optional: workspace-local wrappers (less boilerplate)

If your workspace always uses the same synced repo (@test_optimization_data) and the same underlying test rule symbols, create thin local wrappers so package BUILD files do not repeat topt_data and *_test_rule.

Keep plain wrappers and Test Optimization wrappers separate. A wrapper that is loaded by non-instrumented BUILD files must not load @test_optimization_data//:export.bzl, because the load itself consumes the sync repository. Put the @test_optimization_data load only in the wrapper used by instrumented targets.

Single-service wrapper pattern for a simple Go workspace:

# tools/build/dd_go_test.bzl
load("@datadog-rules-test-optimization-go//:topt_go_test.bzl", "dd_topt_go_test")
load("@test_optimization_data//:export.bzl", "topt_data")

def dd_go_test(name, **kwargs):
    dd_topt_go_test(
        name = name,
        topt_data = topt_data,
        orchestrion_mode = "test_optimization",
        orchestrion_pin_files = [
            "//:go.mod",
            "//:go.sum",
            "//:orchestrion.tool.go",
            "//:orchestrion.yml",
        ],
        **kwargs
    )
# package BUILD.bazel
load("//tools/build:dd_go_test.bzl", "dd_go_test")

dd_go_test(
    name = "pkg_go_test",
    srcs = ["*_test.go"],
    embed = [":pkg_lib"],
)

You can apply the same pattern to all companion macros:

# tools/build/dd_py_test.bzl
load("@datadog-rules-test-optimization-python//:topt_py_test.bzl", "dd_topt_py_test")
load("@test_optimization_data//:export.bzl", "topt_data")

def dd_py_test(name, **kwargs):
    dd_topt_py_test(
        name = name,
        topt_data = topt_data,
        **kwargs
    )
# tools/build/dd_java_test.bzl
load("@datadog-rules-test-optimization-java//:topt_java_test.bzl", "dd_topt_java_test")
load("@test_optimization_data//:export.bzl", "topt_data")

def dd_java_test(name, **kwargs):
    dd_topt_java_test(
        name = name,
        topt_data = topt_data,
        agent_jar = "@dd_java_agent//file",
        **kwargs
    )
# tools/build/dd_nodejs_test.bzl
load("@aspect_rules_js//js:defs.bzl", "js_test")
load("@datadog-rules-test-optimization-nodejs//:topt_nodejs_test.bzl", "dd_topt_nodejs_test")
load("@test_optimization_data//:export.bzl", "topt_data")

def dd_nodejs_test(name, **kwargs):
    dd_topt_nodejs_test(
        name = name,
        topt_data = topt_data,
        nodejs_test_rule = js_test,
        **kwargs
    )
# tools/build/dd_dotnet_test.bzl
load("@datadog-rules-test-optimization-dotnet//:topt_dotnet_test.bzl", "dd_topt_dotnet_test")
load("@test_optimization_data//:export.bzl", "topt_data")
load("//:dotnet_test_adapter.bzl", "dotnet_csharp_test_adapter")

def dd_dotnet_test(name, **kwargs):
    dd_topt_dotnet_test(
        name = name,
        topt_data = topt_data,
        dotnet_test_rule = dotnet_csharp_test_adapter,
        **kwargs
    )
# tools/build/dd_ruby_test.bzl
load("@rules_ruby//ruby:defs.bzl", "rb_test")
load("@datadog-rules-test-optimization-ruby//:topt_ruby_test.bzl", "dd_topt_ruby_test")
load("@test_optimization_data//:export.bzl", "topt_data")

def dd_ruby_test(name, **kwargs):
    dd_topt_ruby_test(
        name = name,
        topt_data = topt_data,
        ruby_test_rule = rb_test,
        **kwargs
    )

For multi-service repos, either:

  1. Keep topt_service as a wrapper argument and pass topt_data_by_service
  2. Create one wrapper per service with topt_service pre-set

Bzlmod + multi-service monorepo

Use the Go extension multi-service form only for multi-service Go setups. It is not a generic mixed-runtime extension; every service configured through it is materialized as runtime_name = "go". For Python/Java/NodeJS/.NET/Ruby multi-service onboarding, use docs/Language_Onboarding.md.

go_topt = use_extension(
    "@datadog-rules-test-optimization-go//:topt_go_extension.bzl",
    "test_optimization_go_extension",
)

go_topt.test_optimization_go(
    name = "test_optimization_data",
    services = ["go-service-a", "go-service-b"],
    runtime_version = "1.25.0",
)

use_repo(
    go_topt,
    "test_optimization_data",
    "test_optimization_data_go_service_a",
    "test_optimization_data_go_service_b",
)

For macro consumers, load topt_data_by_service from @test_optimization_data//:export.bzl and select by topt_service.

Bzlmod + mixed-language monorepo

For mixed-runtime repos, keep one sync repo per runtime or runtime/service slice and use the matching companion on top of each exported dataset. Do not reuse one shared sync repo across different runtimes.

topt_go = use_extension(
    "@datadog-rules-test-optimization//tools/core:test_optimization_sync.bzl",
    "test_optimization_sync_extension",
)
topt_go.test_optimization_sync(
    name = "test_optimization_data_go",
    service = "go-service",
    runtime_name = "go",
    runtime_version = "1.25.0",
)

topt_ruby = use_extension(
    "@datadog-rules-test-optimization//tools/core:test_optimization_sync.bzl",
    "test_optimization_sync_extension",
)
topt_ruby.test_optimization_sync(
    name = "test_optimization_data_ruby",
    service = "ruby-service",
    runtime_name = "ruby",
    runtime_version = "3.3.9",
)

use_repo(topt_go, "test_optimization_data_go")
use_repo(topt_ruby, "test_optimization_data_ruby")

Then load the matching export in each runtime-specific wrapper or BUILD file:

  • Go targets use @test_optimization_data_go//:export.bzl
  • Ruby targets use @test_optimization_data_ruby//:export.bzl

Root doctor/uploader wiring in a mixed-runtime workspace must bundle every matching context target so validation and upload enrichment use the correct context.json per payload:

load("@datadog-rules-test-optimization//tools/core:test_optimization_doctor.bzl", "dd_test_optimization_doctor")
load("@datadog-rules-test-optimization//tools/core:test_optimization_uploader.bzl", "dd_payload_uploader")

dd_test_optimization_doctor(
    name = "dd_test_optimization_doctor",
    data = [
        "@test_optimization_data_go//:test_optimization_context",
        "@test_optimization_data_ruby//:test_optimization_context",
    ],
)

dd_payload_uploader(
    name = "dd_upload_payloads",
    data = [
        "@test_optimization_data_go//:test_optimization_context",
        "@test_optimization_data_ruby//:test_optimization_context",
    ],
)

WORKSPACE mode

WORKSPACE remains supported for v1. Minimal setup:

# WORKSPACE
load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")

git_repository(
    name = "datadog-rules-test-optimization",
    remote = "https://github.com/DataDog/rules_test_optimization.git",
    commit = "<commit-sha>",
)

load("@datadog-rules-test-optimization//tools/core:test_optimization_sync.bzl", "test_optimization_sync")

test_optimization_sync(
    name = "test_optimization_data",
    service = "my-service",  # recommended; otherwise falls back to DD_SERVICE or unnamed-service
)

For Go in WORKSPACE mode, keep the core and Go companion as separate external repositories and load dd_topt_go_test from @datadog-rules-test-optimization-go//:topt_go_test.bzl. Prefer the public WORKSPACE helper so the Go companion and Orchestrion-enabled rules_go fork use the same commit, repo mapping, and upstream support line:

load("@datadog-rules-test-optimization//tools/go:workspace_repositories.bzl", "datadog_go_test_optimization_workspace_repositories")

datadog_go_test_optimization_workspace_repositories(
    rto_commit = "<commit-sha>",
    rules_go_repo_name = "io_bazel_rules_go",
    rules_go_upstream = "v0_60_0",
    rules_go_variant = "base",
)

Use rules_go_variant = "base". When multiple upstream versions are supported, use rules_go_upstream to select the upstream support line; omit it to keep the repository default. The base variant is a complete rules_go tree and does not require patches, patch_tool, or a consumer-owned patch directory. The public WORKSPACE helper also expects the default tool-repo name rules_go_orchestrion_tool, so consumers should not rename that repository. When Go tests live below the module root, pass the module-root pin files through orchestrion_pin_files (for example ["//:go.mod", "//:orchestrion.tool.go"]) or inject them from a repo-local wrapper. For large monorepos where root-level tool imports would churn or invalidate the main Go module, keep Orchestrion tool wiring in Bazel and use a repo-local wrapper with package-local pin files or orchestrion_pin_files = []; do not add a root orchestrion.tool.go just to satisfy the wrapper pattern.

For Python in WORKSPACE mode, declare rules_python and the core repository first, then use the public Python helper to declare only the Python companion:

load("@datadog-rules-test-optimization//tools/python:workspace_repositories.bzl", "datadog_python_test_optimization_workspace_repositories")

datadog_python_test_optimization_workspace_repositories(
    rto_commit = "<commit-sha>",
    rules_python_repo_name = "rules_python",
)

The Python helper deliberately does not declare Python toolchains, pip_parse, pytest, ddtrace, or lockfiles. Keep those dependencies in the consumer repository's existing rules_python setup, then load dd_topt_py_test from @datadog-rules-test-optimization-python//:topt_py_test.bzl.

For Java in WORKSPACE mode, declare rules_java and the core repository first, then use the public Java helper to declare only the Java companion:

load("@datadog-rules-test-optimization//tools/java:workspace_repositories.bzl", "datadog_java_test_optimization_workspace_repositories")

datadog_java_test_optimization_workspace_repositories(
    rto_commit = "<commit-sha>",
    rules_java_repo_name = "rules_java",
)

The Java helper deliberately does not declare Java toolchains, test framework dependencies, or the dd-java-agent artifact. Keep those dependencies in the consumer repository's existing Java setup, then load dd_topt_java_test from @datadog-rules-test-optimization-java//:topt_java_test.bzl.

Use docs/Installation_Reference.md for mirrored http_archive, Go toolchain setup, uploader wiring, and full WORKSPACE details. You can also print a starting WORKSPACE snippet without modifying files:

bazel run @datadog-rules-test-optimization-go//:dd_topt_go_bootstrap -- \
  --workspace-mode \
  --print-workspace-snippet \
  --rto-commit <commit-sha> \
  --service my-service \
  --runtime-version 1.25.0 \
  --rules-go-upstream v0_60_0 \
  --rules-go-variant base

For larger WORKSPACE monorepos, let bootstrap write the generic local files and keep only the final WORKSPACE placement as a manual review step:

bazel run @datadog-rules-test-optimization-go//:dd_topt_go_bootstrap -- \
  --workspace-mode \
  --service my-service \
  --runtime-version 1.25.0 \
  --sync-repo-name test_optimization_data \
  --rules-go-upstream v0_60_0 \
  --rules-go-variant base \
  --rules-go-repo-name io_bazel_rules_go \
  --write-bazelrc \
  --write-root-targets \
  --write-orchestrion-files \
  --write-wrapper-template

WORKSPACE mode does not edit WORKSPACE. It writes only Datadog-managed local blocks/files such as .bazelrc, root doctor/uploader targets, orchestrion.tool.go, orchestrion.yml, and an optional repo-local wrapper template. By default it also avoids running Go module commands; pass an explicit --go-mod-sync=targeted when you want bootstrap to repair the local Orchestrion tool graph.

If your WORKSPACE repo also checks in Gazelle-style go_repository(...) declarations, ask bootstrap to validate those pins instead of discovering the drift later during Bazel analysis:

bazel run @datadog-rules-test-optimization-go//:dd_topt_go_bootstrap -- \
  --workspace-mode \
  --write-orchestrion-files \
  --go-mod-sync=targeted \
  --check-go-repositories \
  --go-repositories-file repositories.bzl \
  --go-repositories-refresh-command './tools/update-go-repositories.sh'

Bootstrap does not rewrite repositories.bzl itself. It checks only the Orchestrion-related modules it owns and runs the repository-owned refresh command only after targeted Go module sync succeeds. Use --print-go-repository-updates to print the expected versions without hiding the fact that the repo still owns the actual refresh command.

Bootstrap can also generate an operator-owned validation script for large repositories. The script repeats the RFC flow without hiding Bazel behavior: sync -> controls -> instrumented tests -> doctor -> dry-run uploader -> optional upload. Upload is disabled unless the operator passes --upload. Set DD_TEST_OPTIMIZATION_REPORT_DIR to keep its doctor-report.json, uploader-dry-run-report.json, and optional uploader-upload-report.json outside the script's temporary directory.

bazel run @datadog-rules-test-optimization-go//:dd_topt_go_bootstrap -- \
  --workspace-mode \
  --print-validation-script \
  --bazel-command bazel \
  --bazel-config test-optimization \
  --sync-repo-name test_optimization_data \
  --control-target //pkg/plain:go_default_test \
  --expected-target //pkg:go_default_test

Use --write-validation-script to create tools/test_optimization/validate_go_pilot.sh. For large monorepos, add --large-monorepo --min-free-disk-gb=25 --shutdown-bazel-on-exit so the script prints disk warnings before each heavy phase and shuts down Bazel when it exits. The script never deletes caches and never passes DD_GIT_* through --test_env.

Other languages

Use the core-only path above, or mirror the companion pattern used by Go/Python/Java/NodeJS/.NET/Ruby, then wire your language test rule/macro so it:

  1. Includes @test_optimization_data//:test_optimization_files in data
  2. Sets DD_TEST_OPTIMIZATION_MANIFEST_FILE to the manifest runfile path
    • At runtime, resolve the payload root as dirname(DD_TEST_OPTIMIZATION_MANIFEST_FILE).
  3. Sets DD_TEST_OPTIMIZATION_PAYLOADS_IN_FILES = "true"
  4. Writes payloads under TEST_UNDECLARED_OUTPUTS_DIR/payloads/{tests,coverage}
  5. Adds @test_optimization_data//:test_optimization_context to uploader data

For a generic wrapper pattern, see Other languages (without companion macro).

Requirements

Command convention

  • Consumer repository commands in this README use bazel
  • Repository-maintainer workflows in this repo use ./bazelw (see docs/Maintainers.md)
  • This repository intentionally does not define a root .bazelrc. Example workspaces keep local .bazelrc files; CI and maintainer docs provide canonical flags.

Compatibility snapshot

Component Recommended baseline Notes
Bazel 8.5.1 Repository baseline (.bazelversion) and primary CI lanes
rules_go (Go users) 0.60.0 / rules_go_upstream = "v0_60_0" README examples use the default support line; importpath inference requires 0.51.0+
Go toolchain (example) 1.25.0 Consumer repositories may use another supported version
Module versions 1.2.0 metadata BCR publication is pending; use commit pin/override install paths
  • Bazel 8.5.1 (repo baseline) - Matches .bazelversion and primary CI lanes
  • WORKSPACE compatibility lane - CI intentionally validates --noenable_bzlmod --enable_workspace on Bazel 8.4.1 during migration away from WORKSPACE mode in Bazel 9+
  • Bazel 5.0+ minimum capability - Earliest Bazel line with required TEST_UNDECLARED_OUTPUTS_DIR payload support
  • Tracer/runtime with DD Test Optimization file-mode support - Must honor DD_TEST_OPTIMIZATION_MANIFEST_FILE and DD_TEST_OPTIMIZATION_PAYLOADS_IN_FILES
  • rules_go v0.51.0+ (for Go importpath inference) - This repository reads GoInfo/GoArchive providers when selecting per-module payloads
  • DD_SITE format - Accepts bare host, app/api-prefixed host, or full URL; leading/trailing ASCII whitespace is trimmed, then normalized to https://api.<site>
  • Uploader tooling (per platform) - Required for bazel run //:dd_upload_payloads
    • Linux: bash, curl, find, stat (GNU), awk, and one of md5sum or shasum
    • macOS: bash (3.2+), curl, find, stat (BSD), awk, and one of md5 or shasum
    • Windows: powershell.exe (Windows PowerShell 5.1+ or PowerShell 7+); the uploader uses .NET HttpClient and is intentionally PowerShell-only (no Git Bash dependency)

Optional tooling:

  • jq (Linux/macOS) - Used to enrich test payloads with context.json. If missing, uploads proceed without enrichment.
  • python3 - Used for uploader payload schema validation and Unix telemetry metadata extraction. If missing, schema validation is skipped and telemetry files fail individually with a warning.

Contract gate checklist

Before rollout in a consumer repository, confirm the tracer/runtime implementation:

  • resolves DD_TEST_OPTIMIZATION_MANIFEST_FILE through Bazel runfiles
  • enables file-mode output when DD_TEST_OPTIMIZATION_PAYLOADS_IN_FILES=true
  • writes JSON payloads under TEST_UNDECLARED_OUTPUTS_DIR/payloads/tests, TEST_UNDECLARED_OUTPUTS_DIR/payloads/coverage, and TEST_UNDECLARED_OUTPUTS_DIR/payloads/telemetry

The extension performs these HTTP POST transactions (via host HTTP tooling: curl on Unix/macOS, PowerShell on Windows):

  • Settings: always executed. Parses feature flags from response.
  • Known Tests: executed only when known_tests_enabled: true in Settings.
  • Test Management Tests: executed only when test_management.enabled: true in Settings.
  • Flaky Tests: executed only when flaky_test_retries_enabled: true in Settings. The raw backend response is persisted under cache/http/flaky_tests.json and then split into per-module flaky_tests.json files.

All outputs are written under a configurable directory (default: .testoptimization) and are grouped under a single filegroup target. The exact manifest path is exported via topt_data["manifest_path"].

What gets created

Given an external repository name <repo_name> created by the extension, the generated BUILD inside the external repo contains:

  • A core filegroup target named test_optimization_files which includes cache/http/settings.json and manifest.txt
  • Files (always created; some may be minimal stubs if the corresponding feature is disabled):
    • cache/http/settings.json (Settings API response)
    • manifest.txt (Payload manifest; version marker for change tracking, currently version=1)
    • cache/http/known_tests.json (Known Tests API response or minimal stub)
    • cache/http/test_management.json (Test Management Tests API response or minimal stub)
    • cache/http/flaky_tests.json (Flaky Tests API raw response or minimal stub {"data": []})
    • context.json (Non-secret CI/Git/OS/runtime tags)
    • Per-module Known Tests/Test Management/Flaky Tests (via filegroups): each module has a target exposing canonical runfiles under <manifest_dir>/cache/http/ with known_tests.json, test_management.json, and flaky_tests.json, scoped to that module. Physical files are stored under <out_dir>/module_<sanitized>/ (default <out_dir> is .testoptimization).

Reference settings with a single label:

@<repo_name>//:test_optimization_files

Per-module files and labels

When Known Tests are enabled, the combined response data.attributes.tests is a map keyed by module name. For convenience and performance, the sync rule automatically splits this response into per-module files and creates one public target per module. The same splitting is performed for Test Management tests (test_management.json), keyed by module under data.attributes.modules. Flaky Tests (flaky_tests.json) are also split per module by grouping the raw data array entries by entry.attributes.configurations.test.bundle:

  • Each module target exposes canonical runfiles (stable names, regardless of out_dir):
    • <manifest_dir>/cache/http/known_tests.json (module-scoped; same shape as combined)
    • <manifest_dir>/cache/http/test_management.json (module-scoped; same shape as combined)
    • <manifest_dir>/cache/http/flaky_tests.json (module-scoped; raw envelope with filtered data array)
  • Each module also becomes a public target: :module_<sanitized_module> that includes:
    • <manifest_dir>/cache/http/settings.json
    • <manifest_dir>/manifest.txt
    • <manifest_dir>/cache/http/known_tests.json (always present; stub when empty)
    • <manifest_dir>/cache/http/test_management.json (always present; stub when empty)
    • <manifest_dir>/cache/http/flaky_tests.json (always present; stub {"data": []} when empty)
  • These per-module files are not bundled into :test_optimization_files

Sanitization rules for <sanitized_module> (file paths and target labels):

  • Lowercase input
  • Characters outside [a-z0-9_] are replaced with _
  • Consecutive underscores are collapsed, then leading/trailing underscores are trimmed
  • If collisions occur after sanitization, numeric suffixes like _2, _3 are appended deterministically

Labels are computed from the union of module names across known tests and test management so a module_<sanitized> target always refers to a single module (avoids cross-feature collisions).

Example usage:

# Consume only the module "pkg/foo" tests metadata
filegroup(
    name = "dd_known_tests_pkg_foo",
    srcs = [
        "@test_optimization_data//:module_pkg_foo",
    ],
)

# If you need file paths at test time, use rlocationpaths on the selector target
# provided by the dd_topt_go_test macro (see [`modules/go/topt_go_test.bzl`](modules/go/topt_go_test.bzl)).

Advanced installation and setup

The quickstarts above cover the most common onboarding paths.

For complete setup matrices and advanced options, use:

Uploading test, coverage, and telemetry payloads

The uploader is a normal Bazel rule (not a test) that runs via bazel run after your tests complete. It discovers all test payloads written to TEST_UNDECLARED_OUTPUTS_DIR (Bazel's built-in directory for undeclared test outputs) and uploads them to Datadog.

How it works

  1. Tests write payloads to $TEST_UNDECLARED_OUTPUTS_DIR/payloads/tests/*.json, $TEST_UNDECLARED_OUTPUTS_DIR/payloads/coverage/*.json, and $TEST_UNDECLARED_OUTPUTS_DIR/payloads/telemetry/*.json
  2. Bazel automatically collects these to bazel-testlogs/<package>/<target>/test.outputs/
  3. After tests complete, run the doctor via bazel run to validate local outputs before upload
  4. Then run the uploader via bazel run
  5. The uploader discovers all test.outputs/ directories, waits for quiescence, uploads, and deletes files

Telemetry-specific notes:

  • Telemetry files must contain one raw top-level tracer telemetry request body per file.
  • Telemetry uploads are reconstructed from the raw body plus the uploader mode.
  • Telemetry does not use test-payload enrichment (context.json, CODEOWNERS, or schema validation).
  • DD_TEST_OPTIMIZATION_FILTER_PREFIX=1 still filters only test and coverage filenames; telemetry files remain eligible regardless of filename prefix.

Basic usage

# RECOMMENDED: Run tests with BEP, validate payloads, then upload payloads.
tools/test_optimization/run_test_optimization_ci.sh \
  --report-dir .topt/reports \
  --support-bundle .topt/reports/dd-test-optimization-support.zip \
  //...

# Add --upload only when the real upload should run after doctor and dry-run pass.
DD_API_KEY="$DD_API_KEY" DD_SITE="$DD_SITE" \
  tools/test_optimization/run_test_optimization_ci.sh \
    --report-dir .topt/reports \
    --support-bundle .topt/reports/dd-test-optimization-support.zip \
    --upload \
    //...
# RECOMMENDED: Run tests with BEP, validate payloads, then upload payloads.
.\tools\test_optimization\run_test_optimization_ci.ps1 `
  -ReportDir .topt\reports `
  -SupportBundle .topt\reports\dd-test-optimization-support.zip `
  //...

# Add -Upload only when the real upload should run after doctor and dry-run pass.
$env:DD_API_KEY = "<your-api-key>"
$env:DD_SITE = "datadoghq.com"
.\tools\test_optimization\run_test_optimization_ci.ps1 `
  -ReportDir .topt\reports `
  -SupportBundle .topt\reports\dd-test-optimization-support.zip `
  -Upload `
  //...

IMPORTANT: Always preserve the test exit code. Upload failed-test payloads when doctor and dry-run enrichment pass, but do not run the real upload after a doctor or dry-run failure. Using plain ; causes CI to report success even when tests fail.

Important runtime requirements

  1. Use bazel run (not bazel test) for uploader execution.
  2. Use a single uploader target per workspace (do not run concurrent uploaders).
  3. Tests must run locally or use --remote_download_minimal --remote_download_regex=.*test[.]outputs.* --zip_undeclared_test_outputs with BEP artifact staging. Enable remote BEP artifact resolution when BEP points at remote-only carriers; HTTP/HTTPS outputs.zip carriers can be staged natively, while bytestream/CAS/custom-auth providers still need a downloader executable.
  4. Run uploader on the same machine/workspace where tests executed.
  5. DD_TEST_OPTIMIZATION_CONTEXT_JSON is a legacy explicit override for advanced workflows that already resolved one specific context.json path. Do not use it as the normal mixed-runtime wiring path; mixed-runtime workspaces should pass every relevant :test_optimization_context target in uploader data and let the uploader select per payload.

Avoid cached-output uploads

When Bazel downloads test.outputs/ from a local, disk, remote, or sandbox cache, those payloads can describe a previous test execution. Keep test caching enabled, but write a BEP JSON file from the same bazel test invocation and configure doctor/uploader with matching wrapper or CLI arguments so only fresh TestResult outputs are used:

tools/test_optimization/run_test_optimization_ci.sh //...

The wrapper creates a temporary BEP file for each Bazel test invocation and passes those files to doctor/uploader as repeatable --bep-json flags. The default freshness source/mode is auto: when BEP is explicitly configured the uploader uses it, otherwise it can use an explicitly configured legacy execution-log fallback. Artifact discovery defaults to local bazel-testlogs unless the wrapper or CLI sets --artifact-source=bep. In CI, uploads fail closed unless an explicit freshness source is available; outside CI the uploader preserves historical behavior with a warning. The uploader does not auto-discover default BEP paths because stale BEP files can authorize stale local outputs. To opt out explicitly, set DD_TEST_OPTIMIZATION_FRESHNESS_MODE=disabled or pass --allow-cached-payload-uploads after the uploader target's -- separator. On Unix, BEP freshness filtering requires jq; if a required BEP file is missing or malformed, the uploader exits with a configuration error instead of guessing.

BEP artifact resolution

BEP freshness and BEP artifact resolution are separate controls. Freshness prevents uploads from cached TestResult outputs. Artifact resolution lets the doctor and uploader materialize fresh test.outputs referenced by BEP when those files are not already present under bazel-testlogs/.

The recommended CI baseline keeps stable Bazel behavior in .bazelrc and lets the wrapper create per-run BEP and artifact-staging paths:

test:test-optimization --remote_download_minimal
test:test-optimization --remote_download_regex=.*test[.]outputs.*
test:test-optimization --zip_undeclared_test_outputs
tools/test_optimization/run_test_optimization_ci.sh //...

If the test run uses --zip_undeclared_test_outputs, test.outputs/ contains outputs.zip instead of loose payloads/tests/*.json files. --artifact-source=bep makes doctor/uploader stage the BEP-referenced local zip and extract payloads before discovery. Manually expanded, the wrapper flow looks like this:

bep_json="$(mktemp "${TMPDIR:-/tmp}/dd-topt-bep.XXXXXX.json")"
artifact_staging_dir="$(mktemp -d "${TMPDIR:-/tmp}/dd-topt-artifacts.XXXXXX")"
bazel test --config=test-optimization --build_event_json_file="$bep_json" //...

bazel run //:dd_test_optimization_doctor -- \
  --bep-json="$bep_json" \
  --freshness-source=bep \
  --freshness-mode=required \
  --artifact-source=bep \
  --artifact-staging-dir="$artifact_staging_dir"

bazel run //:dd_upload_payloads -- \
  --bep-json="$bep_json" \
  --freshness-source=bep \
  --freshness-mode=required \
  --artifact-source=bep \
  --artifact-staging-dir="$artifact_staging_dir"

For CI diagnostics, prefer one report directory per CI job:

tools/test_optimization/run_test_optimization_ci.sh \
  --report-dir .topt/reports \
  --support-bundle .topt/reports/dd-test-optimization-support.zip \
  //...

The wrapper writes .topt/reports/doctor-report.json and .topt/reports/uploader-dry-run-report.json. When --upload is also used, it writes the real upload result to .topt/reports/uploader-upload-report.json instead of overwriting the dry-run report. Reports include a result block with status, reason_code, reason, next_steps, and the exact counts needed to answer why payloads were or were not uploaded: expected/seen targets, BEP fresh/cached/remote-only outputs, artifact staging, discovered payloads, processed/skipped/failed payloads, and upload attempts.

For support tickets, prefer .topt/reports/dd-test-optimization-support.zip. For the simplest customer ask, run the doctor with --support-bundle=<path> and attach the resulting doctor-only bundle. In CI, prefer the wrapper --support-bundle=<path> because it also includes uploader dry-run and optional upload reports. Bundles are redacted and bounded by default and include summary.md, selected BEP summaries, effective flags, runtime metadata, and a redaction manifest. They do not include raw payloads, raw CI logs, raw environment variables, or raw BEP files. The support runbook in docs/Troubleshooting.md describes which bundle to request and which files to inspect first.

Manual flows can still pass --report-json=<path> to the doctor or uploader, or set DD_TEST_OPTIMIZATION_DOCTOR_REPORT_JSON / DD_TEST_OPTIMIZATION_UPLOADER_REPORT_JSON. If a repository cannot use the doctor or wrapper support-bundle entrypoints but has the helper script, create the redacted zip manually:

python3 tools/test_optimization/create_support_bundle.py \
  --report-dir .topt/reports \
  --report-json .topt/reports/doctor-report.json \
  --report-json .topt/reports/uploader-dry-run-report.json \
  --output .topt/reports/dd-test-optimization-support.zip \
  --workspace-root "$PWD" \
  --output-base "$(bazel info output_base)"

When only Markdown is possible, turn archived JSON reports into a short customer-facing fallback summary:

python3 tools/test_optimization/render_report_summary.py \
  .topt/reports/doctor-report.json \
  .topt/reports/uploader-dry-run-report.json \
  --output .topt/reports/upload-diagnostics.md

If your CI cannot materialize test.outputs/ or outputs.zip with the selective remote download flags, enable remote BEP artifact staging. Plain HTTP/HTTPS outputs.zip BEP carriers do not need a downloader:

bazel run //:dd_upload_payloads -- \
  --bep-json="$bep_json" \
  --freshness-source=bep \
  --freshness-mode=required \
  --artifact-source=bep \
  --remote-artifacts=download \
  --artifact-staging-dir="$artifact_staging_dir"

Customer-facing mode summary:

Mode Use when Key flags
Local discovery bazel-testlogs/**/test.outputs/payloads/... exists locally No artifact flags; use BEP freshness in CI
Recommended CI Remote cache/RBE can download test outputs locally .bazelrc test config has --remote_download_minimal --remote_download_regex=.*test[.]outputs.* --zip_undeclared_test_outputs; wrapper passes --build_event_json_file=<temp> to each bazel test, then passes repeatable --bep-json=<temp> --freshness-source=bep --freshness-mode=required --artifact-source=bep --artifact-staging-dir=<temp-dir> to doctor/uploader
CI without zipped outputs Tests leave loose payload files under local test.outputs Same BEP freshness env; DD_TEST_OPTIMIZATION_ARTIFACT_SOURCE=bep is valid but not required for zip extraction
HTTP/HTTPS BEP staging BEP points at an http:// or https:// outputs.zip carrier and no custom auth is needed `--artifact-source=bep --remote-artifacts=download
Custom remote/CAS staging BEP points at bytestream://, CAS, internal artifact APIs, or HTTP endpoints requiring custom auth `--artifact-source=bep --remote-artifacts=download
Auto staging Mixed local/staged migration path `--artifact-source=auto --remote-artifacts=download

Native HTTP/HTTPS staging is unauthenticated: it does not add custom headers, read credentials, use cookies, or consult netrc. It retries transient network failures, truncated responses, HTTP 408, HTTP 429, and HTTP 5xx responses three times with exponential backoff before applying the selected download/required failure behavior. HTTP query strings, fragments, usernames, and passwords are redacted from doctor/uploader warnings and JSON reports.

Custom remote/CAS artifact download requires a downloader executable configured by the consumer environment. DD_TEST_OPTIMIZATION_BEP_ARTIFACT_DOWNLOADER must point to one executable file; if the provider needs an interpreter, fixed arguments, custom auth headers, mTLS, signed header refresh, or a CAS client, wrap them in a script and configure the wrapper path. The downloader must write an outputs.zip archive to the requested --output path before --bep-artifact-downloader-timeout-sec expires. The public rule only defines the downloader contract; it does not ship credentials or a Datadog-internal CAS client.

Artifact staging requires Python at uploader runtime. Bash resolves DD_TEST_OPTIMIZATION_PYTHON, then PYTHON, then python3, then python; PowerShell uses the same discovery order. Existing local-only uploader flows remain usable without Python except for support bundle generation and the pre-existing optional schema and telemetry helpers. Bash BEP freshness parsing still requires jq whenever BEP freshness validation is enabled in the Bash uploader path.

Enrichment dry-run

Payload files written by tests are intentionally raw Bazel outputs. They do not need to contain every Git, Bazel, CODEOWNERS, or runtime tag before upload. The uploader enriches test payloads at bazel run time using context.json plus bazel_target_metadata.json.

Use dry-run mode when you want to prove that the final outbound body would be properly enriched without uploading data or deleting local payload files. When invoking the uploader manually, use the same $bep_json path generated by the matching bazel test --build_event_json_file=... invocation:

bazel run --config=test-optimization //:dd_upload_payloads -- \
  --bep-json="$bep_json" \
  --freshness-source=bep \
  --freshness-mode=required \
  --dry-run \
  --validate-enrichment

By default this validates that the enriched test payload has git.repository_url, git.commit.sha, bazel.target, and bazel.package. Add repeatable --expected-enriched-tag=<tag-name> arguments when a repository needs extra tags validated before upload. Go rollouts that need to prove per-module selection can add --expected-enriched-tag=bazel.go.payload_selection.

Full uploader reference

For complete uploader details, use docs/Uploader_Reference.md, including:

  • uploader target attributes and optional environment variables
  • agentless vs EVP credential modes and endpoint behavior
  • retry/reliability semantics and exit codes
  • metadata enrichment (context.json, CODEOWNERS) and schema validation

Convenience macro: dd_topt_go_test

The dd_topt_go_test macro creates a go_test target with Datadog Test Optimization data/env wiring included, and always runs through an internal Orchestrion-enabled wrapper target.

By default, it sets rundir to the current Bazel package when not explicitly provided. If you enable stage_sources = True, it instead defaults rundir to . unless you already set rundir yourself.

For a fresh single-service Go workspace, prefer the guided bootstrap flow above. It generates the local dd_go_test wrapper and the uploader target for you.

Use the raw macro directly only when you need the lower-level API.

Before using it directly, configure the Go companion extension in MODULE.bazel:

bazel_dep(name = "datadog-rules-test-optimization-go", version = "1.2.0")
bazel_dep(name = "rules_go", version = "0.60.0")

go_topt = use_extension(
    "@datadog-rules-test-optimization-go//:topt_go_extension.bzl",
    "test_optimization_go_extension",
)

go_topt.test_optimization_go(
    name = "test_optimization_data",
    service = "go-service",
    runtime_version = "1.25.0",
    module_path = "github.com/example/service",
)

use_repo(go_topt, "test_optimization_data")

module_path should match the Go module path from go.mod. The sync rule still honors GO_MODULE_PATH first for CI overrides, but the explicit attr is the recommended default because it avoids repo-local --repo_env glue.

Then run the Datadog bootstrap helper once so Orchestrion is pinned into the workspace Go module. Repository/bootstrap resolution may use network access. After that, Orchestrion build actions consume a declared offline Go module download cache staged in @rules_go_orchestrion_tool; they do not depend on a warmed host Go module cache. Test payloads still use the Bazel file-output contract: the tracer writes JSON files under TEST_UNDECLARED_OUTPUTS_DIR, and the uploader enriches those JSON files with repository and Bazel metadata. Pass --dd-trace-go-version <query> if you want a non-default tracer version; otherwise the default is v2.9.0.

bazel run @datadog-rules-test-optimization-go//:dd_topt_go_bootstrap

The bootstrap's default module-sync mode is targeted, which avoids a broad go mod tidy. Use --go-mod-sync=tidy only when you want bootstrap to tidy the whole module, and use --go-mod-sync=off when another repository-owned command will update go.mod and go.sum. Use --go-binary=/path/to/go if the module must be synced with a specific Go SDK. The path must point to a go or go.exe executable and must not include arguments.

For WORKSPACE repos with checked-in go_repository(...) declarations, add --check-go-repositories after targeted sync. This catches stale repositories.bzl pins for github.com/DataDog/orchestrion and the three dd-trace-go modules before Bazel tries to build with mismatched versions.

If you wire Orchestrion manually instead of using bootstrap, you can also set the tracer versions directly in MODULE.bazel.

Shared-version form:

orchestrion = use_extension("@rules_go//go:extensions.bzl", "orchestrion")
orchestrion.from_source(
    version = "v1.9.0",
    dd_trace_go_version = "v2.9.0",
)
use_repo(orchestrion, "rules_go_orchestrion_tool")

Per-module form:

orchestrion = use_extension("@rules_go//go:extensions.bzl", "orchestrion")
orchestrion.from_source(
    version = "v1.9.0",
    dd_trace_go_versions = {
        "github.com/DataDog/dd-trace-go/v2": "v2.9.0",
        "github.com/DataDog/dd-trace-go/contrib/net/http/v2": "v2.9.0",
        "github.com/DataDog/dd-trace-go/contrib/log/slog/v2": "v2.9.0",
    },
)
use_repo(orchestrion, "rules_go_orchestrion_tool")

The maintained repository integration scripts validate the hermetic Go path with explicit Bazel flags in the script itself. There is no special repo-root --config=hermetic shortcut for this flow.

If both settings are omitted, the default is still v2.9.0. Manual setups must keep the local Go module pins on the same effective versions, or the build will stop with a mismatch error. Do not set both dd_trace_go_version and dd_trace_go_versions in the same orchestrion.from_source(...) call. Bootstrap also refuses to take over tracer settings that are already managed manually outside its own managed block.

Basic usage

load("@rules_go//go:def.bzl", "go_library")
load("@datadog-rules-test-optimization-go//:topt_go_test.bzl", "dd_topt_go_test")
load("@test_optimization_data//:export.bzl", "topt_data")

go_library(
    name = "pkg_lib",
    srcs = ["*.go"],
)

dd_topt_go_test(
    name = "pkg_go_test",
    srcs = ["*_test.go"],
    embed = [":pkg_lib"],  # enables provider-based importpath inference
    topt_data = topt_data,
)

dd_topt_go_test enables CI Visibility and payload-to-files mode by default, so opt-in Go tests emit payload files without extra --test_env settings.

Go Orchestrion mode selection

dd_topt_go_test accepts orchestrion_mode:

Value Description
general Default. Preserves broad generic Orchestrion behavior for consumers that need instrumentation beyond the Test Optimization-only Go testing path.
test_optimization Standard Go testing Test Optimization mode. Keeps the stdlib testing, synthetic testmain, helper packagefile, importcfg, and link support needed for payloads while leaving customer package compiles and external _test package compiles on the normal rules_go path.

For standard Go testing onboarding, including large WORKSPACE monorepos that only need Test Optimization payloads, use the test_optimization mode shown next. The basic snippet above leaves the default general mode implicit.

Standard testing Orchestrion mode

dd_topt_go_test also accepts:

dd_topt_go_test(
    name = "pkg_go_test",
    srcs = ["*_test.go"],
    embed = [":pkg_lib"],
    orchestrion_mode = "test_optimization",
    orchestrion_pin_files = [
        "//:go.mod",
        "//:go.sum",
        "//:orchestrion.tool.go",
        "//:orchestrion.yml",
    ],
    topt_data = topt_data,
)

In this mode, the Go fork keeps the standard-library testing path, synthetic testmain helper packagefiles, importcfg rewrites, and final link support needed for Test Optimization payloads. Customer package compiles and external _test package compiles stay on the normal rules_go compile path.

When enabled, the macro also applies test-binary-only linker flags in Bazel modes where rules_go is not already stripping. Set enable_test_binary_linker_optimization = False on a specific dd_topt_go_test target or repo-local wrapper if a consumer needs the exact caller-provided linker flags during rollout.

This mode is for standard Go testing support. Automatic testify/suite instrumentation is not supported in test_optimization mode.

If the tracer needs runtime-visible source files for AST-derived metadata such as test.source.end, enable source staging explicitly:

dd_topt_go_test(
    name = "pkg_go_test",
    srcs = ["*_test.go"],
    embed = [":pkg_lib"],
    orchestrion_mode = "test_optimization",
    stage_sources = True,
    topt_data = topt_data,
)

stage_sources stages only the target's direct srcs and direct embedsrcs. When enabled, it changes the default rundir to . only if you did not already set rundir. An explicit rundir still wins unchanged.

Then run tests, doctor, enrichment dry-run, and upload with the same BEP-required flow shown in Basic usage: let the wrapper generate a fresh BEP file path for each bazel test invocation, pass those paths to doctor/uploader via repeatable --bep-json, and keep --freshness-source=bep --freshness-mode=required --artifact-source=bep enabled in CI.

If tests use local fixtures

Declare fixture files explicitly in data:

dd_topt_go_test(
    name = "pkg_go_test",
    srcs = ["*_test.go"],
    data = glob(["testdata/**"]),
    orchestrion_mode = "test_optimization",
    topt_data = topt_data,
)

The bootstrap-generated //tools/build:dd_go_test.bzl wrapper forwards **kwargs, so it supports stage_sources = True the same way without wrapper changes.

Import path inference

The macro auto-selects the correct per-module payloads by inferring the Go package importpath using rules_go providers, mirroring how go_test computes it:

  • Precedence:
    1. importpath explicitly set on your go_test invocation (if provided in kwargs)
    2. Inference via embed = [":<go_library>"] by reading GoArchive.importpath from rules_go (recommended)
    3. Fallback: <go module path>/<bazel package> where the Go module path comes from the synced repo's exported topt_data["runtimes"]["go"]["module_path"]

When automatic per-module selection is close but not exact (for example, custom import path layouts), use module_label_override to pin the expected sanitized module suffix:

dd_topt_go_test(
    name = "pkg_go_test",
    srcs = ["*_test.go"],
    embed = [":pkg_lib"],
    module_label_override = "github_com_example_custom_pkg",
    orchestrion_mode = "test_optimization",
    topt_data = topt_data,
)

Multi-service usage

This is the advanced/manual path. Guided bootstrap is intentionally limited to fresh single-service Go workspaces.

load("@datadog-rules-test-optimization-go//:topt_go_test.bzl", "dd_topt_go_test")
load("@test_optimization_data//:export.bzl", "topt_data_by_service")

dd_topt_go_test(
    name = "pkg_go_test",
    srcs = ["*_test.go"],
    orchestrion_mode = "test_optimization",
    topt_data = topt_data_by_service,   # pass mapping
    topt_service = "go_service_a",      # select service
)

Other languages (without companion macro)

Core sync + uploader support is runtime-agnostic and works for any language runtime that honors the file-mode contract. This section is primarily for languages beyond the first-class companions (go, python, java, nodejs, dotnet, ruby).

Repository layout note: keep tools/ runtime-agnostic (tools/core, tools/tests, tools/dev). Add first-class language orchestration under modules/<language>/ companion modules instead of creating tools/<language> placeholder packages.

Generic wrapper pattern

For non-Go rules, wire the same env/data contract in your own test macro:

load("@test_optimization_data//:export.bzl", "topt_data")

manifest_label = "@%s//:%s" % (topt_data["repo_name"], topt_data["manifest_path"])

my_lang_test(
    name = "my_lang_test",
    srcs = ["test_file.ext"],
    data = [
        "@test_optimization_data//:test_optimization_files",
        manifest_label,
    ],
    env = {
        "DD_TEST_OPTIMIZATION_MANIFEST_FILE": "$(rlocationpath %s)" % manifest_label,
        "DD_TEST_OPTIMIZATION_PAYLOADS_IN_FILES": "true",
    },
)

Depending on your rule set, the exact attributes may differ (env, data, args, wrapper script, etc.), but the required contract is always:

  1. Resolve DD_TEST_OPTIMIZATION_MANIFEST_FILE via runfiles
  2. Set DD_TEST_OPTIMIZATION_PAYLOADS_IN_FILES = "true"
  3. Write payloads to TEST_UNDECLARED_OUTPUTS_DIR/payloads/{tests,coverage}

First-class companion macros in this repository also default DD_SERVICE from the selected sync metadata service name. They never override an explicit caller DD_SERVICE, and they leave env = select(...) unchanged.

Building a first-class companion module

If you want dd_topt_<language>_test-style first-class support in this repo, follow the maintainer checklist in docs/Maintainers.md.

Troubleshooting

Use the quick triage map and detailed playbook in docs/Troubleshooting.md.

Fast checks before diving deep:

  • Verify sync env forwarding (DD_API_KEY, DD_SITE, and required DD_GIT_*) through --repo_env, not --test_env.
  • Force metadata refresh only when you intentionally need fresh backend state:
    • bazel sync --only=<repo_name> --repo_env=FETCH_SALT="$(date +%s)"
    • If Bazel reports WORKSPACE-disabled sync errors, retry with: bazel sync --enable_workspace --only=<repo_name> --repo_env=FETCH_SALT="$(date +%s)"
    • Do not add FETCH_SALT to normal bazel test, doctor, or uploader commands.
  • Confirm payload files exist under bazel-testlogs/*/test.outputs/, or that BEP artifact resolution is configured to stage them before upload.
  • For RBE, prefer the recommended test config: --remote_download_minimal --remote_download_regex=.*test[.]outputs.* --zip_undeclared_test_outputs, plus DD_TEST_OPTIMIZATION_ARTIFACT_SOURCE=bep for doctor/uploader. If BEP references HTTP/HTTPS outputs.zip artifacts, use --artifact-source=bep with --remote-artifacts=download or required. For bytestream/CAS or HTTP endpoints requiring custom auth, also configure a downloader.
  • Enable debug logging on sync/uploader rules for richer diagnostics
  • If needed, file an issue with sanitized logs:
    • open an issue in the repository issue tracker

Tips

  • Maintainers: this repository's ./bazelw supports FETCH_SALT_TTL (for example: FETCH_SALT_TTL=3600 ./bazelw build //tools/... //examples/...).
  • For debugging, set debug = True when calling the extension to get verbose logs, including request bodies and detected OS info.

About

A Bazel extension for Test Optimization product

Resources

License

Contributing

Security policy

Stars

1 star

Watchers

0 watching

Forks

Packages

 
 
 

Contributors