Skip to content

benchmarking: dial atenet-router over SPIFFE mTLS on port 443 - #3089

Merged
Max Smythe (maxsmythe) merged 1 commit into
agent-substrate:mainfrom
sairajp-rewind:fix/benchmarks-router-mtls
Oct 24, 2026
Merged

Max Smythe (maxsmythe) merged 1 commit into
agent-substrate:mainfrom
sairajp-rewind:fix/benchmarks-router-mtls

Conversation

@sairajp-rewind

@sairajp-rewind Sairaj Pokale (sairajp-rewind) commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

What this PR does and why we need it ?

Following #2977 (5d5dd7ae), atenet-router no longer exposes plaintext HTTP on :80 and requires SPIFFE mTLS on :443 (https://atenet-router.ate-system.svc), verifying the server leaf against /run/servicedns-ca/ca.crt (SAN atenet-router.ate-system.svc) and presenting /run/podidentity.podcert.ate.dev/credential-bundle.pem.
This PR updates all benchmark data-plane clients to dial atenet-router over SPIFFE mTLS on :443:

  • internal/benchmarking/boomer/boomerutil:
    • Add DefaultRouterURL = "https://atenet-router.ate-system.svc" and RouterTLSConfig(), cached with sync.OnceValues and returning a Clone() configured with RootCAs from credbundle.ParsePool("/run/servicedns-ca/ca.crt"), GetClientCertificate from credbundle.ClientLoader("/run/podidentity.podcert.ate.dev/credential-bundle.pem"), and ServerName: "atenet-router.ate-system.svc".
    • Add unit tests in client_test.go covering mTLS handshake success, mismatched ServerName rejection, and missing credential files.
  • cmd/benchmarking/boomer-worker:
    • Default --router-url to boomerutil.DefaultRouterURL and fail fast on startup if --router-url does not start with https://.
    • Configure http.Transport.TLSClientConfig via boomerutil.RouterTLSConfig() and keep ForceAttemptHTTP2 = false so Go does not implicitly enable HTTP/2 ALPN when TLSClientConfig is non-nil (preserving HTTP/1.1 connection-pool semantics for glutton and durdir).
  • internal/benchmarking/boomer/ateenv:
    • Update routerGRPCTarget to require https:// and default to :443.
    • Dial atenet-router with credentials.NewTLS(tlsCfg) from boomerutil.RouterTLSConfig().
  • benchmarking/nighthawk-ingress & benchmarking/nighthawk-egress:
    • Update DEFAULT_ROUTER_URL to https://atenet-router.ate-system.svc:443 and configure _warm_actor requests.Session with verify = "/run/servicedns-ca/ca.crt" and cert = "/run/podidentity.podcert.ate.dev/credential-bundle.pem".
    • Reject non-https:// URIs in build_spec_dict and populate tls_context (sni: "atenet-router.ate-system.svc", tls_certificates client cert + key from /run/podidentity.podcert.ate.dev/credential-bundle.pem, trusted_ca from /run/servicedns-ca/ca.crt, and match_typed_subject_alt_names exact DNS match on atenet-router.ate-system.svc).
  • benchmarking/locust/tests/counter_demo.py:
    • Switch ROUTER_URL to https://atenet-router.ate-system.svc and configure http_session.verify and http_session.cert.

Testing

  • Unit tests: go test -race -count=1 ./cmd/benchmarking/... ./internal/benchmarking/... ./benchmarking/..., benchmarking/automation unit tests, and Nighthawk proto round-trip & URI validation tests (test_spec_round_trips_through_real_protos, test_rejects_non_https_uri).
  • Live GKE cluster smoke tests (glutton, durdir, and ate-env on gVisor): 0 failures across all runs; verified atenet-router Envoy metrics (ssl_handshake > 0, ssl_fail_verify_* == 0, HTTP/1.1 connections for glutton/durdir, and HTTP/2 connections for ate-env).

Fixes #3088

  • Tests pass
  • Docs updated

Comment thread internal/ateapiauth/client.go Resolved
Comment thread cmd/benchmarking/boomer-worker/main.go Resolved
@chw120

Copy link
Copy Markdown
Collaborator

LGTM

atenet-router removed plaintext HTTP ingress on port 80 and now requires SPIFFE client certificates over TLS on port 443 with the 3-label SAN atenet-router.ate-system.svc.

Add boomerutil.RouterTLSConfig() using credbundle.ParsePool and credbundle.ClientLoader, and update boomer-worker, ateenv, nighthawk-ingress, and counter_demo.py to verify /run/servicedns-ca/ca.crt against atenet-router.ate-system.svc and present /run/podidentity.podcert.ate.dev/credential-bundle.pem. Keep ForceAttemptHTTP2 disabled on boomer-worker's HTTP transport so VUs do not multiplex onto a single Envoy worker thread.

@maxsmythe Max Smythe (maxsmythe) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, this is a benchmarking-only PR

@maxsmythe
Max Smythe (maxsmythe) added this pull request to the merge queue Oct 24, 2026
Merged via the queue into agent-substrate:main with commit 1756c0f Oct 24, 2026
16 checks passed
Giuseppe Cofano (GiuseppeCofano) pushed a commit to GiuseppeCofano/substrate that referenced this pull request Nov 6, 2026
…substrate#3089)

Following agent-substrate#2977 (`5d5dd7ae`), `atenet-router` no longer exposes
plaintext HTTP on `:80` and requires SPIFFE mTLS on `:443`
(`https://atenet-router.ate-system.svc`), verifying the server leaf
against `/run/servicedns-ca/ca.crt` (SAN `atenet-router.ate-system.svc`)
and presenting `/run/podidentity.podcert.ate.dev/credential-bundle.pem`.
This PR updates all benchmark data-plane clients to dial `atenet-router`
over SPIFFE mTLS on `:443`:
- **`internal/benchmarking/boomer/boomerutil`**:
- Add `DefaultRouterURL = "https://atenet-router.ate-system.svc"` and
`RouterTLSConfig()`, cached with `sync.OnceValues` and returning a
`Clone()` configured with `RootCAs` from
`credbundle.ParsePool("/run/servicedns-ca/ca.crt")`,
`GetClientCertificate` from
`credbundle.ClientLoader("/run/podidentity.podcert.ate.dev/credential-bundle.pem")`,
and `ServerName: "atenet-router.ate-system.svc"`.
- Add unit tests in `client_test.go` covering mTLS handshake success,
mismatched `ServerName` rejection, and missing credential files.
- **`cmd/benchmarking/boomer-worker`**:
- Default `--router-url` to `boomerutil.DefaultRouterURL` and fail fast
on startup if `--router-url` does not start with `https://`.
- Configure `http.Transport.TLSClientConfig` via
`boomerutil.RouterTLSConfig()` and keep `ForceAttemptHTTP2 = false` so
Go does not implicitly enable HTTP/2 ALPN when `TLSClientConfig` is
non-nil (preserving HTTP/1.1 connection-pool semantics for `glutton` and
`durdir`).
- **`internal/benchmarking/boomer/ateenv`**:
- Update `routerGRPCTarget` to require `https://` and default to `:443`.
- Dial `atenet-router` with `credentials.NewTLS(tlsCfg)` from
`boomerutil.RouterTLSConfig()`.
- **`benchmarking/nighthawk-ingress` &
`benchmarking/nighthawk-egress`**:
- Update `DEFAULT_ROUTER_URL` to
`https://atenet-router.ate-system.svc:443` and configure `_warm_actor`
`requests.Session` with `verify = "/run/servicedns-ca/ca.crt"` and `cert
= "/run/podidentity.podcert.ate.dev/credential-bundle.pem"`.
- Reject non-`https://` URIs in `build_spec_dict` and populate
`tls_context` (`sni: "atenet-router.ate-system.svc"`, `tls_certificates`
client cert + key from
`/run/podidentity.podcert.ate.dev/credential-bundle.pem`, `trusted_ca`
from `/run/servicedns-ca/ca.crt`, and `match_typed_subject_alt_names`
exact `DNS` match on `atenet-router.ate-system.svc`).
- **`benchmarking/locust/tests/counter_demo.py`**:
- Switch `ROUTER_URL` to `https://atenet-router.ate-system.svc` and
configure `http_session.verify` and `http_session.cert`.
- Unit tests: `go test -race -count=1 ./cmd/benchmarking/...
./internal/benchmarking/... ./benchmarking/...`,
`benchmarking/automation` unit tests, and Nighthawk proto round-trip &
URI validation tests (`test_spec_round_trips_through_real_protos`,
`test_rejects_non_https_uri`).
- Live GKE cluster smoke tests (`glutton`, `durdir`, and `ate-env` on
gVisor): 0 failures across all runs; verified `atenet-router` Envoy
metrics (`ssl_handshake > 0`, `ssl_fail_verify_* == 0`, HTTP/1.1
connections for `glutton`/`durdir`, and HTTP/2 connections for
`ate-env`).

Fixes agent-substrate#3088

- [x] Tests pass
- [x] Docs updated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Benchmarks dial removed plaintext port 80 on atenet-router

3 participants