Skip to content

feat(gateway): verify an external gateway with a CA alone - #13909

Merged
rsliter merged 4 commits into
v1from
feat/13834-gateway-ca-only-tls
Oct 28, 2026
Merged

rsliter merged 4 commits into
v1from
feat/13834-gateway-ca-only-tls

Conversation

@gaveezy

@gaveezy gaveezy commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Failure

gateway.tls required ca, certificate and key together, and NemoClaw then presented the client pair (#13834). OpenShell gateways accept OIDC bearer clients without a client certificate, and OpenShell's SDK and CLI pin a CA with no client identity for them. So the only way to reach a token-authenticated external gateway with a private CA was SSL_CERT_FILE, which replaces the system roots for NemoClaw and every provider it starts.

Separately, an inference endpoint signed by a private CA can't be trusted at all. The sandbox supervisor makes that TLS connection, and OpenShell's provider registrations at the pinned v0.1.5 take no CA. The docs didn't name the supervisor image as the trust store, and limits.md linked a docs page instead of an issue.

Decision

flowchart LR
    TLS["gateway.tls"] --> CA["ca alone:<br/>pin the CA, no client identity"]
    TLS --> Pair["ca + certificate + key:<br/>mutual TLS, as before"]
    TLS -.-> Partial["certificate or key alone,<br/>or no ca: rejected"]
    Inference["Private-CA inference endpoint"] -.-> Limit["limits.md: #13834,<br/>NVIDIA/OpenShell#3295"]
Loading

Validation

Each new test failed against the code before its change, for the stated reason, and now passes:

  • config_schema::external_gateway_tls_takes_a_ca_alone_or_with_a_client_certificate_and_key: the parser and schema rejected a CA alone. It also checks that ca with only certificate or only key, and certificate with key but no ca, are rejected.
  • tls::a_ca_alone_verifies_the_gateway_without_a_client_certificate: against a fixture gateway that requests no client certificate, the connection was refused with Authentication. It now creates the workspace, and a CA from another authority fails with "the gateway's TLS server certificate is not signed by the configured certificate authority".
  • client::tests::tls_references_name_a_ca_alone_or_with_a_certificate_and_key: the data source reported "Incomplete TLS credential references" for a CA alone; partial pairs still fail.

Checks:

  • The existing mutual TLS test still passes, including its server-trust, client-trust and missing-key failures.
  • schema --check, docs --check, and cargo clippy --workspace --all-targets -D warnings pass.
  • cargo ci passed locally on v1 876b44f15, and again after merging v1 15df7d09b: fmt, clippy, build, 1,734 unit tests, schema, bundle, and 156 lifecycle tests.
  • Not run: a live external OIDC gateway (docs/limits.md).

For #13834; PRs into v1 don't close issues, so it needs closing by hand after merge. Part of #13837.

gateway.tls required ca, certificate and key together, and NemoClaw then
presented the client pair. OpenShell gateways accept OIDC bearer
clients without a client certificate, and OpenShell's SDK and CLI pin a
CA without a client identity for them, so a token-authenticated gateway
with a private CA could only be reached by replacing the process-wide
roots with SSL_CERT_FILE.

Make certificate and key optional, but together. With only a CA, the
connection pins that CA and presents no client identity. The
openshell_gateway data source applies the same rule to its tls_*_env
inputs, and cached clients are keyed by the optional identity.

Inference endpoints signed by a private CA stay a limit: the supervisor
makes that TLS connection, and OpenShell's provider registrations take
no CA. limits.md lists it with NVIDIA/OpenShell#3295, and inference.md
names the supervisor image as the trust store.
@github-actions

Copy link
Copy Markdown
Contributor

linux_arm64 / Build and test failed only on the time budget: bake::agent_builds_default_to_the_host_platform_and_reject_others and bake::amd64_builds_every_harness took 23 s each against 8 s; all 1,734 tests passed, and the same job passed on v1 15df7d0.
@gaveezy
gaveezy requested a review from cv October 28, 2026 13:55
@gaveezy

gaveezy commented Oct 28, 2026

Copy link
Copy Markdown
Contributor Author
Authoring session — opening
Field Status Value Evidence or limit
Harness unavailable - Not reported at the author's request.
Skills unavailable - Not reported at the author's request.
Instruction files unavailable - Not reported at the author's request.
Tool calls unavailable - Not reported at the author's request.
Session window unavailable - Not reported at the author's request.
Compactions unavailable - Not reported at the author's request.
Models unavailable - Not reported at the author's request.
Reasoning levels unavailable - Not reported at the author's request.

@wscurran wscurran added area: networking DNS, proxy, TLS, ports, host aliases, or connectivity area: security Security controls, permissions, secrets, or hardening feature PR adds or expands user-visible functionality security v1 NemoClaw v1 branch labels Oct 28, 2026

@rsliter rsliter left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved at 5ef9565. #13080 explicitly accepted CA-only TLS for external gateways and #13834 defines the narrow behavior and validation. The schema and provider reject partial client identities, and the live TLS fixture verifies bearer access with the configured CA plus wrong-CA refusal. Existing mutual TLS remains covered. The exact-head v1 build, lifecycle, live Docker/Kind, docs, and required policy checks pass. No scope expansion or actionable defect found.

@rsliter
rsliter merged commit cb5690d into v1 Oct 28, 2026
58 of 60 checks passed
@rsliter
rsliter deleted the feat/13834-gateway-ca-only-tls branch October 28, 2026 21:57
@wscurran wscurran removed the security label Oct 29, 2026
cv added a commit that referenced this pull request Nov 11, 2026
…ms (#14413)

The release notes now list every user-visible change on `v1` since the
`v0.1.0-rc9` tag, including the breaking ones, and four example pages
drop claims that later changes made false.

## Problem

`docs/about/release-notes.md` listed three breaking changes and none of
the features, fixes, and onboarding changes that reached `v1` after
`v0.1.0-rc9`; it didn't name the configuration shapes the parser now
rejects or the intent record version `v0.1.0-rc9` writes.
Some example claims went stale:

- `examples/spark/README.md` said the pinned OpenShell has the Podman
TLS initialization blocker, NVIDIA/OpenShell#3427; its fix merged before
OpenShell 0.1.0, and `v1` pins 0.1.7.
- `examples/station/README.md` and `examples/gtc-dc/README.md` required
exactly one visible GPU on the host; since #13929,
`container.gpuDevices` lets a service on a host with more GPUs request
one.
- `examples/hcl/travel-team` said Fabric's Deep Agents adapter can't
turn reasoning off, NVIDIA/NeMo-Fabric#433; NVIDIA/NeMo-Fabric#439 fixed
that, and the pinned catalog's Deep Agents `reasoning_effort` accepts
`none`.
- The travel team cited #13313 for the booking key its service doesn't
receive, but #13313 closed when `openshell_api` landed.

## Change

- `docs/about/release-notes.md` gains "Changes Since v0.1.0-rc9",
grouped by area, from the commits in `v0.1.0-rc9..origin/v1`; where a
later commit changed an earlier one, the entry describes the current
code, and changes added and removed after the tag aren't listed:
- Agents and interfaces: #14037, #14038, #14269, #13981, #14024, #14186,
#14052, #14385, #13971, #13858, #13947, #14039, #14042, #14044, #14343.
- Inference: #13929, #14391, #14396, #14398, #14400, #13963, #13912,
#13964, #14388, #14390, #14397.
- Gateways and Kubernetes: #13909, #13970, #13922 (an existing
namespace, refused at plan by #13756, is now bound; cluster-wide reads
dropped), #14014, #14169, #13921.
- Plan, apply, and diagnostics: #14224, #13756, #14141, #14133, #14135,
#13775, #13744, #13701.
- Onboarding: #13693, #13695, #13741, #13757, #13851, #13978, #14228,
#14233, #14334, #14336.
- Examples and builds: #14166, #14184, #14188, #14191, #14192, #14200,
#13889, #14267, #14312, #14315, #14355, b30e9b4, #13781, #13813,
#14246, and the pins in `versions.json` and `image/fabric/Dockerfile`.
- Its breaking changes since the tag:
- the intent record moving from 15, which `v0.1.0-rc9` writes, through
16 (#14266), 17 (#14343), 18 (#13929), 19 (#14391) and 20 (#14394);
  - name-keyed `sandboxes` and `inferenceProviders` (#13703, #13705);
- a recipe `compatibility.gpu` outside every hardware profile (#13912);
- model tuning the adapter doesn't accept (#14111), and Hermes's
`max_tokens` losing its default (#14296);
  - `skills.paths` resolution (#14296);
  - endpoint and `networkCIDR` validation (#13847, #14081);
- the HCL `gateway_values` attribute (#14014), and Kubernetes ownership
receipts from before #13922 and #14014.
  The two earlier items stay, under their own heading.
- `examples/spark/README.md` replaces the closed OpenShell#3427 blocker
with the Podman and remote-model limits in `docs/resources/limits.md`.
- `examples/station/README.md` and `examples/gtc-dc/README.md` require
one GPU visible in the model container, and point to
`container.gpuDevices` for a host with more.
- `examples/hcl/travel-team/README.md` and `WALKTHROUGH.md` drop the
NeMo-Fabric#433 limitation, and the travel team's booking-key statements
link #14417 instead of #13313.

Not listed, because both changes landed after the tag: Deep Agents Code
(#14043, #14250), `compatibility.gpuCount` (#14391, #14394), and the
"Creating infrastructure state" progress step (#14169, #14248).

Checked and left unchanged: the root `README.md`,
`crates/nemoclaw-onboarding/README.md`,
`crates/nemoclaw-cli/assets/README.md`, `internal-docs/README.md`, the
other example READMEs, and every example YAML header comment. The GPU
autoscaling README and `load.sh` read the JSON fields `OperationResult`
serializes, and NVIDIA/OpenShell#4287 is fixed only after the pinned
0.1.7. The two pre-tag release-note items still match the schema and
SDK. `docs/README.md` is left to #14412, which retires the stale field
reference snapshot it describes.

Not verified: whether Hermes sends the booking key's placeholder to MCP
servers; #14417 tracks the live check.

## Beyond CI

Each relative link and anchor in the changed files resolves to an
existing file and heading.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: networking DNS, proxy, TLS, ports, host aliases, or connectivity area: security Security controls, permissions, secrets, or hardening feature PR adds or expands user-visible functionality v1 NemoClaw v1 branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants