You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit c6e9dc7
Browse filesBrowse the repository at this point in the historyBrowse files
docs(observability): describe the OTLP relay on the split boundary
Document the relay as it works on the RFC 0012 architecture: the reserved
unroutable destination, the seccomp-staged path into the supervisor, the
gateway-side environment injection, capability gating, buffering, drain
bounds, and the enriched resource attributes. The Kubernetes runtime page
notes that agent telemetry crosses the Pod boundary over the existing
channel with no NetworkPolicy change, and the gateway config reference
explains what `[openshell.gateway.otlp]` now enables for sandboxes.
The AGENTS.md row for openshell-supervisor-process reflects what the crate
owns after the isolation split.
Refs NVIDIA#2641
Signed-off-by: Roland Huß <[email protected]>
required fields and `at_least_one` constraints against the vendored OCSF 1.8
583
583
schemas.
584
584
585
+
## Telemetry Relay
586
+
587
+
The supervisor relays OpenTelemetry trace data from agent processes to the
588
+
gateway over the session stream, so OTel-instrumented agents reach an
589
+
external collector without any sandbox egress.
590
+
591
+
The relay is opt-in on the gateway. When `[openshell.gateway.otlp]` is
592
+
configured, `CreateSandbox` sets `OTEL_EXPORTER_OTLP_ENDPOINT` to
593
+
`http://192.0.0.8:4318` and `OTEL_EXPORTER_OTLP_PROTOCOL` to `http/protobuf`
594
+
in the sandbox environment unless the caller already set an endpoint. The
595
+
values persist in the stored spec, so restarts inherit them.
596
+
597
+
`192.0.0.8` is the RFC 7600 dummy address and is never routed. A workload
598
+
connect to any non-loopback address is intercepted by the sandbox seccomp
599
+
broker and staged for the supervisor as a `PendingTcpOpen`, so the address is
600
+
a label the supervisor switches on. The proxy recognises it ahead of host
601
+
mapping, policy, and SSRF validation and hands the staged stream to the OTLP
602
+
receiver instead of dialing upstream. The receiver speaks HTTP/1.1 on that
603
+
stream and accepts `POST /v1/traces` as protobuf or JSON. No socket is bound,
604
+
the driver outer fence is untouched, and the isolation contract and boundary
605
+
protocol are unchanged. Loopback cannot serve this purpose because the broker
606
+
completes loopback connects locally without mediation.
607
+
608
+
```
609
+
Agent process --> connect(192.0.0.8:4318) --> seccomp broker stages the open
Copy file name to clipboardExpand all lines: docs/kubernetes/sandbox-runtime.mdx
+11Lines changed: 11 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -85,6 +85,17 @@ OpenShell labels, or read bootstrap Secrets. Set
85
85
`supervisor.sandboxRuntime.networkPolicyEnforced: true` only after you verify that the
86
86
cluster CNI enforces both ingress and egress policies for these namespaces.
87
87
88
+
## Relay Agent Telemetry
89
+
90
+
When the gateway has `[openshell.gateway.otlp]` configured, agents in the
91
+
workload Pod export OpenTelemetry traces to `http://192.0.0.8:4318`. That
92
+
address is never routed. The sandbox seccomp broker stages the connection for
93
+
the supervisor over the existing boundary channel, and the supervisor serves
94
+
the OTLP receiver on that stream, adds sandbox attributes to the spans, and
95
+
forwards them to the collector through the gateway. The workload egress fence
96
+
stays empty and no NetworkPolicy change is needed. See
Copy file name to clipboardExpand all lines: docs/reference/gateway-config.mdx
+6Lines changed: 6 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -297,6 +297,12 @@ Only OpenTelemetry traces are exported. Inbound gRPC and HTTP requests produce s
297
297
298
298
The gateway forwards the OTLP configuration, configured gateway name, configured compute driver, and W3C trace context to managed external drivers. Built-in drivers also export their spans to the same collector through dedicated in-process providers. Driver spans retain the gateway trace context, use a distinct service name such as `openshell-driver-docker` or `openshell-driver-podman`, and carry the same `openshell.gateway.name` and `openshell.gateway.compute_driver` resource attributes as gateway spans. Compute-driver client and server spans use the same fully qualified protobuf operation name, such as `openshell.compute.v1.ComputeDriver/CreateSandbox`, in both the span name and `rpc.method`. The service name and span kind distinguish each side. Backend-prefixed child spans identify implementation work. A streaming watch records a terminal status when observed; consumer teardown without a terminal status leaves the span status unset. Operator-run external drivers own their own telemetry configuration.
299
299
300
+
### Agent Telemetry Relay
301
+
302
+
Configuring this table also enables the sandbox telemetry relay. `CreateSandbox` sets `OTEL_EXPORTER_OTLP_ENDPOINT=http://192.0.0.8:4318` and `OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf` in each new sandbox's environment unless the caller already supplied an endpoint. The values persist in the sandbox spec and survive restarts. OTel SDKs inside the sandbox export to that address over HTTP/protobuf or HTTP/JSON.
303
+
304
+
The address is never routed. The sandbox stages the connection for the supervisor, which serves the OTLP receiver itself, adds `openshell.sandbox.id`, `openshell.workspace.id`, `openshell.sandbox.policy`, `openshell.sandbox.user`, `openshell.sandbox.image`, `openshell.sandbox.driver`, and `openshell.telemetry.source=agent`, and forwards the spans to the gateway. The gateway exports them to the same `endpoint` through a dedicated exporter, so agent spans arrive with the supervisor's resource attributes rather than the gateway's. Agent traces need no sandbox egress and no network policy entry. Telemetry never blocks sandbox control traffic: the relay buffers up to 4096 items per sandbox and drops the newest on overflow.
305
+
300
306
For Helm deployments, set `server.otlp.endpoint` to render this table. The
301
307
optional `server.otlp.serviceName` value overrides the gateway service name;
0 commit comments