Skip to content

Commit 5a5b192

Browse files
authored
docs: remove inaccurate early-stage architecture documentation (agent-substrate#3517)
1. Removes the `status: planned` banner so the page no longer claims to be aspirational. 2. Rewrites the trade-offs section into present-tense implementation context (data locality, control-plane boundaries, and OpenTelemetry observability) and removes the outdated "haven't started to tackle" list. 3. Documents `atenet-egress`, DNS relay filtering, and `EgressPolicy` credential injection under the Networking Stack. 4. Replaces the inline Security & Isolation bullet list with links to the main Security Overview and Threat Model documents. AI use: AI-assisted, I reviewed and tested it. - [x] Tests pass - [x] Docs updated --------- Signed-off-by: Greg Castle <[email protected]>
1 parent 43dc887 commit 5a5b192

1 file changed

Lines changed: 34 additions & 78 deletions

File tree

‎docs/concepts/architecture.md‎

Lines changed: 34 additions & 78 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,6 @@ linkTitle: "Architecture"
44
weight: 1
55
description: >
66
The components of Agent Substrate and how actors, workers, and the control plane interact.
7-
status: planned
8-
status_note: >
9-
Much of this architecture is aspirational and not yet implemented.
107
---
118
# Agent Substrate Architecture
129

@@ -69,45 +66,33 @@ inspect network traffic. Agent Substrate includes a lightweight,
6966
substrate-aware network proxy which can inspect incoming traffic and trigger
7067
the resumption of actors as needed.
7168

72-
### New Problems
69+
### Trade-offs and Design Considerations
7370

7471
Of course, there's no such thing as a free lunch, and these approaches have
7572
trade-offs.
7673

7774
First, we trade one problem (many idle Pods consuming resources) for another.
78-
Doing suspend and resume at this scale introduces a vast data management
75+
Doing suspend and resume at this scale introduces a large data management
7976
problem. We need to store the state of millions of actors, which can update
80-
many times per second. Agent Substrate will need to consider data locality as
81-
a first-class concern. When an event comes in, we need to know where the
82-
actor's most recent state is stored, and either route the event to that
83-
location or move the state to where the event can be serviced.
84-
85-
In addition to that, we must acknowledge that building a new production-grade
86-
control-plane component is a non-trivial effort. It will need to be highly
87-
available, secure, and performant, and it will need to integrate well with
88-
Kubernetes.
89-
90-
Another problem to consider is observability and debugging. With actors being
91-
multiplexed onto and off of workers all the time, it will be more difficult to
92-
understand what is happening. Agent Substrate will need to provide robust
93-
tooling for monitoring and debugging actors, including the ability to thread
94-
together metrics and logs for an actor over time, to inspect the state of
95-
suspended actors, and to trace the events that lead to their current state.
96-
97-
### Problems We Haven't Addressed Yet
98-
99-
As early as this project is, there are several problems that we haven't even
100-
started to tackle yet, but which we know will need to be solved eventually.
101-
102-
* Autoscaling: We will need to be able to automatically scale the number of workers up
103-
and down based on demand. Kubernetes Pod autoscaling may be sufficient or
104-
it might not.
105-
* Peer to peer state sharing: Relying on data locality presents a risk of
106-
data loss. We probably need some form of state-sharing to protect against
107-
that, which needs to be balanced with saving state to a "permanent" store.
108-
* Identity and policy: We know that agents' needs for identity are very
109-
different than traditional workload identity and end-user credentials.
110-
* There are certainly more that we haven't thought of yet!
77+
frequently. Agent Substrate balances fast local reactivation against durable
78+
remote storage: `atelet` keeps a node-local copy of an actor's latest snapshot
79+
after suspend so subsequent resumes on that node avoid an object-store
80+
round-trip, but local copies are ephemeral (held only until
81+
`--snapshot-detach-delay`) and asynchronous suspends risk state loss if the node
82+
fails before background upload completes (see [Actor Lifecycle](#actor-lifecycle)).
83+
84+
Second, a specialized control plane must integrate cleanly with Kubernetes
85+
while enforcing its own security boundaries. `ate-api-server` authenticates
86+
callers via mTLS and OIDC and authorizes every RPC per atespace with an embedded
87+
OpenFGA model, while `atecontroller` reconciles declarative `WorkerPool` CRDs
88+
into Kubernetes Deployments.
89+
90+
Third, with actors multiplexed onto and off of workers, operators need
91+
actor-level and pool-level visibility. Agent Substrate emits OpenTelemetry
92+
metrics, structured logs, and lifecycle events across its components so
93+
operators can monitor worker capacity, track suspend/resume latency, and
94+
correlate activity across an actor's lifetime through container logs (see the
95+
[Observability](../guides/observability.md) guide).
11196

11297
## North Star Metrics
11398

@@ -363,7 +348,7 @@ A `WorkerPool` selects a **sandbox class** (`spec.sandboxClasses[].name`), and e
363348

364349
### Networking Stack (`atenet` + `atunnel`)
365350

366-
Handles actor-aware routing and automatic re-animation.
351+
Handles actor-aware ingress routing, on-demand resumption, and policy-enforced egress.
367352

368353
* **Ingress Routing**: `atenet-router` runs Envoy with an `ext_proc` external
369354
processor. A higher-order system connects to the router and supplies the
@@ -389,7 +374,14 @@ Handles actor-aware routing and automatic re-animation.
389374
HTTP(S) traffic over the tunnel is supported today; raw TCP or other
390375
non-HTTP protocols on a non-default port are not yet reachable this way.
391376

392-
* **Latency**: The data plane is optimized for sub-100ms activation by bypassing Kubernetes' eventual consistency and performing atomic physical assignments.
377+
* **Egress Filtering and Credential Injection**: Outbound traffic from an
378+
Actor is tunneled through `atunnel` to `atenet-egress`, which enforces
379+
destination policy and resolves credential replacement effects without
380+
placing ambient secrets in the sandbox (see the
381+
[Network Egress Contract](../guides/network-egress.md) and
382+
[Egress Credential Injection](../guides/egress-credential-injection.md)).
383+
384+
* **Latency**: The data plane is optimized for low-latency activation by bypassing Kubernetes' eventual consistency and performing atomic physical assignments.
393385

394386
### System Infra Isolation
395387

@@ -554,43 +546,7 @@ Snapshots are stored durably in object storage (GCS or S3) through `objectstore-
554546

555547
## Security & Isolation
556548

557-
Agent Substrate is built on a **Defense-in-Depth** model:
558-
559-
* **Sandboxed Execution**: Every actor runs inside a hardened kernel-space
560-
isolation layer (e.g. gVisor), preventing container escapes.
561-
562-
* **Actor Identity**: Every interaction is powered by a unique, Agent
563-
Substrate-managed identity that is independent of the underlying hardware.
564-
This ensures that actors maintain their own granular permissions and
565-
security context even as they migrate across physical nodes or code
566-
versions.
567-
568-
* **API Authorization**: Every `ate-api-server` RPC is authorized. Users
569-
and automation get owner, editor, or viewer roles on the global scope or
570-
on an atespace through AccessPolicy (`kubectl ate access-policy`), and
571-
Substrate's components hold fixed system roles bound to their pod
572-
certificates. The atespace is the finest boundary today. See
573-
[authorization.md](../guides/authorization.md) for the roles, the check behind each
574-
RPC, and known gaps.
575-
576-
* **Ingress Authentication and Routing**: The gateway routes each request to
577-
the actor its `ate-target-actor` header names, and only to recognized,
578-
registered actors. It accepts only mTLS clients whose SPIFFE ID is on a
579-
static allowlist; any listed client may reach any actor, as per-actor
580-
ingress authorization is not yet supported.
581-
The installer takes that allowlist from `--ingress-allowed-spiffe-ids`
582-
(`ATE_INGRESS_ALLOWED_SPIFFE_IDS` or `ingressAllowedSPIFFEIDs` in its config).
583-
Both dataplanes must verify the client certificate against the Pod Identity
584-
CA during the TLS handshake on both HTTPS and TLS CONNECT listeners.
585-
The SPIFFE URI SAN allowlist may be checked during or after TLS, but an
586-
unlisted client must not reach actor routing. A CONNECT tunnel may be
587-
accepted before that check, but traffic inside it must still be denied.
588-
Neither dataplane serves plaintext ingress listeners.
589-
590-
* **Network Policy**: Agent Substrate leverages standard Kubernetes
591-
NetworkPolicy for connectivity control. Policies can be applied at the
592-
`WorkerPool` boundary to restrict ingress/egress traffic for all actors
593-
hosted within that pool.
594-
595-
* **mTLS Everywhere**: All internal system communication (e.g., Control Plane
596-
to Atelet) is secured via mutual TLS with short-lived certificates.
549+
See the [Security Overview](../security/security.md) and
550+
[Threat Model](../security/threat-model.md) for how Agent Substrate isolates
551+
workloads, authenticates and authorizes callers, and enforces network and
552+
credential boundaries.

0 commit comments

Comments
 (0)