|
| 1 | +# Brainstorm: Warm Pool with Always-On Supervisor (gRPC-Push Identity) |
| 2 | + |
| 3 | +**Date:** 2026-07-10 |
| 4 | +**Status:** active |
| 5 | + |
| 6 | +## Problem Framing |
| 7 | + |
| 8 | +The existing warm pool RFC (`rfc/NNNN-warm-pool-feasibility/`) proposes annotation-based identity binding: the supervisor starts at claim time, detects its identity via the Kubernetes downward API, and bootstraps normally. This approach delivers ~3-4s claim latency (2.3s claim + 1.5s supervisor startup). |
| 9 | + |
| 10 | +For workloads that need sub-1s sandbox provisioning, the supervisor startup at claim time is the bottleneck. The 1.5s breaks down as ~80ms for 8 gRPC calls and ~1.4s for process startup plus OPA policy compilation. |
| 11 | + |
| 12 | +An alternative architecture keeps the supervisor always running in the warm pod. Global OPA policies (applicable to every sandbox) are pre-compiled at pool time. At claim time, the gateway pushes the sandbox identity directly to the supervisor via gRPC, the supervisor compiles only sandbox-specific policies, and the session activates. This eliminates process startup entirely and reduces OPA compilation to the per-sandbox delta. |
| 13 | + |
| 14 | +This brainstorm scopes the alternative RFC, its relationship to the annotation RFC, and how gateway-managed pool lifecycle works across multiple sandbox images. |
| 15 | + |
| 16 | +## Approaches Considered |
| 17 | + |
| 18 | +### A: Two standalone RFCs with cross-references |
| 19 | + |
| 20 | +Create a new RFC (`NNNN-warm-pool-grpc-push`) that stands independently alongside the existing annotation RFC. Both get an "Alternative Approaches" section with a comparison table. No ordering or primary/secondary framing. |
| 21 | + |
| 22 | +- Pros: Each RFC is self-contained, reviewable independently. Maintainers can accept one, both, or neither. |
| 23 | +- Cons: Some duplication in shared context (motivation, measurements). |
| 24 | + |
| 25 | +### B: Single RFC with two design variants |
| 26 | + |
| 27 | +Merge both approaches into one RFC with variant sections. |
| 28 | + |
| 29 | +- Pros: No duplication, single document. |
| 30 | +- Cons: Very long. Harder to discuss one variant without the other. |
| 31 | + |
| 32 | +### C: Split RFC by concern (3 documents) |
| 33 | + |
| 34 | +Shared pool infrastructure RFC plus two identity binding RFCs. |
| 35 | + |
| 36 | +- Pros: Cleanest separation. |
| 37 | +- Cons: Three documents is heavy to manage. |
| 38 | + |
| 39 | +## Decision |
| 40 | + |
| 41 | +**Approach A: Two standalone RFCs with cross-references.** The shared context is short enough to duplicate. The pool management section (gateway-managed pools, multi-image config) lives in the new gRPC-push RFC since it's the one proposing gateway-managed pools. The annotation RFC can reference it. |
| 42 | + |
| 43 | +## Key Requirements |
| 44 | + |
| 45 | +### gRPC-Push RFC content |
| 46 | + |
| 47 | +1. **Always-on supervisor in warm pods.** Supervisor starts at pool provisioning time. Pre-compiles global OPA policies. Exposes a new gRPC endpoint (`ActivateSandbox` or similar) for identity push. |
| 48 | + |
| 49 | +2. **Two-tier OPA compilation.** Global policies (applicable to all sandboxes) compile at pool time. Sandbox-specific policies (per-sandbox rules, provider-specific constraints) compile at claim time after identity push. The split must be explicit in the RFC. |
| 50 | + |
| 51 | +3. **Identity binding via gRPC push.** After the operator binds the claim, the gateway reads the pod IP from `.status.sandbox.podIPs`, connects directly to the supervisor's gRPC endpoint, and pushes sandbox identity (ID, name, policy config). No downward API, no annotation propagation delay. |
| 52 | + |
| 53 | +4. **Gateway-managed pool lifecycle.** The gateway owns SandboxTemplate + SandboxWarmPool resource creation, scaling, and cleanup. Gateway config defines which images get pools and their sizes. Gateway reconciles pool state against desired config. |
| 54 | + |
| 55 | +5. **Pool-per-image for known images.** Gateway config lists images with pool sizes. Example: |
| 56 | + ```yaml |
| 57 | + warmPools: |
| 58 | + - image: ghcr.io/nvidia/openshell-community/sandboxes/base:latest |
| 59 | + replicas: 5 |
| 60 | + - image: ghcr.io/nvidia/openshell-community/sandboxes/ollama:latest |
| 61 | + replicas: 2 |
| 62 | + ``` |
| 63 | +
|
| 64 | +6. **Unknown images always cold-start.** Custom Dockerfiles, arbitrary image refs, first-time images go through the existing cold-start path. No auto-promotion in the initial design. |
| 65 | +
|
| 66 | +7. **Dynamic pool promotion as documented extension.** The RFC includes a "Future Extensions" section describing auto-promotion: gateway creates a pool for an image after seeing it N times. Not part of the initial design. |
| 67 | +
|
| 68 | +### Cross-referencing and comparison |
| 69 | +
|
| 70 | +8. **Both RFCs get an "Alternative Approaches" section.** Each links to the other with a comparison table covering: |
| 71 | + - Claim-to-ready latency |
| 72 | + - Supervisor complexity (idle-start vs always-on) |
| 73 | + - Gateway complexity (claim-only vs pool reconciler + gRPC client) |
| 74 | + - OPA compilation (full at claim vs pre-compiled global + delta) |
| 75 | + - Security surface (downward API vs new gRPC endpoint on supervisor) |
| 76 | + - Alignment with upstream (craig-kindo's validated approach vs novel) |
| 77 | + - Resource cost of idle pools (sleeping process vs running supervisor) |
| 78 | +
|
| 79 | +9. **Equal framing.** Neither RFC is "primary." Both are valid alternatives for different latency targets and complexity appetites. |
| 80 | +
|
| 81 | +### Upstream references (shared) |
| 82 | +
|
| 83 | +- [OpenShell#2157](https://github.com/NVIDIA/OpenShell/issues/2157): Feature issue for warm-pool provisioning |
| 84 | +- [OpenShell#1447](https://github.com/NVIDIA/OpenShell/issues/1447): craig-kindo's annotation-based implementation (validates the other RFC) |
| 85 | +- [agent-sandbox#1118](https://github.com/kubernetes-sigs/agent-sandbox/pull/1118): Operator adoption finalization improvement |
| 86 | +- [agent-sandbox#384](https://github.com/kubernetes-sigs/agent-sandbox/issues/384): Upstream file-based env injection tracking |
| 87 | +
|
| 88 | +## Open Questions |
| 89 | +
|
| 90 | +- What is the new gRPC endpoint name and proto definition for the supervisor identity push? |
| 91 | +- How does the gateway discover pool pod IPs before the supervisor has registered? (likely from `.status.sandbox.podIPs` on the adopted Sandbox resource) |
| 92 | +- What happens if the gRPC push fails (supervisor crashed, network partition)? Fall back to cold start or retry? |
| 93 | +- How does gateway config validation prevent conflicting pool definitions (same image, different pool sizes)? |
| 94 | +- Should the gateway pool reconciler run as a background loop or be event-driven (watch SandboxWarmPool status)? |
| 95 | +- How does the always-on supervisor handle image updates? The supervisor process is running an older binary while the pool template references a newer image. |
0 commit comments