This document illustrates the API proposed in NVIDIA/OpenShell#3073. The protobuf contract is authoritative. The examples use protobuf text-style objects to show event order without tying the discussion to one generated API.
HttpResponsePreReturn.Evaluate is a bidirectional gRPC stream for one HTTP response and one middleware stage. OpenShell writes HttpResponseEvent messages. The middleware writes matching HttpResponseEventResult messages.
HTTP_REQUEST/PRE_CREDENTIALS runs after network policy admits the destination and before OpenShell injects credentials. HTTP_RESPONSE/PRE_RETURN starts after OpenShell receives the final upstream response head and before it returns that response to the sandbox. Both evaluations carry the same context.request_id.
sequenceDiagram
participant C as Sandbox client
participant S as OpenShell supervisor
participant U as Upstream
participant M as Response middleware stage
C->>S: HTTP request
S->>U: Forward admitted request
U-->>S: Final response head
S->>M: preflight with stage-local permitted_body_modes
M-->>S: preflight_result with skip, inspect, or block_delivery
loop Body units for an inspecting stage
S->>M: body with sequence and optional end_of_stream
M-->>S: body_result with the matching sequence
end
opt Stage completed body inspection normally
S->>M: trailers, including an empty set
M-->>S: trailers_result
end
S-->>M: session_end
Interim 1xx responses pass through unchanged before the hook starts. 101 upgrades, successful CONNECT tunnels, and post-upgrade traffic bypass generic response processing.
- OpenShell starts with one
preflight. It includes the current head, request context, effective payload limit, and modes available to this stage. - The middleware returns one
preflight_resultwithskip,inspect, orblock_delivery. - OpenShell derives response-shape eligibility from the original final head, then applies each stage's payload limit. Stages may receive different
permitted_body_modeslists. skip,block_delivery, andinspect { body_mode: HEADERS_ONLY }receive no body or trailer events.WHOLE_BODY_BYTESproduces one body unit withend_of_stream: true, including a present empty sequence-1 unit for an empty body-capable response.- A normally completed
STREAM_BYTESstage receives one or more units. Exactly one hasend_of_stream: true; it may be empty when OpenShell learns about EOF after sending the last data. - Streaming inputs are at most
min(64 KiB, max_payload_bytes). Each replacement may use up tomax_payload_bytes. STREAM_BYTESis unit-local in V1. Every result accounts for its complete input unit; middleware cannot retain input across units.- Every body unit requires one lockstep result with the matching sequence.
pass_throughpreserves it,transformreplaces it,block_deliverystops delivery, andskip_remainingfinalizes it before opting this stage out of the suffix. - A stage that finishes body inspection normally receives one
trailersevent after its final body result, including when the set is empty, and returns onetrailers_result. - Trailer mutations may modify or remove names in the event. V1 cannot create a missing trailer name.
reason,reason_code,findings, andmetadatasit beside the selected action on every result envelope. Invalid diagnostics invalidate the complete result and followon_error.- OpenShell may finish an opened stream with one best-effort
session_end. It requires no result.
OpenShell -> middleware
preflight {
context { request_id: "req-7f3" sandbox_id: "sandbox-42" }
target { method: "GET" host: "api.example.com" path: "/profile" }
status_code: 200
headers { name: "content-type" value: "application/json" }
headers { name: "content-length" value: "31" }
headers { name: "etag" value: "upstream-version" }
middleware_name: "response-redactor"
max_payload_bytes: 4194304
permitted_body_modes: HTTP_RESPONSE_BODY_MODE_HEADERS_ONLY
permitted_body_modes: HTTP_RESPONSE_BODY_MODE_WHOLE_BODY_BYTES
permitted_body_modes: HTTP_RESPONSE_BODY_MODE_STREAM_BYTES
}
middleware -> OpenShell
preflight_result {
inspect {
body_mode: HTTP_RESPONSE_BODY_MODE_WHOLE_BODY_BYTES
header_mutations { remove { name: "etag" } }
}
reason_code: "json_response_selected"
metadata { key: "rule_set" value: "customer-data-v3" }
}
OpenShell -> middleware
body {
sequence: 1
data: "{\"name\":\"Ada\",\"token\":\"secret\"}"
end_of_stream: true
}
middleware -> OpenShell
body_result {
sequence: 1
transform { data: "{\"name\":\"Ada\",\"token\":\"[redacted]\"}" }
reason_code: "sensitive_value_redacted"
}
OpenShell -> middleware
trailers {}
middleware -> OpenShell
trailers_result {
reason_code: "no_upstream_trailers"
}
OpenShell -> middleware
session_end { reason: MIDDLEWARE_SESSION_END_REASON_NORMAL }
The empty trailer event and empty mutation list are explicit. No trailer names are created. For an empty body-capable response, OpenShell still sends sequence 1 with present empty data and end_of_stream: true; middleware may replace it with nonempty bytes.
The original response has Content-Length: 98304. Both stages may stream it, but only the stage with the larger limit may select whole-body mode.
OpenShell -> stage-a
preflight {
context { request_id: "req-mixed" }
status_code: 200
headers { name: "content-length" value: "98304" }
middleware_name: "large-document-scanner"
max_payload_bytes: 131072
permitted_body_modes: HTTP_RESPONSE_BODY_MODE_HEADERS_ONLY
permitted_body_modes: HTTP_RESPONSE_BODY_MODE_WHOLE_BODY_BYTES
permitted_body_modes: HTTP_RESPONSE_BODY_MODE_STREAM_BYTES
}
OpenShell -> stage-b
preflight {
context { request_id: "req-mixed" }
status_code: 200
headers { name: "content-length" value: "98304" }
middleware_name: "small-window-redactor"
max_payload_bytes: 65536
permitted_body_modes: HTTP_RESPONSE_BODY_MODE_HEADERS_ONLY
permitted_body_modes: HTTP_RESPONSE_BODY_MODE_STREAM_BYTES
}
The original response head defines shape eligibility. Each stage's effective limit then narrows its list. For an eligible response without a known length, a stage may select WHOLE_BODY_BYTES and later receive whole_body_over_capacity through its on_error behavior.
Each streaming result accounts for exactly one input unit. This stage transforms two independently meaningful event units and does not retain bytes between them.
OpenShell -> middleware
preflight {
context { request_id: "req-814" }
target { method: "GET" host: "events.example.com" path: "/events" }
status_code: 200
headers { name: "content-type" value: "text/event-stream" }
middleware_name: "stream-scrubber"
max_payload_bytes: 65536
permitted_body_modes: HTTP_RESPONSE_BODY_MODE_HEADERS_ONLY
permitted_body_modes: HTTP_RESPONSE_BODY_MODE_STREAM_BYTES
}
middleware -> OpenShell
preflight_result {
inspect { body_mode: HTTP_RESPONSE_BODY_MODE_STREAM_BYTES }
reason_code: "stream_selected"
}
OpenShell -> middleware
body { sequence: 1 data: "data: status=ready\n\n" }
middleware -> OpenShell
body_result { sequence: 1 pass_through {} }
OpenShell -> middleware
body { sequence: 2 data: "data: token=secret\n\n" }
middleware -> OpenShell
body_result {
sequence: 2
transform { data: "data: token=[redacted]\n\n" }
reason_code: "token_redacted"
}
# EOF was learned after sequence 2, so OpenShell sends an empty final unit.
OpenShell -> middleware
body { sequence: 3 data: "" end_of_stream: true }
middleware -> OpenShell
body_result { sequence: 3 pass_through {} }
OpenShell -> middleware
trailers {}
middleware -> OpenShell
trailers_result {}
The stage cannot withhold a prefix from one unit and complete it from a later unit. Middleware that needs bounded cross-boundary processing must select WHOLE_BODY_BYTES. Cross-unit streaming requires a future explicit hold or ownership operation where OpenShell retains the original bytes, so fail-open recovery remains safe.
OpenShell -> prefix-inspector
body { sequence: 1 data: "{\"kind\":\"public-catalog\",\"items\":[..." }
prefix-inspector -> OpenShell
body_result {
sequence: 1
skip_remaining { pass_through {} }
reason_code: "public_document"
}
# This stage receives no later body or trailer events. The current output and
# units 2..n continue through every later stage in the chain.
OpenShell -> prefix-inspector
session_end { reason: MIDDLEWARE_SESSION_END_REASON_NORMAL }
skip_remaining.transform replaces only the current input unit before the stage opts out.
Repeated trailer names remain separate and ordered.
OpenShell -> middleware
trailers {
headers { name: "digest" value: "sha-256=:old:" }
headers { name: "x-review" value: "first" }
headers { name: "x-review" value: "second" }
}
middleware -> OpenShell
trailers_result {
trailer_mutations {
set { name: "x-review" value: "approved" on_existing: REPLACE_ALL }
}
trailer_mutations { remove { name: "digest" } }
reason_code: "trailers_updated"
}
An empty trailer_mutations list preserves all current trailers. This result is invalid because x-new-digest was absent from the event:
middleware -> OpenShell
trailers_result {
trailer_mutations {
set { name: "x-new-digest" value: "sha-256=:new:" }
}
}
OpenShell rejects the entire result and applies the stage's on_error. New trailer-name creation is deferred beyond V1.
block_delivery is a successful policy decision enforced regardless of on_error. The upstream request has already executed.
middleware -> OpenShell
preflight_result {
block_delivery {}
reason: "Secret material appeared in the upstream response"
reason_code: "response_secret_detected"
}
OpenShell -> sandbox
HTTP/1.1 403 Forbidden
Content-Type: application/json
Content-Length: <canonical JSON byte length>
Connection: close
{"error":"middleware_denied","reason_code":"response_secret_detected",...}
OpenShell does not return free-form reason. It stops later middleware evaluation and ends still-writable opened stages with MIDDLEWARE_DENIAL. An earlier handled failure still wins if it already stopped evaluation.
For a HEAD request, OpenShell sends the same canonical headers and content length but omits the JSON body.
OpenShell -> middleware
body { sequence: 8 data: "secret suffix" end_of_stream: true }
middleware -> OpenShell
body_result {
sequence: 8
block_delivery {}
reason_code: "late_secret_detected"
}
OpenShell aborts downstream delivery. It does not add an error body, terminating chunk, or error trailer, and it does not reuse the upstream connection. The sandbox may already have received a prefix.
Diagnostics are envelope fields for every action. If reason_code exceeds 64 bytes, a finding exceeds its limit, or metadata is invalid, OpenShell rejects the entire result and follows on_error without applying the selected action.
OpenShell -> middleware
body { sequence: 2 data: "second unit" }
middleware -> OpenShell
body_result { sequence: 1 pass_through {} }
OpenShell -> middleware, when still writable
session_end {
reason: MIDDLEWARE_SESSION_END_REASON_PROTOCOL_ERROR
protocol_error { middleware_exchange {} }
}
# The sandbox peer disconnects during delivery.
session_end { reason: MIDDLEWARE_SESSION_END_REASON_DOWNSTREAM_DISCONNECT }
# Upstream disconnects after a valid response or upgrade.
session_end { reason: MIDDLEWARE_SESSION_END_REASON_UPSTREAM_DISCONNECT }
UPSTREAM_FAILURE is reserved for rejection or failure before a valid response or upgrade.
The middleware RPC timeout and 30-second per-unit chain cap do not cover reading a complete upstream body. The contract therefore does not name a whole_body_accumulation_timeout yet. Before the response dispatcher lands, OpenShell needs a dedicated platform setting and default with a total, non-resetting deadline. STREAM_BYTES has no total response-lifetime deadline.
A non-OK gRPC status, an early response-stream close, a missing result, or a result deadline failure follows on_error. OpenShell may be unable to send session_end because the middleware transport is already closed. Middleware should release state when either session_end arrives or the inbound request stream ends.
- Treat the first event, missing oneofs, sequence gaps, duplicate preflight, invalid diagnostics, and unexpected event order as invalid input.
- Return one result for preflight, every body unit including the final unit, and every trailer event. Return none for
session_end. - Branch on explicit actions and terminal reasons. Do not infer actions from optional data.
- Select only a mode listed in this stage's
permitted_body_modes. - Treat every
STREAM_BYTESunit independently. Return its complete pass-through or replacement output before accepting the next unit. - Expect trailers after a normally completed body inspection, even if the set is empty. Do not create a trailer name absent from the event.
- Release state when
session_endarrives or the request stream ends. Do not assume a final body unit will precede every terminal event. - Keep diagnostics bounded and free of response secrets. OpenShell returns
reason_codeto the sandbox only for accepted pre-commit block delivery. - Do not rely on body-unit boundaries for application framing.
- Serve this service alongside
SupervisorMiddleware, whose manifest advertisesHTTP_RESPONSE/PRE_RETURN.