Skip to content

Instantly share code, notes, and snippets.

@pimlock
Last active September 4, 2026 04:58
Show Gist options
  • Select an option

  • Save pimlock/236d4887f734071ebcfd0b7f7fc7c2d0 to your computer and use it in GitHub Desktop.

Select an option

Save pimlock/236d4887f734071ebcfd0b7f7fc7c2d0 to your computer and use it in GitHub Desktop.
OpenShell HTTP response pre-return middleware API interaction examples

OpenShell HTTP response pre-return middleware examples

This document illustrates the API proposed in NVIDIA/OpenShell#3073. The protobuf contract is authoritative. The examples use protobuf text-style objects to show event order without tying the discussion to one generated API.

HttpResponsePreReturn.Evaluate is a bidirectional gRPC stream for one HTTP response and one middleware stage. OpenShell writes HttpResponseEvent messages. The middleware writes matching HttpResponseEventResult messages.

Where the hooks run

HTTP_REQUEST/PRE_CREDENTIALS runs after network policy admits the destination and before OpenShell injects credentials. HTTP_RESPONSE/PRE_RETURN starts after OpenShell receives the final upstream response head and before it returns that response to the sandbox. Both evaluations carry the same context.request_id.

sequenceDiagram
    participant C as Sandbox client
    participant S as OpenShell supervisor
    participant U as Upstream
    participant M as Response middleware stage

    C->>S: HTTP request
    S->>U: Forward admitted request
    U-->>S: Final response head
    S->>M: preflight with stage-local permitted_body_modes
    M-->>S: preflight_result with skip, inspect, or block_delivery
    loop Body units for an inspecting stage
        S->>M: body with sequence and optional end_of_stream
        M-->>S: body_result with the matching sequence
    end
    opt Stage completed body inspection normally
        S->>M: trailers, including an empty set
        M-->>S: trailers_result
    end
    S-->>M: session_end
Loading

Interim 1xx responses pass through unchanged before the hook starts. 101 upgrades, successful CONNECT tunnels, and post-upgrade traffic bypass generic response processing.

Stream rules

  • OpenShell starts with one preflight. It includes the current head, request context, effective payload limit, and modes available to this stage.
  • The middleware returns one preflight_result with skip, inspect, or block_delivery.
  • OpenShell derives response-shape eligibility from the original final head, then applies each stage's payload limit. Stages may receive different permitted_body_modes lists.
  • skip, block_delivery, and inspect { body_mode: HEADERS_ONLY } receive no body or trailer events.
  • WHOLE_BODY_BYTES produces one body unit with end_of_stream: true, including a present empty sequence-1 unit for an empty body-capable response.
  • A normally completed STREAM_BYTES stage receives one or more units. Exactly one has end_of_stream: true; it may be empty when OpenShell learns about EOF after sending the last data.
  • Streaming inputs are at most min(64 KiB, max_payload_bytes). Each replacement may use up to max_payload_bytes.
  • STREAM_BYTES is unit-local in V1. Every result accounts for its complete input unit; middleware cannot retain input across units.
  • Every body unit requires one lockstep result with the matching sequence. pass_through preserves it, transform replaces it, block_delivery stops delivery, and skip_remaining finalizes it before opting this stage out of the suffix.
  • A stage that finishes body inspection normally receives one trailers event after its final body result, including when the set is empty, and returns one trailers_result.
  • Trailer mutations may modify or remove names in the event. V1 cannot create a missing trailer name.
  • reason, reason_code, findings, and metadata sit beside the selected action on every result envelope. Invalid diagnostics invalidate the complete result and follow on_error.
  • OpenShell may finish an opened stream with one best-effort session_end. It requires no result.

Example 1: whole-body transform followed by empty trailers

OpenShell -> middleware
preflight {
  context { request_id: "req-7f3" sandbox_id: "sandbox-42" }
  target { method: "GET" host: "api.example.com" path: "/profile" }
  status_code: 200
  headers { name: "content-type" value: "application/json" }
  headers { name: "content-length" value: "31" }
  headers { name: "etag" value: "upstream-version" }
  middleware_name: "response-redactor"
  max_payload_bytes: 4194304
  permitted_body_modes: HTTP_RESPONSE_BODY_MODE_HEADERS_ONLY
  permitted_body_modes: HTTP_RESPONSE_BODY_MODE_WHOLE_BODY_BYTES
  permitted_body_modes: HTTP_RESPONSE_BODY_MODE_STREAM_BYTES
}

middleware -> OpenShell
preflight_result {
  inspect {
    body_mode: HTTP_RESPONSE_BODY_MODE_WHOLE_BODY_BYTES
    header_mutations { remove { name: "etag" } }
  }
  reason_code: "json_response_selected"
  metadata { key: "rule_set" value: "customer-data-v3" }
}

OpenShell -> middleware
body {
  sequence: 1
  data: "{\"name\":\"Ada\",\"token\":\"secret\"}"
  end_of_stream: true
}

middleware -> OpenShell
body_result {
  sequence: 1
  transform { data: "{\"name\":\"Ada\",\"token\":\"[redacted]\"}" }
  reason_code: "sensitive_value_redacted"
}

OpenShell -> middleware
trailers {}

middleware -> OpenShell
trailers_result {
  reason_code: "no_upstream_trailers"
}

OpenShell -> middleware
session_end { reason: MIDDLEWARE_SESSION_END_REASON_NORMAL }

The empty trailer event and empty mutation list are explicit. No trailer names are created. For an empty body-capable response, OpenShell still sends sequence 1 with present empty data and end_of_stream: true; middleware may replace it with nonempty bytes.

Example 2: body modes differ by stage limit

The original response has Content-Length: 98304. Both stages may stream it, but only the stage with the larger limit may select whole-body mode.

OpenShell -> stage-a
preflight {
  context { request_id: "req-mixed" }
  status_code: 200
  headers { name: "content-length" value: "98304" }
  middleware_name: "large-document-scanner"
  max_payload_bytes: 131072
  permitted_body_modes: HTTP_RESPONSE_BODY_MODE_HEADERS_ONLY
  permitted_body_modes: HTTP_RESPONSE_BODY_MODE_WHOLE_BODY_BYTES
  permitted_body_modes: HTTP_RESPONSE_BODY_MODE_STREAM_BYTES
}

OpenShell -> stage-b
preflight {
  context { request_id: "req-mixed" }
  status_code: 200
  headers { name: "content-length" value: "98304" }
  middleware_name: "small-window-redactor"
  max_payload_bytes: 65536
  permitted_body_modes: HTTP_RESPONSE_BODY_MODE_HEADERS_ONLY
  permitted_body_modes: HTTP_RESPONSE_BODY_MODE_STREAM_BYTES
}

The original response head defines shape eligibility. Each stage's effective limit then narrows its list. For an eligible response without a known length, a stage may select WHOLE_BODY_BYTES and later receive whole_body_over_capacity through its on_error behavior.

Example 3: unit-local streaming transform

Each streaming result accounts for exactly one input unit. This stage transforms two independently meaningful event units and does not retain bytes between them.

OpenShell -> middleware
preflight {
  context { request_id: "req-814" }
  target { method: "GET" host: "events.example.com" path: "/events" }
  status_code: 200
  headers { name: "content-type" value: "text/event-stream" }
  middleware_name: "stream-scrubber"
  max_payload_bytes: 65536
  permitted_body_modes: HTTP_RESPONSE_BODY_MODE_HEADERS_ONLY
  permitted_body_modes: HTTP_RESPONSE_BODY_MODE_STREAM_BYTES
}

middleware -> OpenShell
preflight_result {
  inspect { body_mode: HTTP_RESPONSE_BODY_MODE_STREAM_BYTES }
  reason_code: "stream_selected"
}

OpenShell -> middleware
body { sequence: 1 data: "data: status=ready\n\n" }

middleware -> OpenShell
body_result { sequence: 1 pass_through {} }

OpenShell -> middleware
body { sequence: 2 data: "data: token=secret\n\n" }

middleware -> OpenShell
body_result {
  sequence: 2
  transform { data: "data: token=[redacted]\n\n" }
  reason_code: "token_redacted"
}

# EOF was learned after sequence 2, so OpenShell sends an empty final unit.
OpenShell -> middleware
body { sequence: 3 data: "" end_of_stream: true }

middleware -> OpenShell
body_result { sequence: 3 pass_through {} }

OpenShell -> middleware
trailers {}

middleware -> OpenShell
trailers_result {}

The stage cannot withhold a prefix from one unit and complete it from a later unit. Middleware that needs bounded cross-boundary processing must select WHOLE_BODY_BYTES. Cross-unit streaming requires a future explicit hold or ownership operation where OpenShell retains the original bytes, so fail-open recovery remains safe.

Example 4: stop inspecting after a prefix

OpenShell -> prefix-inspector
body { sequence: 1 data: "{\"kind\":\"public-catalog\",\"items\":[..." }

prefix-inspector -> OpenShell
body_result {
  sequence: 1
  skip_remaining { pass_through {} }
  reason_code: "public_document"
}

# This stage receives no later body or trailer events. The current output and
# units 2..n continue through every later stage in the chain.
OpenShell -> prefix-inspector
session_end { reason: MIDDLEWARE_SESSION_END_REASON_NORMAL }

skip_remaining.transform replaces only the current input unit before the stage opts out.

Example 5: trailer pass-through, mutation, and rejection

Repeated trailer names remain separate and ordered.

OpenShell -> middleware
trailers {
  headers { name: "digest" value: "sha-256=:old:" }
  headers { name: "x-review" value: "first" }
  headers { name: "x-review" value: "second" }
}

middleware -> OpenShell
trailers_result {
  trailer_mutations {
    set { name: "x-review" value: "approved" on_existing: REPLACE_ALL }
  }
  trailer_mutations { remove { name: "digest" } }
  reason_code: "trailers_updated"
}

An empty trailer_mutations list preserves all current trailers. This result is invalid because x-new-digest was absent from the event:

middleware -> OpenShell
trailers_result {
  trailer_mutations {
    set { name: "x-new-digest" value: "sha-256=:new:" }
  }
}

OpenShell rejects the entire result and applies the stage's on_error. New trailer-name creation is deferred beyond V1.

Example 6: block delivery

block_delivery is a successful policy decision enforced regardless of on_error. The upstream request has already executed.

Before commitment

middleware -> OpenShell
preflight_result {
  block_delivery {}
  reason: "Secret material appeared in the upstream response"
  reason_code: "response_secret_detected"
}

OpenShell -> sandbox
HTTP/1.1 403 Forbidden
Content-Type: application/json
Content-Length: <canonical JSON byte length>
Connection: close

{"error":"middleware_denied","reason_code":"response_secret_detected",...}

OpenShell does not return free-form reason. It stops later middleware evaluation and ends still-writable opened stages with MIDDLEWARE_DENIAL. An earlier handled failure still wins if it already stopped evaluation.

HEAD before commitment

For a HEAD request, OpenShell sends the same canonical headers and content length but omits the JSON body.

After commitment

OpenShell -> middleware
body { sequence: 8 data: "secret suffix" end_of_stream: true }

middleware -> OpenShell
body_result {
  sequence: 8
  block_delivery {}
  reason_code: "late_secret_detected"
}

OpenShell aborts downstream delivery. It does not add an error body, terminating chunk, or error trailer, and it does not reuse the upstream connection. The sandbox may already have received a prefix.

Failure scenarios

Invalid result diagnostics

Diagnostics are envelope fields for every action. If reason_code exceeds 64 bytes, a finding exceeds its limit, or metadata is invalid, OpenShell rejects the entire result and follows on_error without applying the selected action.

Invalid sequence

OpenShell -> middleware
body { sequence: 2 data: "second unit" }

middleware -> OpenShell
body_result { sequence: 1 pass_through {} }

OpenShell -> middleware, when still writable
session_end {
  reason: MIDDLEWARE_SESSION_END_REASON_PROTOCOL_ERROR
  protocol_error { middleware_exchange {} }
}

Directional disconnects

# The sandbox peer disconnects during delivery.
session_end { reason: MIDDLEWARE_SESSION_END_REASON_DOWNSTREAM_DISCONNECT }

# Upstream disconnects after a valid response or upgrade.
session_end { reason: MIDDLEWARE_SESSION_END_REASON_UPSTREAM_DISCONNECT }

UPSTREAM_FAILURE is reserved for rejection or failure before a valid response or upgrade.

Whole-body accumulation deadline

The middleware RPC timeout and 30-second per-unit chain cap do not cover reading a complete upstream body. The contract therefore does not name a whole_body_accumulation_timeout yet. Before the response dispatcher lands, OpenShell needs a dedicated platform setting and default with a total, non-resetting deadline. STREAM_BYTES has no total response-lifetime deadline.

Middleware RPC failure

A non-OK gRPC status, an early response-stream close, a missing result, or a result deadline failure follows on_error. OpenShell may be unable to send session_end because the middleware transport is already closed. Middleware should release state when either session_end arrives or the inbound request stream ends.

Middleware implementation checklist

  • Treat the first event, missing oneofs, sequence gaps, duplicate preflight, invalid diagnostics, and unexpected event order as invalid input.
  • Return one result for preflight, every body unit including the final unit, and every trailer event. Return none for session_end.
  • Branch on explicit actions and terminal reasons. Do not infer actions from optional data.
  • Select only a mode listed in this stage's permitted_body_modes.
  • Treat every STREAM_BYTES unit independently. Return its complete pass-through or replacement output before accepting the next unit.
  • Expect trailers after a normally completed body inspection, even if the set is empty. Do not create a trailer name absent from the event.
  • Release state when session_end arrives or the request stream ends. Do not assume a final body unit will precede every terminal event.
  • Keep diagnostics bounded and free of response secrets. OpenShell returns reason_code to the sandbox only for accepted pre-commit block delivery.
  • Do not rely on body-unit boundaries for application framing.
  • Serve this service alongside SupervisorMiddleware, whose manifest advertises HTTP_RESPONSE/PRE_RETURN.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment