Skip to content

Instantly share code, notes, and snippets.

@pimlock
Last active September 19, 2026 00:59
Show Gist options
  • Select an option

  • Save pimlock/c33582aadad2e246d2538a710589b4cc to your computer and use it in GitHub Desktop.

Select an option

Save pimlock/c33582aadad2e246d2538a710589b4cc to your computer and use it in GitHub Desktop.
OpenShell two-mode HTTP middleware proposal: BUFFERED, STREAM, mandatory fail-closed

HTTP middleware: two body modes, fail closed

Companions: state diagrams and interactive walkthrough.

New design proposal following discussion of failure recovery and ownership. This is a standalone draft, not the currently implemented API. The accompanying http_middleware.proto defines a shared request/response contract with only BUFFERED and STREAM body modes.

Decision

All middleware failures fail closed. OpenShell never spools data. BUFFERED gathers one bounded body in RAM. STREAM sends input and receives independent output chunks; middleware owns its processing/storage and returns all selected output. There are no chunk IDs, ownership ACKs, prefix commits or replay copies in STREAM. It may emit output before input EOF or wait for the entire body.

Header-only acceptance is a successful preflight Continue decision, not a body mode. Midstream skip_remaining is deferred. Explicit Continue, Unchanged and Reject decisions are distinct from failures.

What the RFC says about fail-open

RFC 0009 does give fail-open a concrete use case: its optional trace-exporter example bypasses exporter failures while redaction stages fail closed. Its risks section explicitly recognizes that fail-closed middleware outages can become workload outages. Failure-policy alignment with gateway interceptors is another documented choice, not a streaming requirement. The RFC does not establish adoption or demonstrate a need for fail-open signing/redaction.

References in the RFC:

  • Lines 433–443: optional trace-exporter example.
  • Lines 506–508: availability tradeoff and owned processing.
  • Lines 329–335: shared failure-policy vocabulary with gateway interceptors.
  • Lines 139 and 559: post-transformation body-aware policy enforcement.

This proposal defers optional observer availability guarantees. Such needs may later justify a separate observer hook or a negotiated recoverable mode. Existing explicit fail_open configurations must be rejected with a migration error, not silently converted. The existing shared middleware policy also affects WebSocket hooks; production migration must decide that scope explicitly. Gateway interceptors have a separate contract and are not changed by this HTTP draft.

Mode and use-case selection

Use case Choice Where processing bytes live
Nothing to inspect after seeing headers Preflight Continue Body bypasses this stage
Small whole-body JSON redaction BUFFERED Bounded OpenShell RAM
Independent per-chunk transformation STREAM Middleware processes and emits immediately
Cross-boundary redaction with a 2 MiB working window STREAM Middleware owns the working window
Five-chunk lookahead STREAM Middleware keeps five, or transiently six before release
Signature header requiring the whole body hash STREAM Middleware owns the spool and delays OutputStart
Large whole-body rewrite, such as experimental Git push signing STREAM Middleware owns input/replacement storage

Window size is an implementation/configuration detail of middleware, not an OpenShell recovery allowance. OpenShell enforces its transport queue ceilings; it cannot directly enforce an external process's internal memory allocation.

Reading the exchanges

O is OpenShell; M is middleware. These are explanatory sketches, not protobuf TextFormat. Limits and headers are abbreviated. A Preflight Inspect decision selects exactly one offered mode. Begin follows chain admission. A stream result is forwarded to the next stage when capacity permits; upstream sees it only after all later gates and policy checks permit delivery. RPC framing defines order.

1. Preflight skip or header-only acceptance

O -> M  Preflight(head, headers, permitted_modes=[BUFFERED, STREAM], limits)
M -> O  PreflightResult(continue_without_body={}, header_changes=[allowed changes])
O       Validate changes; body bypasses this stage and continues to later stages.

This result is terminal for the stage. No Begin, body events or Finish follows. A preflight RPC failure rejects; it never becomes an implicit Continue. A service can instead return Reject with an audit-safe reason code.

2. Small whole-body redaction: BUFFERED

O -> M  Preflight(permitted_modes=[BUFFERED, STREAM], max_buffered_body_bytes=2097152)
M -> O  PreflightResult(inspect={buffered: {max_body_bytes: 2097152}})
O -> M  Begin
O       Hold outgoing head; collect the complete body in bounded RAM.
O -> M  BufferedBody(data='{"token":"example"}', visible_trailers=[])
M -> O  BufferedResult(replacement='{"token":"[redacted]"}')
O       Validate terminal result and framing; release selected head and body.

No InputChunk, InputEnd, OutputStart or Finish occurs. Input/replacement each fit the selected cap. Overflow rejects and never spills to disk or switches modes. BufferedResult(unchanged={}) can reuse the input on success; it is not a failure fallback. Header changes based on the full body require the offered late-header capability and existing header permissions. Count originals, replacements and serialization copies in memory admission.

3. Incremental transformation: STREAM

O -> M  Preflight(permitted_modes=[STREAM, ...], limits)
M -> O  PreflightResult(inspect={stream: {}})
O -> M  Begin
M -> O  OutputStart()                  // head is final; input may still be empty
O -> M  InputChunk(data="abc")
M -> O  OutputChunk(data="ABC")
O -> M  InputChunk(data="def")
M -> O  OutputChunk(data="DEF")
O -> M  InputEnd(visible_trailers=[])
M -> O  Finish()

This particular processor happens to emit one output per input. The protocol does not require that correspondence or wait for a result before sending the next input. There are no sequence numbers or ACKs. Bounded queues and independent input/output pumps supply backpressure. Finish must follow consumption of InputEnd.

4. Windowed redaction: also STREAM

O -> M  Preflight(...)
M -> O  PreflightResult(inspect={stream: {}})
O -> M  Begin
M -> O  OutputStart()
O -> M  InputChunk(data="se")
M       Keep "se" in its own working window; emit nothing yet.
O -> M  InputChunk(data="cret")
M       Window now contains "secret".
O -> M  InputChunk(data="ok")
M       Window now contains "secretok"; enough context to resolve the prefix.
M -> O  OutputChunk(data="[redacted]")
M       Release "secret" from its own window; keep "ok".
O       Pass the whole replacement onward. No commit event is needed.
O -> M  InputEnd(visible_trailers=[])
M -> O  OutputChunk(data="ok")
M -> O  Finish()

OpenShell never holds those originals for recovery. If middleware fails while holding them, delivery fails. The complete [redacted] replacement is one output message, assuming it fits the per-message cap.

For five-chunk lookahead, middleware can receive chunks A–E, receive F, emit the processed A and retain B–F. That temporarily holds six chunks. A strict five-chunk memory budget requires releasing before admitting F. Use byte limits as well as chunk counts in the implementation, because chunk boundaries/sizes are arbitrary. This storage discipline does not require changes to the wire protocol.

5. Whole-body signature header: STREAM

The trusted signer must finish the payload hash before constructing its final signature header. That algorithm chooses to wait for EOF; the mode does not require it. The public remote service cannot receive managed credentials or write Authorization. This example uses the trusted internal equivalent.

O -> M  Preflight(...)
M -> O  PreflightResult(inspect={stream: {}})
O -> M  Begin
O -> M  InputChunk(data=first bytes)
M       Hash and store input in middleware-owned spool.
        ... more input with bounded transport backpressure, no ACKs ...
O -> M  InputEnd(visible_trailers=[])
M       Finalize hash and authorized signature.
M -> O  OutputStart(header_changes=[authorized signature headers], output_body_bytes=B)
M -> O  OutputChunk(data=first bytes read from its spool)
        ... stream unchanged body back with arbitrary bounded chunking ...
M -> O  Finish()

OpenShell retains no replay copy and offers no stream pass-through reference. Middleware returns the whole body even if unchanged. No stage may subsequently alter signed content. OutputStart may precede input EOF for algorithms that do not need the entire input. See AWS signed-payload documentation.

6. Large whole-body rewrite: STREAM

Use the same exchange, but prepare replacement output in middleware-owned storage. OutputStart supplies its length when known; output length and chunking can differ from input. An experimental Git push signer could consume/spool a pack, validate and rewrite objects, then return the replacement push body. Commit-ID changes, key scope, helper networking and process isolation remain separate product/security decisions, not guarantees of the streaming protocol.

7. Response hook

Both services use exactly the same messages and modes. Each invocation advertises the supported eligible subset; shared schema does not silently enable behavior. Response implementation can initially offer BUFFERED and preflight Continue, then offer the new duplex STREAM only when its runtime supports it. The old lockstep response stream is not automatically the new contract.

O -> M  Preflight(response={status_code:200, request_method:"GET"},
                 permitted_modes=[BUFFERED], late_header_modes=[], ...)
M -> O  PreflightResult(inspect={buffered:{max_body_bytes:2097152}})
O -> M  Begin
O -> M  BufferedBody(data=eligible response body, visible_trailers=[])
M -> O  BufferedResult(replacement=redacted response)
O       Deliver selected response head/body to the sandbox.

Preserve bodyless, encoded, range and no-transform eligibility, protected header rules and trailer restrictions. Streaming responses can omit total-lifetime and total-byte limits while keeping bounded queues, idle handling and cancellation. Failure before response commitment can replace the response with an error; failure afterward aborts delivery. Neither undoes the upstream request.

Failure, policy and termination

  • Every middleware error fails closed, including startup unavailability, malformed results, timeouts and BUFFERED overflow. There is no on_error choice.
  • OutputStart freezes headers. A later failure aborts partial delivery if headers or body already went out. Fail-closed cannot retract previously sent bytes.
  • Finish is mandatory after InputEnd for STREAM, including empty bodies. Empty input uses InputEnd without InputChunk; empty output uses OutputStart and Finish. An empty output on nonempty input is deletion. RPC close is not success by itself.
  • Input and output pumps operate independently. Middleware needing the entire body must consume it into its own storage, rather than deadlock waiting for output-side progress. Transport/RPC queues must be bounded in practice.
  • Body-aware GraphQL/JSON-RPC/MCP policy checks may require full transformed input before further exposure. Keep bounded validation or omit incompatible STREAM capability; do not remove enforcement or add implicit spooling.
  • Late head changes must be validated before dependent downstream decisions. The whole chain may contain a BUFFERED stage that withholds upstream delivery.

skip_remaining and future recovery

Preflight Continue is safe because no body has been transferred. Midstream skip_remaining is deferred: stopping a duplex stage requires a handoff for bytes already sent or in flight. Middleware can instead stop costly inspection and echo the rest itself; this keeps the RPC hop but needs no protocol extension.

A future recoverable mode can be added as a new Inspect oneof member with its own input/result messages. It must be negotiated before body delivery and retain originals from the start. A matching pass/replace result can acknowledge each lockstep input; no separate ACK is necessarily needed. It cannot retroactively make STREAM recoverable. Earlier transformed output would normally remain.

Capability negotiation is required in addition to protobuf extensibility. Older peers may see an unknown oneof variant as unset; reject unsupported variants and never treat them as Continue or successful completion. See the protobuf guide.

Design history

The earlier alternatives remain unchanged for comparison:

Production migration must cover stored configs, SDKs, both HTTP hooks, shared WebSocket failure-policy scope, observability and removal of legacy recovery branches. This sketch omits detailed diagnostics/session-end structures; preserve or explicitly migrate them rather than implicitly deleting that functionality.

The standalone proto was validated with protoc. The interactive walkthrough was checked in Chromium across 9 scenarios and 120 frame/gate combinations, including desktop and mobile layout, buffer limits, header timing and failure paths.

// Design proposal: two body modes, mandatory fail-closed middleware.
// Standalone sketch, not a wire-compatible edit of the production proto.
// Field numbers are illustrative. Production integration must preserve context,
// configuration, diagnostics, authentication and session cancellation contracts.
syntax = "proto3";
package openshell.middleware.http.twomode.draft;
service HttpRequestPreCredentials {
rpc Evaluate(stream HttpEvent) returns (stream HttpResult);
}
service HttpResponsePreReturn {
rpc Evaluate(stream HttpEvent) returns (stream HttpResult);
}
message HttpEvent {
oneof event {
Preflight preflight = 1;
// Sent only after an Inspect plan is accepted and the chain is admitted.
Begin begin = 2;
// STREAM only. Ordered transport, no sequence IDs or per-chunk ACKs.
InputChunk input = 3;
InputEnd input_end = 4;
// BUFFERED only: replaces InputChunk and InputEnd entirely.
BufferedBody buffered_body = 5;
}
}
message HttpResult {
oneof event {
PreflightResult preflight_result = 1;
BufferedResult buffered_result = 2;
OutputStart output_start = 3;
OutputChunk output = 4;
Finish finish = 5;
// Explicit rejection is terminal at preflight or during body processing.
Reject reject = 6;
}
}
enum BodyMode {
BODY_MODE_UNSPECIFIED = 0;
BUFFERED = 1;
STREAM = 2;
}
message Header {
string name = 1;
bytes value = 2;
}
// Placeholder for the existing ordered HeaderMutation contract.
message HeaderChange {
string name = 1;
oneof operation {
bytes set_value = 2;
// If selected, must be true.
bool remove = 3;
}
}
message Preflight {
oneof head {
RequestHead request = 1;
ResponseHead response = 2;
}
repeated Header visible_headers = 3;
Limits limits = 4;
// Intersection of runtime support, service capabilities, policy and message
// eligibility. Empty allows only preflight Continue or Reject.
repeated BodyMode permitted_modes = 5;
optional uint64 declared_input_bytes = 6;
// Subset of permitted_modes allowed to return later changes to otherwise
// mutable headers. Never grants credentials or protected-header authority.
// May initially be empty on the response hook to preserve its current rules.
repeated BodyMode late_header_modes = 7;
}
message RequestHead {
string method = 1;
// Production integration uses the existing admitted target and context.
string admitted_target = 2;
}
message ResponseHead {
uint32 status_code = 1;
string request_method = 2;
}
// Applicable limits must be positive; absent optional limits mean no additional
// total/lifetime cap, not unbounded queues. Registration and policy own ceilings.
// Input/output queues include RPC adapter queues, not only application channels.
message Limits {
uint64 max_chunk_bytes = 1;
uint64 max_buffered_body_bytes = 2;
uint64 max_input_queue_bytes = 3;
uint64 max_input_queue_messages = 4;
uint64 max_output_queue_bytes = 5;
uint64 max_output_queue_messages = 6;
optional uint64 max_total_input_bytes = 7;
optional uint64 max_total_output_bytes = 8;
uint64 idle_timeout_ms = 9;
optional uint64 session_timeout_ms = 10;
}
message PreflightResult {
// Exactly one. No body delivery has occurred when this decision is made.
oneof decision {
Continue continue_without_body = 1;
Inspect inspect = 2;
}
// Preflight head changes are validated before the next stage uses the head.
repeated HeaderChange header_changes = 3;
}
// Successful decision, not failure recovery. Terminal for this stage. No Begin,
// body events or Finish follow; body bypasses this stage but not later stages.
message Continue {}
message Inspect {
oneof mode {
BufferedMode buffered = 1;
StreamMode stream = 2;
// Future negotiated modes may be added with new field numbers. Existing
// STREAM semantics must never acquire an implicit recovery guarantee.
}
}
message BufferedMode {
// Positive and <= offered max_buffered_body_bytes. Applies separately to
// input and replacement. Input overflow rejects; no fallback or disk spill.
uint64 max_body_bytes = 1;
}
// Middleware assumes responsibility for all output at accepted preflight.
// OpenShell retains no recovery originals. Middleware may use an internal
// window, process immediately, or spool on its own. No ownership ACK exists.
message StreamMode {}
message Begin {}
message BufferedBody {
// One complete normalized body, including empty body, gathered in bounded RAM.
bytes data = 1;
repeated Header visible_trailers = 2;
}
// BUFFERED terminal result. Validate all fields before releasing selected output.
// No InputEnd, OutputStart, OutputChunk or Finish occurs in this mode.
message BufferedResult {
oneof body {
// Reuses the retained buffered input. This is successful pass-through, not
// a fail-open option; failure never falls back to the input.
Unchanged unchanged = 1;
// Present empty bytes delete the entire body.
bytes replacement = 2;
}
// Must be empty unless BUFFERED is offered in late_header_modes.
repeated HeaderChange header_changes = 3;
repeated HeaderChange trailer_changes = 4;
}
message Unchanged {}
message InputChunk {
// Nonempty, <= max_chunk_bytes. No application/transport boundary meaning.
bytes data = 1;
}
message InputEnd {
// Exactly once after input, including for empty input. No input follows.
repeated Header visible_trailers = 1;
}
// STREAM: exactly once after Begin, before any output. May precede input EOF
// or even the first InputChunk. Freezes the output head. Does not stop input.
message OutputStart {
// Must be empty unless STREAM is offered in late_header_modes. Protected
// fields stay forbidden remotely; trusted signers use internal authority.
repeated HeaderChange header_changes = 1;
// If present, OpenShell derives framing and checks the actual output length.
// Otherwise it chooses streaming framing. Raw framing writes are forbidden.
optional uint64 output_body_bytes = 2;
}
message OutputChunk {
// Nonempty, <= max_chunk_bytes. Arbitrary input/output cardinality. OpenShell
// validates and forwards each message with backpressure, without prefix commits.
bytes data = 1;
}
// STREAM terminal result after middleware consumes InputEnd and sends all output.
// OutputStart is mandatory, including for empty output. No later result follows.
// EOF on the RPC does not substitute for Finish. No repeated sequence counters.
message Finish {
repeated HeaderChange trailer_changes = 1;
}
message Reject {
// Validated audit-safe code, never raw content or diagnostics.
string reason_code = 1;
}
// Runtime invariants:
// 1. All middleware failures are fail closed, including preflight, availability,
// timeout, capacity and protocol errors. Reject on_error=fail_open config;
// do not silently reinterpret it. Explicit Continue/Unchanged are successes.
// 2. OpenShell never spools data. BUFFERED gathers bounded input; STREAM has only
// bounded transport queues. Per-message, aggregate-memory and chain admission
// limits account for originals, output, serialization copies and concurrency.
// 3. Input and output pumps run independently. Input backpressure must not block
// reading output. Bounded queues/transport flow control, not ACKs, limit memory.
// Flow control may pause input; it never implies ownership or byte recovery.
// 4. Finishing before consuming InputEnd is invalid. There is no midstream
// skip_remaining or implicit input cancellation. Middleware may stop expensive
// inspection and echo the remaining input itself, retaining the RPC path.
// 5. Head release is irreversible. Before network commitment, failure produces
// a local error; afterward, abort partial delivery. No fail-open, replay or
// automatic retry. Other stages may still gate output before network contact.
// 6. BUFFERED holds its head until its result; STREAM until OutputStart. Input
// must continue feeding a downstream whole-body stage while it holds the head.
// Later decisions must use final validated headers from earlier stages.
// 7. Preserve body-aware policy re-evaluation BEFORE downstream exposure. If
// full-body checks are required, use bounded validation or omit STREAM; never
// forward early or spool implicitly to bypass that requirement.
// 8. Same body types on both hooks, independently negotiated support/eligibility.
// Preserve response bodyless/encoded/range/no-transform restrictions, header
// and trailer authority, credential isolation and generation cancellation.
// 9. Future recoverable modes require capability negotiation and retention from
// preflight. Unknown/unset event alternatives fail validation. A new oneof
// member alone does not make older peers understand the protocol.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment