Skip to content

Instantly share code, notes, and snippets.

@pimlock
Last active September 19, 2026 00:59
Show Gist options
  • Select an option

  • Save pimlock/5d67d4452d44ef2a9523ed295e96f87d to your computer and use it in GitHub Desktop.

Select an option

Save pimlock/5d67d4452d44ef2a9523ed295e96f87d to your computer and use it in GitHub Desktop.
OpenShell two-mode HTTP middleware: lifecycle and failure state diagrams

Two-mode HTTP middleware: state diagrams

Companions: proto and exchanges and interactive walkthrough.

New proposal: BUFFERED and STREAM, all failures closed. No chunk IDs, ownership ACKs, prefix commits, recovery windows or midstream skip_remaining. Preflight Continue remains a successful terminal decision. OpenShell never spools data.

Each diagram follows one stage unless noted. “Release” means making output available to the next stage; later stages/policy checks may still prevent network delivery. The recipient is upstream for requests and the sandbox for responses. These diagrams describe a proposal, not the currently implemented runner.

1. Admission and preflight

stateDiagram-v2
    [*] --> AwaitDecision: O sends Preflight with eligible supported modes
    AwaitDecision --> Done: Continue with valid allowed head changes
    AwaitDecision --> Gathering: Inspect BUFFERED accepted then Begin
    AwaitDecision --> Streaming: Inspect STREAM accepted then Begin
    AwaitDecision --> Denied: explicit Reject
    AwaitDecision --> Failed: timeout or unavailable service or invalid decision
    Done --> [*]: body bypasses only this stage
    Denied --> [*]: local denial
    Failed --> [*]: local error
Loading

Continue has no body events and no Finish. Preflight failure never means Continue. An unoffered mode or unknown result variant is invalid, never a silent fallback.

2. Small whole-body redaction: BUFFERED

stateDiagram-v2
    [*] --> Gathering: valid Inspect BUFFERED then Begin
    Gathering --> Gathering: collect bounded input in OpenShell RAM
    Gathering --> AwaitResult: EOF then BufferedBody with trailers
    AwaitResult --> Validating: BufferedResult
    Validating --> Done: valid selected body and allowed mutations
    Gathering --> Failed: capacity exceeded or input failure
    AwaitResult --> Failed: timeout or middleware failure
    Validating --> Failed: invalid replacement or mutations
    AwaitResult --> Denied: Reject
    Failed --> [*]: reject and release buffers
    Denied --> [*]: reject and release buffers
    Done --> [*]: release head and selected body when chain permits
Loading

OpenShell withholds the head, supplies one complete body and validates one terminal result. Unchanged reuses the input only on success. Failure never forwards the original as a fallback. Overflow never spills to disk. Empty input still produces one BufferedBody. InputEnd, OutputStart and Finish are not used.

3. General duplex lifecycle: STREAM

stateDiagram-v2
    [*] --> InputOpenHeadHeld: valid Inspect STREAM then Begin
    InputOpenHeadHeld --> InputOpenHeadHeld: consume InputChunk
    InputOpenHeadHeld --> InputOpenOutputOpen: OutputStart freezes head
    InputOpenHeadHeld --> InputEndedHeadHeld: consume InputEnd
    InputOpenOutputOpen --> InputOpenOutputOpen: consume InputChunk OR emit OutputChunk
    InputOpenOutputOpen --> InputEndedOutputOpen: consume InputEnd
    InputEndedHeadHeld --> InputEndedOutputOpen: OutputStart freezes head
    InputEndedOutputOpen --> InputEndedOutputOpen: emit OutputChunk
    InputEndedOutputOpen --> Done: Finish after all selected output
    InputOpenHeadHeld --> Failed: error or unexpected event
    InputOpenOutputOpen --> Failed: error or premature Finish
    InputEndedHeadHeld --> Failed: error or premature Finish
    InputEndedOutputOpen --> Failed: error or length mismatch
    Failed --> [*]: fail closed
    Done --> [*]
Loading

OutputStart may precede the first input chunk. OutputChunk may not precede OutputStart. Finish must follow consumption of InputEnd. Input and output are independently serviced; bounded queues provide backpressure. Middleware owns any window or spool. No input/output correspondence or acknowledgements are tracked.

4. Windowed redaction inside STREAM

These states show middleware storage. OpenShell does not keep a recovery copy.

stateDiagram-v2
    state "Middleware window empty" as Empty
    state "Middleware holds se" as Prefix
    state "Middleware holds secret" as Secret
    state "Middleware holds secretok" as Lookahead
    state "Middleware holds ok" as Tail
    [*] --> Empty: Begin then OutputStart
    Empty --> Prefix: InputChunk se
    Prefix --> Secret: InputChunk cret
    Secret --> Lookahead: InputChunk ok
    Lookahead --> Tail: OutputChunk with whole redacted replacement
    Tail --> Empty: InputEnd then OutputChunk ok
    Empty --> [*]: Finish after EOF
Loading

The replacement is the entire [redacted] string in one message. OpenShell forwards it when the next stage can consume it; no commit event is involved. Middleware can drop its resolved prefix once it hands off that output. If it fails while holding unresolved bytes, delivery fails rather than recovering them.

Five-chunk lookahead uses the same mode: middleware holds A–E, reads F, emits processed A and retains B–F. That briefly needs six chunks of storage. A strict five-chunk cap requires releasing before reading F. No wire change is needed.

5. Sign an unchanged whole body: STREAM

stateDiagram-v2
    [*] --> HashAndStore: Begin
    HashAndStore --> HashAndStore: consume InputChunk into middleware spool
    HashAndStore --> Sign: consume InputEnd
    Sign --> Returning: OutputStart with authorized signature header
    Returning --> Returning: OutputChunk from middleware spool
    Returning --> Done: Finish and output length valid
    HashAndStore --> Failed: input or storage failure
    Sign --> Failed: signing failure
    Returning --> Failed: output or transport failure
    Failed --> [*]: fail closed
    Done --> [*]: middleware cleans its storage
Loading

Waiting for EOF is the signer's algorithm choice, not a STREAM restriction. The public hook remains credential-blind; this is the trusted internal signer using the same lifecycle. Middleware returns all body bytes even when unchanged. Output starts after the final head. Output failures may leave a partial upload.

6. Whole-body transformation: STREAM

stateDiagram-v2
    [*] --> Store: Begin
    Store --> Store: consume InputChunk
    Store --> Transform: InputEnd
    Transform --> Return: OutputStart with final allowed headers
    Return --> Return: OutputChunk from replacement storage
    Return --> Done: Finish
    Store --> Failed: failure
    Transform --> Failed: failure or Reject
    Return --> Failed: failure or Reject
    Failed --> [*]: fail closed
    Done --> [*]
Loading

Middleware owns both input and replacement storage. Output chunking/length can differ arbitrarily within limits. Experimental Git push rewriting fits this transport shape but still needs separate key, object-validation and isolation decisions. Signatures must cover the final representation sent upstream.

7. Response support and downstream gates

stateDiagram-v2
    [*] --> Eligibility: final upstream response head available
    Eligibility --> Preflight: offer implemented eligible modes only
    Preflight --> Done: Continue
    Preflight --> Buffered: select offered BUFFERED
    Preflight --> Stream: select offered STREAM
    Preflight --> Failed: select unoffered mode or invalid decision
    Buffered --> Done: accept terminal BufferedResult
    Stream --> Done: consume InputEnd then accept Finish
    Failed --> [*]: fail closed
    Done --> [*]: stage complete
Loading

STREAM can release bytes before its Done state. A later BUFFERED stage retains them in its own bounded RAM and holds the outgoing head until its own result. Completing the first stage does not release the second gate automatically. Initial response support may omit STREAM until the new duplex runner exists. No failure or denial reverses side effects of an already-executed upstream request.

8. One failure rule

stateDiagram-v2
    [*] --> Stop: middleware error or explicit Reject
    Stop --> LocalError: outgoing head not yet committed
    Stop --> AbortDelivery: outgoing head already committed
    LocalError --> [*]: direction-appropriate error or denial
    AbortDelivery --> [*]: terminate partial delivery
Loading

No path forwards retained originals on failure. No automatic retry or replay. Body-aware policy checks remain mandatory before downstream exposure; select bounded validation or reject incompatible streaming instead of bypassing checks.

Future extensions and history

Midstream skip_remaining needs a handoff for in-flight bytes and is deferred. Middleware may stop expensive inspection and echo the remaining stream itself. A future recoverable lockstep mode must negotiate retention at preflight; a new oneof alternative alone does not make older peers compatible. Such a mode would not retroactively change STREAM's failure contract.

Earlier alternatives, preserved unchanged:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment