Skip to content

RFC 0015: normalized provider→channel stream grammar#16

Open
Marvinthebored wants to merge 9 commits into
openclaw:mainfrom
Marvinthebored:0008-provider-stream-grammar
Open

RFC 0015: normalized provider→channel stream grammar#16
Marvinthebored wants to merge 9 commits into
openclaw:mainfrom
Marvinthebored:0008-provider-stream-grammar

Conversation

@Marvinthebored

@Marvinthebored Marvinthebored commented Jun 15, 2026

Copy link
Copy Markdown

Summary

Normalize every provider wire format into one event grammar — final text, thinking/reasoning, narration/commentary, tool activity, usage, errors — emit always, archive always, gate only at presentation. The RFC is the design narrative; the pinned companion spec is the contract; the vendored sidecar (rfcs/0015/agent-event-io-contract.md, from openclaw/openclaw#92216) is the base contract with four explicit [AMENDS BASE] points.

Since this was drafted (2026-06-15), the framework has been substantially ratified by landed commits: its core behaviors were implemented piecewise on main through maintainer-reviewed PRs. This RFC thus seeks to provide the canonical documentation for future or peripheral development, and the normative grammar for the parts not yet upstream.

Real behavior proof — the landed-implementation ledger

The proof for this contract is main itself. Each row is a merged, maintainer-reviewed PR implementing a slice of the grammar (each carries its own live proof in its PR thread):

PR Merged Contract slice
openclaw/openclaw#92216 2026-06-13 Base I/O contract behavior-half: gateway mirrors hidden commentary-phase events
openclaw/openclaw#93343 2026-06-16 Codex envelope: commentary dedup via stable-id idempotency
openclaw/openclaw#95283 2026-06-22 F2: reasoning/answer boundary seal under /reasoning on
openclaw/openclaw#97875 2026-06-30 Telegram: durable reasoning, presentation-gated
openclaw/openclaw#96106 2026-07-01 F1 Anthropic: pre-tool phase: commentary tagging at the parser, display gated downstream
openclaw/openclaw#98907 2026-07-03 Telegram: full /reasoning off|stream|on streamed-lane gate + core-owned durable commentary (closed openclaw/openclaw#90962)
openclaw/openclaw#99401 2026-07-04 F1e claude-cli thinking into the same gates

Update 2026-07-05: openclaw/openclaw#99401 has merged — every implementation slice in the ledger is now on main. Its merge also answers, in practice, the reasoning archival/privacy acceptance question for the CLI envelope: the same emit-to-bus / gate-at-presentation model the RFC's privacy-boundary section states.

Update 2026-07-06 (addressing the review's remaining decision points):

Update 2026-07-07 (docs repair per the review's two P2 findings): the vendored spec is now internally consistent with the RFC — base-contract reference repointed to the in-tree sidecar, stale "F5 catalogue pending" superseded, and the §7 tier taxonomy unified on the RFC's T0/T1/T2 table (legacy A→T1, B→T2, C→T2-with-declared-clamp, D→T0; truth table, worked projections, and glossary relabeled). The reasoning archival boundary is unchanged; the sync delta is recorded in the vendoring provenance header for auditability against the pinned original.

What changed in this revision (2026-07-04)

  • Renumbered 0008 → 00150008-context-engine-runtime-settings was accepted on main in the interim (review finding). File, sidecar dir, and all internal refs updated. (The head branch keeps its original name; branch names are cosmetic.)
  • Companion spec pinned at 0d354d9ed075 — the normative grammar no longer floats (review finding).
  • New "Reasoning privacy & archival boundary" section — the P1: explicit policy that archival is display-independent by design, no new retention/access/export surface, mirroring stays inside the gateway trust boundary, presentation gates are the only user-facing exposure decision. Stated so maintainers can ratify or amend it, not discover it.
  • Reference-impl section → implementation-status ledger (above); the original stacked drafts (pipeline: normalized provider→channel stream grammar (core) openclaw#93342, the Discord overlay) are demoted to lineage.
  • Unresolved questions refreshed — deepseek boundary (resolved by fix(openai-completions): seal native reasoning before the answer under /reasoning on openclaw#95283), F5 catalogue completeness, and the base-contract behavior-half moved to "Resolved since the initial draft"; still open: sidecar doc home, channel tier minimums, harness home.
  • Sidecar frontmatter moved to byte 0 (parser compatibility).

Marking ready for review: the remaining open questions are maintainer-preference items, not design blockers. Frontmatter stays status: draft per the template — acceptance is the maintainers' act at merge.

🤖 Generated with Claude Code

Emit always, archive always, gate only at presentation. Digests the companion
spec (openclaw-provider-stream-spec) and references the stacked reference-impl
PRs. Extends the agent event I/O contract (#92216) with four [AMENDS BASE]
points.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
@clawsweeper

clawsweeper Bot commented Jun 15, 2026

Copy link
Copy Markdown

Codex review: needs maintainer review before merge. Reviewed July 6, 2026, 8:30 PM ET / 00:30 UTC.

Summary
Adds RFC 0015 and two sidecar Markdown contracts for a normalized provider-to-channel stream grammar, reasoning archival and presentation gating, channel tiers, and conformance expectations.

Reproducibility: not applicable. This PR proposes RFC documentation rather than a current-main bug reproduction path.

Review metrics: 2 noteworthy metrics.

  • Diff Scope: 3 Markdown files added, 1330 additions. The PR is documentation-only but large enough that maintainers should review the normative sidecars, not only the RFC summary.
  • Prior Findings Checked: 2 previous P2 findings repaired. The current head addresses the previously reported spec-header and tier-taxonomy inconsistencies.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit ✨ media proof bonus
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • none.

Risk before merge

  • [P1] Merging this RFC ratifies that wire-carried reasoning may be archived independently of display settings, so the privacy and access-control boundary needs maintainer acceptance even though the diff is Markdown-only.
  • [P1] The RFC frontmatter remains status: draft with a blank implementation issue, while the repository lifecycle says accepted RFCs should update that metadata before merge.
  • [P1] The conformance harness and golden captures remain in the companion repository, so maintainers need to accept that split or request a final-home amendment.

Maintainer options:

  1. Ratify The Archival Boundary (recommended)
    If maintainers agree that raw reasoning belongs in the operator/audit archive behind presentation gates, update accepted metadata and merge.
  2. Amend The RFC Before Merge
    Ask for a narrow docs revision that changes the privacy boundary, final sidecar home, or harness ownership before acceptance.
  3. Pause The RFC
    Leave the PR open or close it if the normative contract is not ready to become an OpenClaw RFC.

Next step before merge

  • [P2] The remaining action is maintainer acceptance of the RFC’s normative security/privacy and document-location choices, not an automated repair.

Maintainer decision needed

  • Question: Should maintainers ratify RFC 0015's emit-always/archive-always/presentation-gated stream grammar, including raw reasoning archival and the current sidecar/harness split, and accept it for merge?
  • Rationale: The patch is coherent documentation, but accepting it sets normative security/privacy and future integration policy that automation should not decide.
  • Likely owner: ragesaq — They are tied to the base event I/O contract sidecar and adjacent accepted RFC work, making them the best available technical routing candidate.
  • Options:
    • Ratify RFC 0015 (recommended): Update the RFC metadata for acceptance and merge if maintainers agree with the raw-reasoning archive boundary and companion-harness split.
    • Amend Before Acceptance: Request targeted changes to the privacy/access-control wording, sidecar location, or harness ownership before merging.
    • Decline Core RFC: Close or pause this PR if OpenClaw should not make this provider-to-channel grammar a core RFC contract.

Security
Cleared: The diff is Markdown-only and introduces no executable code, dependencies, workflow permissions, or secret handling changes; the security-sensitive policy choice is tracked as merge risk.

Review details

Best possible solution:

Have maintainers explicitly ratify or amend the RFC’s raw-reasoning archival boundary and final document/harness home, then update acceptance metadata before merge.

Do we have a high-confidence way to reproduce the issue?

Not applicable: this PR proposes RFC documentation rather than a current-main bug reproduction path.

Is this the best way to solve the issue?

Unclear: the document is coherent and the prior docs consistency findings look repaired, but accepting the RFC’s security/privacy boundary and final sidecar/harness home is maintainer product judgment.

AGENTS.md: not found in the target repository.

Codex review notes: model internal, reasoning high; reviewed against a8e0ba2701d6.

Label changes

Label changes:

  • add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (linked_artifact): The PR body ties the RFC to merged implementation PRs and the branch vendors the normative text in-tree; no separate runtime proof is needed for the Markdown diff itself.
  • remove rating: 🦐 gold shrimp: Current PR rating is rating: 🐚 platinum hermit, so this older rating label is no longer current.
  • remove status: ⏳ waiting on author: Current PR status label is status: 👀 ready for maintainer look.

Label justifications:

  • P3: This is a draft RFC/documentation proposal with low immediate runtime impact.
  • merge-risk: 🚨 security-boundary: The RFC would ratify raw reasoning archival and presentation-gated exposure as a normative OpenClaw boundary.
  • merge-risk: 🚨 other: The final sidecar and harness ownership choices are maintainer policy decisions that CI cannot settle.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (linked_artifact): The PR body ties the RFC to merged implementation PRs and the branch vendors the normative text in-tree; no separate runtime proof is needed for the Markdown diff itself.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body ties the RFC to merged implementation PRs and the branch vendors the normative text in-tree; no separate runtime proof is needed for the Markdown diff itself.
Evidence reviewed

What I checked:

  • Target AGENTS policy: No AGENTS.md exists inside the target rfcs repository worktree; the AGENTS.md found one directory above is outside this git repository. (a8e0ba2701d6)
  • RFC lifecycle policy: The repository README says new RFCs should not merge while draft, and accepted RFCs should update status and issue metadata before merge. (README.md:80, a8e0ba2701d6)
  • PR diff scope: The merge diff adds three Markdown files and 1330 lines: the RFC plus two sidecar contract/spec files. (e34b6cf81dff)
  • Prior header finding repaired: The vendored stream-grammar spec now states the in-tree sidecar is normative and points its base contract reference to agent-event-io-contract.md. (rfcs/0015/stream-grammar-spec.md:17, e34b6cf81dff)
  • Prior tier taxonomy finding repaired: The vendored spec now uses T0/T1/T2 capability tiers and maps legacy A/B/C/D ids instead of leaving a competing taxonomy. (rfcs/0015/stream-grammar-spec.md:440, e34b6cf81dff)
  • Security boundary under review: The RFC explicitly states that reasoning is archived independently of display settings, with no new export surface and channel presentation gates as the user-facing exposure decision. (rfcs/0015-provider-stream-grammar.md:201, e34b6cf81dff)

Likely related people:

  • kevinlin-openai: Authored the RFC template and lifecycle guidance used to assess this PR's merge-readiness. (role: RFC process owner; confidence: high; commits: f4fdf38f4717, e366ea9825a4; files: README.md, rfcs/0000-template.md)
  • Dallin Romney: Clarified the sidecar-material layout that this PR uses for its two contract files. (role: sidecar layout contributor; confidence: medium; commits: 3aa7d727383f; files: README.md, rfcs/0000-template.md)
  • ragesaq: Authored the accepted Context Engine Runtime Settings RFC and contributed the base event I/O sidecar to this branch. (role: adjacent RFC author and sidecar contributor; confidence: medium; commits: 951ec46d8675, 7028a7cf2550; files: rfcs/0008-context-engine-runtime-settings.md, rfcs/0015/agent-event-io-contract.md)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
Review history (4 earlier review cycles)
  • reviewed 2026-07-04T02:59:48.765Z sha de854b2 :: needs maintainer review before merge. :: none
  • reviewed 2026-07-05T13:35:36.440Z sha 50ae61d :: needs maintainer review before merge. :: none
  • reviewed 2026-07-05T13:39:37.893Z sha 50ae61d :: needs maintainer review before merge. :: none
  • reviewed 2026-07-06T14:42:53.239Z sha e599900 :: needs changes before merge. :: [P2] Refresh the vendored spec header | [P2] Reconcile the channel tier taxonomy

@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. labels Jun 15, 2026
Add the agent event I/O contract that RFC 0008's [AMENDS BASE] points
extend, as the in-tree sidecar rfcs/0008/agent-event-io-contract.md, and
repoint the RFC's base-contract reference from the dead
docs/channels/agent-event-io-contract.md path to it.

The contract was extracted from openclaw/openclaw #92216; the upstream
maintainer asked that it be split out of that behavior PR so it could be
owned and edited separately from runtime code. This RFC sidecar is that
separately-ownable home, and gives the four [AMENDS BASE] points a real,
pinned target instead of a path that 404s on openclaw main.

Resolves the two base-contract review findings (wrong/missing base path;
normative contract living in a mutable external repo) for the base half.
@ragesaq

ragesaq commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

The base contract now has a home in the RFC tree (rfcs/0008/).

I collaborated with @Marvinthebored on the preamble/commentary normalization work behind this RFC, and we agreed the base contract it amends should live alongside the RFC so it can be owned and developed here instead of referenced from elsewhere.

That base contract is agent-event-io-contract.md. It was introduced in openclaw/openclaw#92216, then split out of that PR at maintainer request so it could be owned and edited separately from the behavior change. The RFC repo is exactly that separately-ownable home, so I've brought it back as an RFC sidecar.

The PR into this RFC's branch is Marvinthebored#1 (clean, mergeable). It does two things:

  1. Adds rfcs/0008/agent-event-io-contract.md: the full 349-line contract (provider input contract, gateway session-mirror contract, channel output contract, test contract, and the ClickClack regression example), matching the rfcs/0007/ sidecar precedent.
  2. Repoints the RFC's base reference from docs/channels/agent-event-io-contract.md (which 404s on current main) to the in-tree 0008/agent-event-io-contract.md.

This targets the two P2 findings directly:

  • [P2] base-contract reference (L31-32): the four [AMENDS BASE] points now resolve to a real file in-tree, not a path that 404s on main plus a PR that only touched src/gateway/server-chat.*.
  • [P2] pin the normative contract (L26-30): the base contract is now vendored in rfcs/0008/ rather than referenced from a mutable personal companion repo.

Two items I deliberately left to @Marvinthebored rather than guess: folding the four [AMENDS BASE] amendments into the contract text itself (the Unresolved-questions note is right that an implementer of the base alone shouldn't build a thinking-dropping gateway), and the P3 harness-count sync (RFC says 35; companion STATUS.md says 37/28). I staged the contract as the as-split baseline so the four deltas get reconciled together.

Marvinthebored and others added 2 commits June 16, 2026 01:15
rfc(0008): vendor agent event I/O contract as pinned base
…act; sync harness count

Addresses ragesaq's two follow-ups on #1 and clawsweeper's P3:
- Fold §3.2 (thinking emission unconditional, never dropped), §3.3 (item start at
  earliest id+name), §5.1 (thinking mirrored on its own stream, not as commentary),
  §7.3 (stable/composite idempotency ids, no synthesized counters) into
  rfcs/0008/agent-event-io-contract.md, each marked inline "[RFC 0008 amendment]"
  so the as-split #92216 baseline stays distinguishable.
- Repoint the RFC Unresolved note: amendments folded (no longer pending).
- Sync harness count 35 -> 37 tests / 28 goldens to match companion STATUS.md.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Jun 20, 2026
- Renumber file + sidecar dir + all internal refs (0008-context-engine-runtime-settings
  was accepted on main in the interim)
- Pin the companion spec repo at 0d354d9ed075
- Replace 'stacked draft PRs' reference-impl section with the landed-on-main
  implementation ledger (#92216 #93343 #95283 #97875 #96106 #98907, #99401 open)
- Add explicit 'Reasoning privacy & archival boundary' section (review P1)
- Move resolved unresolved-questions (deepseek boundary → #95283; F5 catalogue;
  base-contract behavior-half → #92216) to a 'Resolved since draft' list

Co-Authored-By: Claude Fable 5 <[email protected]>
@Marvinthebored Marvinthebored changed the title RFC 0008: normalized provider→channel stream grammar RFC 0015: normalized provider→channel stream grammar Jul 4, 2026
@Marvinthebored
Marvinthebored marked this pull request as ready for review July 4, 2026 02:52
@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jul 4, 2026
All seven cited implementation slices are now on main. The privacy-boundary
section's open acceptance question for the CLI envelope was answered by the
merge; restated as shipped policy awaiting RFC-level ratification.

Co-Authored-By: Claude Fable 5 <[email protected]>
@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Jul 5, 2026
… add Shipped: annotations + applying-guide

- Vendor stream-grammar-spec.md (from pinned companion @ 0d354d9ed075) — the
  normative text now lives in the RFC tree; companion keeps evidence + harness
- Answer the channel-tier-minimums unresolved question with a concrete
  proposed T0/T1/T2 table (Discord/Telegram = shipped T1 references)
- Per-clause 'Shipped:' annotations tie each Proposal section to its merged PR
- New 'Applying this RFC to future work' section (contributor checklist)
- Cite independently-reported issues closed by the landings (#96079, #90962)

Co-Authored-By: Claude Fable 5 <[email protected]>
@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Jul 6, 2026
Addresses the two P2 review findings:
- spec preamble: base-contract path repointed to the in-tree sidecar; stale
  'F5 catalogue pending' superseded (completed 2026-06-18)
- §7 tier taxonomy unified on the RFC's T0/T1/T2 (legacy A→T1, B→T2,
  C→T2-with-declared-clamp, D→T0; truth table, worked projections, glossary
  relabeled; the unrelated 'RATIFIED at T2' decision label disambiguated)
Reasoning archival boundary unchanged. Sync delta recorded in the vendoring
provenance header.

Co-Authored-By: Claude Fable 5 <[email protected]>
@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Jul 7, 2026
@clawsweeper

clawsweeper Bot commented Jul 16, 2026

Copy link
Copy Markdown

ClawSweeper status: review started.

I am starting a fresh review of this pull request: RFC 0015: normalized provider→channel stream grammar This is item 1/1 in the current shard. Shard 0/1.

This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking.

Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Telegram: inter-tool commentary clobbers tool progress in non-persist progress mode (diverges from other streaming channels)

3 participants