Skip to content

test(slack): delivery trace goldens for buffered native streaming and block finals#104583

Merged
steipete merged 1 commit into
mainfrom
claude/chanref-p0-traces-slack
Jul 11, 2026
Merged

test(slack): delivery trace goldens for buffered native streaming and block finals#104583
steipete merged 1 commit into
mainfrom
claude/chanref-p0-traces-slack

Conversation

@steipete

Copy link
Copy Markdown
Contributor

Related: #104318

What Problem This Solves

Extends the delivery trace harness (#104478) to Slack — the buffered-ack native-streaming channel whose accepted-vs-visible semantics and stop-as-first-network-call fallback are core inputs to v2's tri-state write-ack design.

Why This Change Was Made

Five golden scenarios drive real dispatchPreparedSlackMessage wiring with a recording WebClient and the real @slack/web-api ChatStreamer: native streaming (tool payload accepted with zero wire traffic under the 256-char SDK buffer; visibility landing at stream start); stop-as-first-network-call (benign rejection → durable full-text postMessage fallback, proving the SDK private-buffer reach-in); blocks+fallback-text final duality; cancel (draft deleted); and documentation that the custom-identity preview-edit tier is structurally unreachable (draft streaming disabled when identity is set because chat.update cannot preserve authorship) — direct evidence for v2's contextual capability predicates. All ts values normalized.

User Impact

None at runtime — test infrastructure.

Evidence

  • Testbox tbx_01kx92cngs8qpp2gm79gdrrtam: full slack suite (118 files / 1822 tests) + pnpm check:changed green; trace-only tbx_01kx92s0dfvmwtg89scw3hhhkg 5/5; byte-stable across macOS record + double verify + Linux Node 24 verify.
  • Autoreview (codex gpt-5.6-sol, xhigh): clean first pass (0.91).
  • Known sensitivity (intentional): goldens pin the SDK's buffer_size=256 default — an @slack/web-api bump changing buffering semantics will correctly drift traces.

@openclaw-barnacle openclaw-barnacle Bot added channel: slack Channel integration: slack size: L maintainer Maintainer-authored PR labels Jul 11, 2026
@clawsweeper clawsweeper Bot added rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. labels Jul 11, 2026
@clawsweeper

clawsweeper Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed July 11, 2026, 1:26 PM ET / 17:26 UTC.

Summary
Adds five Slack delivery-trace golden scenarios and a production-seam test harness for native streaming, first-call fallback, block finals, cancellation, and edit-preview finalization.

PR surface: Source +45, Tests +637. Total +682 across 6 files.

Reproducibility: not applicable. This PR characterizes existing Slack delivery behavior rather than reporting a failing user-visible bug. The scenarios are source-verifiable against current production branches and the pinned Slack SDK.

Review metrics: none identified.

Root-cause cluster
Relationship: partial_overlap
Canonical: #104318
Summary: This PR is a bounded Phase 0 behavior-lock contribution to the channel refactor umbrella and extends the generic harness introduced by a merged predecessor PR.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Next step before merge

  • [P1] No repair is needed; the remaining actions are ordinary protected maintainer review, branch refresh, and exact-head landing gates.

Security
Cleared: The patch changes only Slack test code and committed JSONL fixtures, with no production execution, dependencies, workflows, permissions, secrets, or package-resolution changes.

Review details

Best possible solution:

Land the deterministic Slack traces as the Phase 0 conformance oracle after the branch receives the repository’s normal maintainer refresh and exact-head gates, preserving the documented intentional sensitivity to the pinned Slack SDK buffering contract.

Do we have a high-confidence way to reproduce the issue?

Not applicable; this PR characterizes existing Slack delivery behavior rather than reporting a failing user-visible bug. The scenarios are source-verifiable against current production branches and the pinned Slack SDK.

Is this the best way to solve the issue?

Yes; extending the existing shared delivery-trace harness at the real Slack dispatch seam is the narrowest maintainable way to lock wire behavior before the channel refactor, without adding a parallel runtime path.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 702ba19e3aba.

Label changes

Label changes:

  • add P3: This is low-risk test infrastructure that strengthens refactor safety without changing runtime behavior.
  • add rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The protected maintainer-labeled, test-only PR is outside the external-contributor proof gate; its reported Testbox trace replay, full Slack suite, repeated golden verification, and Linux Node 24 run provide strong validation evidence.

Label justifications:

  • P3: This is low-risk test infrastructure that strengthens refactor safety without changing runtime behavior.
  • rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The protected maintainer-labeled, test-only PR is outside the external-contributor proof gate; its reported Testbox trace replay, full Slack suite, repeated golden verification, and Linux Node 24 run provide strong validation evidence.
Evidence reviewed

PR surface:

Source +45, Tests +637. Total +682 across 6 files.

View PR surface stats
Area Files Added Removed Net
Source 5 45 0 +45
Tests 1 637 0 +637
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 6 682 0 +682

What I checked:

  • Production-seam coverage: The test mocks only the core reply-turn boundary, then invokes the real dispatchPreparedSlackMessage path and records downstream Slack API behavior. (extensions/slack/src/delivery-trace.test.ts:90, 06468a6ed113)
  • Reachable scenario branches: Current dispatch code contains the native-stream acknowledgement tracking, fallback, preview finalization, and cancellation paths represented by the five goldens; those production files have no drift between the PR head and reviewed current main. (extensions/slack/src/monitor/message-handler/dispatch.ts:714, 702ba19e3aba)
  • Pinned dependency contract: The plugin pins @slack/web-api 7.18.0, whose ChatStreamer types and implementation confirm the 256-character default buffer, null pre-flush append result, first startStream flush, subsequent appendStream calls, and stop finalization captured by the tests. (extensions/slack/package.json:13, 702ba19e3aba)
  • Established harness pattern: Current main already exports the shared delivery-trace harness and uses the same setup, recording, normalization, and golden-verification pattern for Feishu, Mattermost, and Microsoft Teams. (src/channels/plugins/contracts/trace/delivery-trace.ts:231, 702ba19e3aba)
  • Validation evidence: The PR body reports focused 5/5 trace replay, the full Slack suite with 1,822 tests, check:changed, repeated byte-stability checks, and Linux Node 24 verification on Testbox. (extensions/slack/src/delivery-trace.test.ts:1, 06468a6ed113)
  • Live protected state: The PR is open, non-draft, mergeable, behind current main, and labeled maintainer; repository policy therefore requires explicit maintainer handling rather than automatic cleanup closure. (06468a6ed113)

Likely related people:

  • natedenh: Authored the merged PR that introduced Slack native text streaming, the central runtime behavior characterized here. (role: introduced behavior; confidence: high; commits: bb9a539d1d3f; files: extensions/slack/src/streaming.ts, extensions/slack/src/monitor/message-handler/dispatch.ts)
  • Solvely-Colin: Authored the merged configurable Slack draft-preview streaming modes that underpin the preview-edit and cancellation scenarios. (role: feature contributor; confidence: high; commits: 89ce1460e15d; files: extensions/slack/src/stream-mode.ts, extensions/slack/src/monitor/message-handler/dispatch.ts)
  • steipete: Recently authored the merged custom-identity streaming work and previously merged native Slack streaming changes, making this person closely connected to the identity and native-stream boundaries under test. (role: recent area contributor and merger; confidence: high; commits: be8b5f6c4f56, bb9a539d1d3f; files: extensions/slack/src/streaming.ts, extensions/slack/src/monitor/message-handler/dispatch.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@steipete
steipete merged commit de8cfd2 into main Jul 11, 2026
138 of 144 checks passed
@steipete
steipete deleted the claude/chanref-p0-traces-slack branch July 11, 2026 17:32
@steipete

Copy link
Copy Markdown
Contributor Author

Merged via squash.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

channel: slack Channel integration: slack maintainer Maintainer-authored PR P3 Low-priority cleanup, docs, polish, ergonomics, or speculative work. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. size: L status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant