Skip to content

fix(messages): keep Codex source replies tool-gated#83571

Merged
steipete merged 3 commits into
mainfrom
fix/codex-direct-visible-replies
May 18, 2026
Merged

fix(messages): keep Codex source replies tool-gated#83571
steipete merged 3 commits into
mainfrom
fix/codex-direct-visible-replies

Conversation

@steipete

@steipete steipete commented May 18, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Keep Codex harness direct/source visible replies on message-tool delivery by default.
  • Add the explicit old-mode opt-out via messages.visibleReplies: "automatic" and document it.
  • Resolve harness delivery defaults from the current selected model/runtime path: one-turn and heartbeat model overrides, persisted or parent session model overrides, channel model overrides, source-provider harness fallback, then configured default.
  • Let provider wildcard channels.modelByChannel.<provider>["*"] apply to direct chats before harness delivery defaults.

Verification

Behavior addressed: Codex-backed direct/source chats stay message-tool-only by default, including first turns before a session entry; messages.visibleReplies: "automatic" restores automatic final delivery; channel/default/session/parent/one-turn/heartbeat model overrides can select a non-Codex runtime before Codex delivery defaults suppress final text.
Real environment tested: local macOS focused tests; Blacksmith Testbox through Crabbox tbx_01krxhbnxfdtbfdezc04g8j3na.
Exact steps or command run after this patch: pnpm test src/auto-reply/reply/dispatch-from-config.test.ts src/channels/model-overrides.test.ts extensions/codex/index.test.ts -- --reporter=verbose; OPENCLAW_TEST_TRUST_BUNDLED_PLUGINS_DIR=1 OPENCLAW_BUNDLED_PLUGINS_DIR=extensions pnpm prompt:snapshots:check; git diff --check; AUTOREVIEW_OPENCLAW_MAINTAINER_VALIDATION=1 codex --dangerously-bypass-approvals-and-sandbox --sandbox danger-full-access review --uncommitted; node scripts/crabbox-wrapper.mjs run --provider blacksmith-testbox --blacksmith-org openclaw --blacksmith-workflow .github/workflows/ci-check-testbox.yml --blacksmith-job check --blacksmith-ref main --idle-timeout 90m --ttl 240m --timing-json -- CI=1 NODE_OPTIONS=--max-old-space-size=4096 OPENCLAW_TEST_PROJECTS_PARALLEL=6 OPENCLAW_VITEST_MAX_WORKERS=1 OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS=900000 OPENCLAW_TESTBOX=1 OPENCLAW_TESTBOX_REMOTE_RUN=1 pnpm check:changed
Evidence after fix: focused tests passed (126 auto-reply, 7 channels, 6 extensions); prompt snapshots current; final autoreview exited with no accepted/actionable findings; Testbox changed gate passed at https://github.com/openclaw/openclaw/actions/runs/26033838659.
Observed result after fix: Codex direct/source defaults are message-tool-only unless config explicitly sets automatic, while selected non-Codex model/channel/default/session/parent/one-turn/heartbeat overrides keep automatic final delivery.
What was not tested: live Telegram/Signal/Codex gateway roundtrip.

@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation gateway Gateway runtime agents Agent runtime and tooling extensions: codex size: S maintainer Maintainer-authored PR labels May 18, 2026
@clawsweeper

clawsweeper Bot commented May 18, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge.

Workflow note: Future ClawSweeper reviews update this same comment in place.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Summary
The PR removes Codex's harness-level direct source message_tool default, keeps external harness deliveryDefaults compatibility, and updates tests, docs, changelog, and prompt snapshot fixture text for the opt-in message-tool path.

Reproducibility: yes. at source level. Current main sets the Codex harness default to message_tool, the direct resolver maps that to message_tool_only, and the current dispatch test asserts the final reply is not queued.

PR rating
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🐚 platinum hermit
Summary: The implementation is focused and source-backed, but merge readiness is capped by missing real behavior proof, failed proof check, and dirty mergeability.

Rank-up moves:

  • Refresh or rebase the PR so it is mergeable against current main.
  • Attach redacted real direct-channel proof, preferably a Telegram/Signal/Feishu/Codex gateway transcript or recording showing unset Codex direct final delivery.
  • Rerun the relevant PR-head checks after the branch refresh.
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Real behavior proof
Needs real behavior proof before merge: The PR body provides source tests, prompt snapshot checks, and CI/Testbox claims, but no after-fix live channel run; the linked Testbox run is cancelled and the head Real behavior proof check failed. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Mantis proof suggestion
A real Telegram direct transcript would materially prove that the generic Codex direct delivery change reaches a user-visible channel instead of only passing source tests. A maintainer can ask Mantis to capture proof by posting a new PR comment that starts with the OpenClaw Mantis account mention, followed by:

telegram live QA: verify a Codex-backed Telegram direct chat with unset messages.visibleReplies sends the final assistant reply without message(action=send).

Risk before merge
Why this matters: - This intentionally changes the unset Codex direct-chat default from tool-only/private final text to automatic visible final delivery, so deployments relying on the previous unset default must explicitly set messages.visibleReplies: "message_tool" before upgrading.

  • The PR is currently dirty/unmergeable against main, and the cited Testbox proof link is cancelled while the head has a failed Real behavior proof check.
  • No live Signal, Telegram, Feishu, or Codex gateway roundtrip proof is attached, so the real channel delivery path remains unproven outside source-level tests.

Maintainer options:

  1. Refresh and prove the default change (recommended)
    Rebase or refresh the dirty branch, rerun checks on the PR head, and attach a redacted real direct-chat proof showing an unset Codex direct final reply reaches the source channel.
  2. Accept the compatibility flip explicitly
    Maintainers can intentionally accept that unset Codex direct chats now auto-post final text, relying on docs and changelog guidance for users who want to keep tool-only delivery.
  3. Pause if the policy is still unsettled
    If the previous Codex unset default remains desired for 1:1 chats, pause this PR and continue the direct-chat compatibility decision on the canonical Codex/GPT DM report.

Next step before merge
Human handling is needed because this protected default-policy PR is dirty/unmergeable and the missing real transport proof cannot be supplied by an automated source patch.

Security
Cleared: The diff does not add dependencies, scripts, permissions, credential handling, or downloaded code; the remaining concern is product/message-delivery compatibility, not a concrete security or supply-chain regression.

Review details

Best possible solution:

Refresh the branch, confirm maintainer acceptance of the Codex direct-chat default change, attach redacted real direct-channel proof, and then land the bounded harness-default removal with the existing docs/changelog coverage.

Do we have a high-confidence way to reproduce the issue?

Yes, at source level. Current main sets the Codex harness default to message_tool, the direct resolver maps that to message_tool_only, and the current dispatch test asserts the final reply is not queued.

Is this the best way to solve the issue?

Yes, mostly. Removing the Codex plugin-local default is the narrow owner-boundary fix and preserves generic harness default compatibility, but the default-policy and upgrade behavior need explicit maintainer acceptance plus real transport proof.

Label justifications:

  • P1: The PR targets a real visible-reply regression where direct channel users can miss final answers from Codex-backed sessions.
  • merge-risk: 🚨 compatibility: Changing the unset Codex direct-chat default can surprise existing deployments that relied on message-tool-only delivery without explicit config.
  • merge-risk: 🚨 message-delivery: The patch changes when final assistant text is sent visibly to source chats, so real transport proof matters beyond unit tests.

Acceptance criteria:

  • Refresh/rebase the branch and rerun PR-head checks.
  • Run the focused source tests covering Codex registration, source reply delivery mode, and dispatch-from-config behavior.
  • Attach redacted live direct-channel proof for unset Codex direct final delivery.

What I checked:

  • Current main Codex default: Current main still gives the Codex harness deliveryDefaults.sourceVisibleReplies: "message_tool", so unset Codex direct/source chats can prefer tool-only visible delivery. (extensions/codex/harness.ts:26, 57c952f67985)
  • Current main direct-mode resolver: For non-group/non-channel chats, current main resolves cfg.messages.visibleReplies ?? params.defaultVisibleReplies; a message_tool default maps to message_tool_only, which suppresses automatic final delivery. (src/auto-reply/reply/source-reply-delivery-mode.ts:70, 57c952f67985)
  • Current regression test expectation: Current main has a dispatch test asserting harness defaults make direct source delivery message_tool_only and do not queue the final reply. (src/auto-reply/reply/dispatch-from-config.test.ts:5252, 57c952f67985)
  • PR implementation shape: The PR diff removes the Codex harness default and changes the direct dispatch test to expect automatic delivery and a queued final reply, while adding a custom harness test that preserves generic deliveryDefaults behavior. (extensions/codex/harness.ts:26, 1699ebb49c60)
  • Related issue discussion: The linked Signal issue discussion identifies the same remaining Codex direct visible-reply policy path and includes a maintainer note that switching Codex to message_tool may be problematic for 1:1 chats.
  • Canonical compatibility report: The open Codex/GPT Feishu DM issue reports queuedFinal=false, replies=0 with unset Codex/GPT behavior and says setting messages.visibleReplies to automatic restored visible replies.

Likely related people:

  • steipete: Current blame and recent commits on the Codex harness default, source-reply resolver, and group visible-reply default path point to this handle/display identity; the PR author also appears in that merged current-main history. (role: recent source-reply/default contributor; confidence: high; commits: 46bad8676c00, 1e5450f23e1c, 1699ebb49c60; files: extensions/codex/harness.ts, src/auto-reply/reply/source-reply-delivery-mode.ts, src/auto-reply/reply/dispatch-from-config.test.ts)
  • pashpashpash: Git history shows adjacent Codex tool-reply and message-tool routing work that shaped the Codex runtime behavior around visible replies. (role: Codex harness/tool-reply contributor; confidence: medium; commits: 439d8edf68e2, 78eb92e62277; files: extensions/codex/harness.ts, extensions/codex/src/app-server/dynamic-tools.ts, src/infra/outbound/message-action-runner.ts)
  • brokemac79: Related history includes the shipped route-preservation work for keeping external channel routes when webchat views a session, adjacent to the wrong-surface/final-delivery problem family. (role: adjacent route-preservation contributor; confidence: medium; commits: 623ba1403108; files: src/auto-reply/reply/session-delivery.ts, src/auto-reply/reply/session-delivery.test.ts)
  • Josh Avant: Recent history includes a companion fix for message-tool session-key route drift, which is adjacent to the message delivery surface reviewed here. (role: adjacent message-tool route-drift contributor; confidence: medium; commits: 8ba2dfa76afb; files: src/agents/tools/message-tool.ts, src/agents/tools/message-tool.test.ts)

Codex review notes: model gpt-5.5, reasoning high; reviewed against 57c952f67985.

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. mantis: telegram-visible-proof Mantis should capture Telegram visible proof. P1 High-priority user-facing bug, regression, or broken workflow. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. labels May 18, 2026
@openclaw-mantis

Copy link
Copy Markdown
Contributor

Mantis Telegram Desktop Proof

Summary: Mantis did not generate before/after GIFs because this PR does not have a clean Telegram-visible before/after proof in the standard Mantis run.

Raw QA files: https://artifacts.openclaw.ai/mantis/telegram-desktop/pr-83571/run-26029658770-1/index.json

amknight added a commit that referenced this pull request May 26, 2026
…ce across all channel ingresses

Sweeps the same root-cause fix as PR #86825 (Mattermost) across the remaining
channels that exhibited the latent gap: Feishu, IRC, Line, MS Teams, Nextcloud
Talk, Signal, ClickClack, QA Channel, and the public 'direct-dm' plugin SDK
helper (with Nostr passing through it as the first internal consumer).

Each channel inbound handler already computed CommandAuthorized for typed slash
commands but never set CommandSource. Without CommandSource: 'text',
resolveCommandTurnContext defaults the turn kind to 'normal', so the explicit-
command exception in source-reply-delivery-mode.ts:54-56 never fires and
acknowledgements ('Session reset.', 'New session started.', 'Stopped N
subagents', etc.) are silently dropped under message_tool_only delivery
modes — most commonly the Codex harness default for DMs that #83571
introduced.

For each channel, this PR sets:

  CommandSource: commandAuthorized && hasControlCommand ? 'text' : undefined

where 'hasControlCommand' uses the channel's existing isControlCommandMessage
helper (not shouldComputeCommandAuthorized, which would be too permissive and
incorrectly tag inline command tokens within ordinary text). For ClickClack
and QA Channel (CommandAuthorized hardcoded true), the tag only depends on
the body being a control command. For Signal and Line, a new optional
'hasControlCommand' field on the entry/params struct carries the signal from
the access-policy layer where it is already computed.

The public direct-dm SDK helper grows an optional commandSource parameter so
third-party plugins built on it can pass through their own control-command
detection; Nostr is updated as the first consumer.

Adds focused regression tests (Signal, MS Teams, direct-dm SDK, plus a
negative-control assertion on the existing MS Teams inline-command test) that
fail without the fix and pass with it.

Fixes #86664.
amknight added a commit that referenced this pull request May 26, 2026
…ce across all channel ingresses

Sweeps the same root-cause fix as PR #86825 (Mattermost) across the remaining
channels that exhibited the latent gap: Feishu, IRC, Line, MS Teams, Nextcloud
Talk, Signal, ClickClack, QA Channel, and the public 'direct-dm' plugin SDK
helper (with Nostr passing through it as the first internal consumer).

Each channel inbound handler already computed CommandAuthorized for typed slash
commands but never set CommandSource. Without CommandSource: 'text',
resolveCommandTurnContext defaults the turn kind to 'normal', so the explicit-
command exception in source-reply-delivery-mode.ts:54-56 never fires and
acknowledgements ('Session reset.', 'New session started.', 'Stopped N
subagents', etc.) are silently dropped under message_tool_only delivery
modes — most commonly the Codex harness default for DMs that #83571
introduced.

For each channel, this PR sets:

  CommandSource: commandAuthorized && hasControlCommand ? 'text' : undefined

where 'hasControlCommand' uses the channel's existing isControlCommandMessage
helper (not shouldComputeCommandAuthorized, which would be too permissive and
incorrectly tag inline command tokens within ordinary text). For ClickClack
and QA Channel (CommandAuthorized hardcoded true), the tag only depends on
the body being a control command. For Signal and Line, a new optional
'hasControlCommand' field on the entry/params struct carries the signal from
the access-policy layer where it is already computed.

The public direct-dm SDK helper grows an optional commandSource parameter so
third-party plugins built on it can pass through their own control-command
detection; Nostr is updated as the first consumer.

Adds focused regression tests (Signal, MS Teams, direct-dm SDK, plus a
negative-control assertion on the existing MS Teams inline-command test) that
fail without the fix and pass with it.

Fixes #86664.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling docs Improvements or additions to documentation gateway Gateway runtime maintainer Maintainer-authored PR mantis: telegram-visible-proof Mantis should capture Telegram visible proof. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. P1 High-priority user-facing bug, regression, or broken workflow. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: L status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant