Skip to content

fix(slack): preserve mention source metadata#75356

Closed
tmimmanuel wants to merge 2 commits into
openclaw:mainfrom
tmimmanuel:fix/75203-webchat-heartbeat-session-routing
Closed

fix(slack): preserve mention source metadata#75356
tmimmanuel wants to merge 2 commits into
openclaw:mainfrom
tmimmanuel:fix/75203-webchat-heartbeat-session-routing

Conversation

@tmimmanuel

@tmimmanuel tmimmanuel commented May 1, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Problem: Slack implicit thread wakes could collapse the wake decision without preserving enough provider-native mention detail for the agent prompt.
  • Why it matters: Agents need to distinguish a direct bot mention from an implicit thread wake that happens to mention another Slack user or user group.
  • What changed: Preserve Slack mention target metadata, user-group mention metadata, implicit wake reasons, and mention source in inbound prompt context; keep selective reply guidance for implicit Slack thread wakes without direct bot mentions.
  • What did NOT change (scope boundary): No Slack docs/changelog changes remain in this PR, and no unrelated Canvas hash drift is included.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor required for the fix
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

Real behavior proof (required for external PRs)

  • Behavior or issue addressed: Slack thread messages that explicitly mention another user now preserve provider-native mention targets and distinguish implicit thread wakes from direct bot mentions.
  • Real environment tested: Local OpenClaw checkout on Ubuntu 24.04 with Node v22.22.0, live Slack socket-mode app, channel #social, PR head 83008331ff8eac444277251d8ceea566d804237b.
  • Exact steps or command run after this patch:
    1. Started the gateway from /home/ubuntu/codes/tmimmanuel/openclaw-pr-75356 on PR head 83008331ff8eac444277251d8ceea566d804237b.
    2. Confirmed Slack socket mode connected in a real Slack workspace.
    3. Used an existing #social Slack thread where OpenClaw had already participated.
    4. Sent a Slack thread message that mentioned another Slack user (@dev), not the bot.
    5. Compared the runtime prompt metadata against the same Slack thread behavior previously observed on main commit 3a0b81af9d7d86807be030415e2d1b1b38dae949.
  • Evidence after fix (screenshot, recording, terminal capture, console output, redacted runtime log, linked artifact, or copied live output): Copied live Slack output and redacted runtime log excerpt from the PR branch:
tmimmanuel: Let me repro the fix behavior.
OpenClaw: Looks like tmimmanuel is testing something related to a fix — probably not needing my input on this one. I'll stay quiet and let them work.
tmimmanuel: @dev I am talking to you.
{
  "is_group_chat": true,
  "was_mentioned": true,
  "explicitly_mentioned_bot": false,
  "mentioned_user_ids": ["U_DEV_REDACTED"],
  "implicit_mention_kinds": ["bot_thread_participant"],
  "mention_source": "implicit_thread"
}
OpenClaw reply guidance: This turn woke from implicit thread participation, not a direct bot mention. If the message is addressed to someone else, reply only when you have a specific useful addition, such as correcting an important mistake or adding requested context; otherwise use the current chat's no-response behavior.
  • Observed result after fix: The runtime context now distinguishes implicit thread participation from a direct bot mention and preserves the actual mentioned Slack user id. The agent receives enough metadata to know that @dev was the explicit Slack mention, not OpenClaw.
  • What was not tested: Slack user-group live membership behavior was covered by focused tests but not by a live workspace user-group repro.
  • Before evidence: On main commit 3a0b81af9d7d86807be030415e2d1b1b38dae949, the same Slack thread path woke OpenClaw but only exposed the collapsed mention state:
{
  "is_group_chat": true,
  "was_mentioned": true
}

The prompt did not expose explicitly_mentioned_bot, mentioned_user_ids, implicit_mention_kinds, or mention_source, so the agent could not reliably tell that the visible Slack mention was aimed at another user.

Root Cause (if applicable)

  • Root cause: The Slack inbound path reduced mention handling to a boolean wake decision before prompt metadata was built, so provider-native mention details could be lost.
  • Missing detection / guardrail: The previous Slack prepare tests did not lock in explicit mention-target metadata for implicit thread wakes.
  • Contributing context (if known): Slack has multiple wake sources: app mentions, user-group mentions, regex mention patterns, implicit thread participation, and command bypasses.

Regression Test Plan (if applicable)

  • Coverage level that should have caught this:
    • Unit test
    • Seam / integration test
    • End-to-end test
    • Existing coverage already sufficient
  • Target test or file: extensions/slack/src/monitor/message-handler/prepare.test.ts, src/auto-reply/reply/inbound-meta.test.ts
  • Scenario the test should lock in: Explicit user/user-group mention metadata and mention source survive Slack prepare and appear in inbound prompt context, including implicit thread wakes and command-bypass wakes.
  • Why this is the smallest reliable guardrail: The bug is in message preparation and prompt metadata construction, both of which are covered without needing a live Slack workspace.
  • Existing test that already covers this (if any): N/A
  • If no new test is added, why not: N/A

User-visible / Behavior Changes

Slack agents receive more precise inbound metadata for group/thread mention decisions. No config or default behavior changes.

Diagram (if applicable)

Before:
Slack message -> boolean wake decision -> prompt lacks native mention detail

After:
Slack message -> mention metadata + wake source -> prompt includes native mention detail

Security Impact (required)

  • New permissions/capabilities? No
  • Secrets/tokens handling changed? No
  • New/changed network calls? No
  • Command/tool execution surface changed? No
  • Data access scope changed? No
  • If any Yes, explain risk + mitigation: N/A

Repro + Verification

Environment

  • OS: Ubuntu 24.04
  • Runtime/container: Node v22.22.0, pnpm v10.33.2
  • Model/provider: openrouter/owl-alpha for live Slack repro
  • Integration/channel: Slack socket mode, #social channel thread
  • Relevant config (redacted): channels.slack.mode="socket", channels.slack.groupPolicy="open", channels.slack.requireMention=false, channels.slack.replyToMode="all", channels.slack.thread.requireExplicitMention=false; visible group replies temporarily set to automatic for screenshot/log proof.

Steps

  1. On main, start the gateway with Slack enabled.
  2. In a Slack channel where OpenClaw can respond, create or reuse a thread where OpenClaw has already participated.
  3. In that thread, send a message that mentions another Slack user, not the bot.
  4. Inspect the OpenClaw runtime context for the resulting Slack session.
  5. Repeat the same thread flow on this PR branch.
  6. Run focused tests and checks:
    • pnpm test extensions/slack/src/monitor/message-handler/prepare.test.ts src/auto-reply/reply/inbound-meta.test.ts -- --reporter=verbose
    • pnpm tsgo:core
    • pnpm tsgo:extensions
    • pnpm exec oxfmt --check --threads=1 extensions/slack/src/monitor/message-handler/prepare.ts src/auto-reply/reply/inbound-meta.ts src/auto-reply/templating.ts extensions/slack/src/monitor/message-handler/prepare.test.ts src/auto-reply/reply/inbound-meta.test.ts
    • git diff --check

Expected

  • Main branch shows the collapsed/insufficient metadata: was_mentioned: true without the native Slack mention target/source fields.
  • PR branch includes explicitly_mentioned_bot, mentioned_user_ids, implicit_mention_kinds, and mention_source for the same behavior path.
  • Slack prepare and inbound prompt tests pass.
  • Type checks pass.
  • Formatter and diff whitespace checks pass.
  • PR file list contains only implementation/test files for this fix.

Actual

  • Main branch reproduced the issue: runtime context only showed was_mentioned: true for the affected implicit-thread path.
  • PR branch fixed the runtime context: explicitly_mentioned_bot: false, mentioned_user_ids: ["U_DEV_REDACTED"], implicit_mention_kinds: ["bot_thread_participant"], and mention_source: "implicit_thread" were present.
  • Focused Vitest run passed 97 tests across 2 shards.
  • pnpm tsgo:core passed.
  • pnpm tsgo:extensions passed.
  • Formatter check passed.
  • git diff --check passed.
  • GitHub PR file list contains only the five Slack/prompt implementation and test files.

Evidence

Attach at least one:

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)

Human Verification (required)

What you personally verified (not just CI), and how:

  • Verified scenarios: Live Slack implicit thread wake with another user mentioned; Slack mention metadata preservation; user-group mention source handling in tests; command-bypass priority in tests; prompt metadata rendering; selective implicit-thread reply guidance.
  • Edge cases checked: Explicit bot mentions, implicit thread wakes, command-bypass wakes, bot user-group membership and non-membership, regex mention roots, direct vs group prompt metadata.
  • What you did not verify: Live Slack user-group membership path.

Review Conversations

  • I replied to or resolved every bot review conversation I addressed in this PR.
  • I left unresolved only the conversations that still need reviewer or maintainer judgment.

Compatibility / Migration

  • Backward compatible? Yes
  • Config/env changes? No
  • Migration needed? No
  • If yes, exact upgrade steps: N/A

Risks and Mitigations

  • Risk: Prompt metadata shape changes could expose Slack-native mention fields in more cases than before.
    • Mitigation: Fields are bounded string arrays/source enums already derived from inbound Slack message metadata and covered by focused prompt tests.

@openclaw-barnacle openclaw-barnacle Bot added app: web-ui App: web-ui gateway Gateway runtime size: S labels May 1, 2026
@clawsweeper

clawsweeper Bot commented May 1, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge.

Summary
The PR adds Slack mention target/source fields to prepared inbound context and renders them in the existing conversation-info metadata block, with focused Slack prepare and inbound prompt tests.

Reproducibility: yes. Source inspection on current main shows Slack can collapse explicit/implicit mention facts into only WasMentioned/was_mentioned, and the PR body provides a live Slack before/after repro for the implicit-thread path.

Real behavior proof
Needs stronger real behavior proof before merge: The PR has useful live Slack logs/screenshots for an earlier head, but the latest head changed the relevant derivation and only has tests/checks as current-head proof. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, ask a maintainer to comment @clawsweeper re-review.

Next step before merge
Contributor action is needed for current-head live Slack proof, and landing metadata should be corrected by a maintainer or the contributor; there is no code repair for ClawSweeper to generate.

Security
Cleared: No concrete security or supply-chain concern found; the diff touches Slack/prompt TypeScript and tests only, with no dependency, CI, secret, permission, or network-surface changes.

Review details

Best possible solution:

Land the additive Slack mention metadata fix with the current stripped conversation-info path, after adding current-head live Slack proof and correcting the landing metadata to close #79025 only.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection on current main shows Slack can collapse explicit/implicit mention facts into only WasMentioned/was_mentioned, and the PR body provides a live Slack before/after repro for the implicit-thread path.

Is this the best way to solve the issue?

Yes for the code direction. Carrying additive Slack mention target/source metadata through prepare and the existing stripped Conversation info block is the narrow maintainable fix without changing Slack activation policy.

What I checked:

Likely related people:

  • steipete: Recent main commits refactored the plugin SDK/channel-ingress surface and repeatedly touched Slack prepare, user-group mentions, and inbound prompt metadata paths used by this PR. (role: recent area contributor; confidence: high; commits: 827b0de0ce74, a0fb7fb04547, 5b1c2ee25f0f; files: extensions/slack/src/monitor/message-handler/prepare.ts, src/auto-reply/reply/inbound-meta.ts, src/auto-reply/templating.ts)
  • obviyus: Recent commits changed inbound prompt context rendering and reply-chain metadata in the same prompt surface that will carry the new mention metadata. (role: recent inbound prompt context contributor; confidence: high; commits: 7d00183d44b2, 40fd42206fd4, ac75d6f76e2f; files: src/auto-reply/reply/inbound-meta.ts, src/auto-reply/templating.ts)
  • zeroth-blip: A recent merged change in the same Slack prepare path fixed implicit-conversation thread routing adjacent to the implicit wake behavior covered here. (role: recent Slack thread-routing contributor; confidence: medium; commits: 741315e65791; files: extensions/slack/src/monitor/message-handler/prepare.ts)
  • bek91: Authored the canonical Slack metadata issue, gave maintainer-style review feedback on this PR, and has recent merged Slack inbound thread session routing history in the same file. (role: adjacent contributor and reviewer; confidence: medium; commits: aac83e00cfe7; files: extensions/slack/src/monitor/message-handler/prepare.ts)
  • vincentkoc: Recent commits optimized Slack thread participation and message hot-path logic near the mention/implicit-thread wake code. (role: recent Slack hot-path contributor; confidence: medium; commits: ac74a928456d, c0302512d4dd, 4eba70b532f8; files: extensions/slack/src/monitor/message-handler/prepare.ts, src/auto-reply/templating.ts)

Remaining risk / open question:

Codex review notes: model gpt-5.5, reasoning high; reviewed against e384932497f9.

@tmimmanuel
tmimmanuel force-pushed the fix/75203-webchat-heartbeat-session-routing branch 2 times, most recently from a8db8db to 21c619b Compare May 1, 2026 02:28
@tmimmanuel
tmimmanuel force-pushed the fix/75203-webchat-heartbeat-session-routing branch from 21c619b to d6ac408 Compare May 1, 2026 02:33
@openclaw-barnacle openclaw-barnacle Bot added the agents Agent runtime and tooling label May 1, 2026
@tmimmanuel
tmimmanuel force-pushed the fix/75203-webchat-heartbeat-session-routing branch from d49e7bb to 88829bd Compare May 4, 2026 21:11
@tmimmanuel
tmimmanuel marked this pull request as draft May 5, 2026 04:47
@openclaw-barnacle openclaw-barnacle Bot added the triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. label May 5, 2026
@tmimmanuel
tmimmanuel force-pushed the fix/75203-webchat-heartbeat-session-routing branch from 88829bd to a7a58f8 Compare May 5, 2026 05:40
@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation size: XS triage: low-signal-docs Candidate: docs-only change looks low signal; maintainer review needed. and removed app: web-ui App: web-ui gateway Gateway runtime agents Agent runtime and tooling size: M labels May 5, 2026
@tmimmanuel tmimmanuel changed the title fix(ui): prevent webchat sends into isolated heartbeat sessions docs: clarify Codex backend OAuth setup for isolated agents May 5, 2026
@tmimmanuel
tmimmanuel marked this pull request as ready for review May 5, 2026 05:45
@openclaw-barnacle openclaw-barnacle Bot removed the triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. label May 5, 2026
@openclaw-barnacle openclaw-barnacle Bot added the triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. label May 8, 2026
@tmimmanuel tmimmanuel changed the title docs: clarify Codex backend OAuth setup for isolated agents fix: preserve slack mention source metadata May 8, 2026
@openclaw-barnacle openclaw-barnacle Bot added proof: supplied External PR includes structured after-fix real behavior proof. and removed triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels May 8, 2026
@openclaw-barnacle openclaw-barnacle Bot added the proof: supplied External PR includes structured after-fix real behavior proof. label May 8, 2026
@tmimmanuel
tmimmanuel force-pushed the fix/75203-webchat-heartbeat-session-routing branch 2 times, most recently from dcb36f5 to a59a347 Compare May 8, 2026 10:52
@openclaw-barnacle openclaw-barnacle Bot added triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. proof: supplied External PR includes structured after-fix real behavior proof. and removed docs Improvements or additions to documentation proof: supplied External PR includes structured after-fix real behavior proof. triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels May 8, 2026
@tmimmanuel

Copy link
Copy Markdown
Contributor Author

Human Verification (required)

What I personally verified (not just CI), and how:

  • Verified scenarios: Live Slack implicit thread wake with another user mentioned; Slack mention metadata preservation; user-group mention source handling in tests; command-bypass priority in tests; prompt metadata rendering; selective implicit-thread reply guidance.
  • Edge cases checked: Explicit bot mentions, implicit thread wakes, command-bypass wakes, bot user-group membership and non-membership, regex mention roots, direct vs group prompt metadata.

I’m attaching screenshots from the live Slack repro. The important behavior is that the fix branch now exposes the explicit mentioned user and wake source metadata, so OpenClaw can distinguish an implicit thread wake from a direct bot mention.

Screenshot 2026-05-08 101405 Screenshot 2026-05-08 102344

@bek91

bek91 commented May 10, 2026

Copy link
Copy Markdown
Contributor

@tmimmanuel

Thanks for taking this on. Directionally the fix looks correct, but not ready to merge yet. Please make the following changes:

  1. The new OpenClaw reply guidance: text is appended as ordinary user-context text in buildInboundUserContextPrefix. That makes it AI-facing, but it is not part of the known stripped metadata sentinels in stripInboundMetadata, so it can leak into visible chat history/UI surfaces. I’d either remove this guidance from the PR, move it into an existing stripped/trusted metadata path, or extend the stripper with explicit regression coverage.

  2. The PR contains unrelated changes, especially the extensions/oc-path changes and import-order changes in unrelated files. Please narrow the branch to the Slack mention metadata fix and its focused tests.

Surface Slack mention metadata (source, was_mentioned, explicitly_mentioned_bot,
mentioned_user_ids, mentioned_subteam_ids, implicit_mention_kinds) through the
inbound prepare pipeline and into the conversation-info metadata payload so the
LLM can reason about whether the bot was directly addressed or only joined an
implicit thread. Closes openclaw#75203.
@tmimmanuel
tmimmanuel force-pushed the fix/75203-webchat-heartbeat-session-routing branch from d8ae0c2 to 73a2306 Compare May 10, 2026 11:38
@openclaw-barnacle openclaw-barnacle Bot added size: M and removed agents Agent runtime and tooling extensions: oc-path size: XL labels May 10, 2026
…ions

# Conflicts:
#	extensions/slack/src/monitor/message-handler/prepare.ts
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 10, 2026
@tmimmanuel

Copy link
Copy Markdown
Contributor Author

@bek91 — thanks for the review. Both asks addressed; force-pushed to 2338e75829.

1. OpenClaw reply guidance: leak risk

Removed buildMentionReplyGuidance and its caller entirely (along with the two tests that asserted the guidance text). The structured mention metadata (mention_source, was_mentioned, explicitly_mentioned_bot, mentioned_user_ids, mentioned_subteam_ids, implicit_mention_kinds) still reaches the LLM, but only inside the existing Conversation info (untrusted metadata): JSON block — which is already in INBOUND_META_SENTINELS in src/auto-reply/reply/strip-inbound-meta.ts, so it gets stripped from any user-visible chat history. No new sentinel surface to maintain.

If we want a selective-reply nudge for mention_source: implicit_thread, that can be a follow-up PR with explicit stripper coverage; out of scope for "preserve mention source metadata".

2. Unrelated changes

The extensions/oc-path/**, src/plugins/discovery.ts, and src/agents/pi-embedded-runner/run/payloads.ts entries were formatter churn (single-line if brace expansion, import reorders) that got baked into the original commits. I reset the branch to upstream/main and reapplied only the slack-related files. PR diff is now:

  • extensions/slack/src/monitor/message-handler/prepare.ts
  • extensions/slack/src/monitor/message-handler/prepare.test.ts
  • src/auto-reply/reply/inbound-meta.ts
  • src/auto-reply/reply/inbound-meta.test.ts
  • src/auto-reply/templating.ts

5 files, +294/-17.

Merge / verification

While narrowing the branch I also re-merged upstream/main, which had refactored the mention/activation flow into messageIngress.activationAccess. Resolved by reading effectiveWasMentioned / shouldBypassMention from the new ingress projection and re-deriving matchedImplicitMentionKinds locally (the activation-access projection doesn't surface that field) so resolveSlackMentionSource keeps the same inputs.

Local proof:

  • pnpm test src/auto-reply/reply/inbound-meta.test.ts extensions/slack/src/monitor/message-handler/prepare.test.ts → 57/57 pass
  • pnpm check:changed → exit 0 (typecheck, lint, runtime import cycles, conflict markers, changelog attributions, sidecar loader guard)

@steipete

Copy link
Copy Markdown
Contributor

Landed a corrected maintainer version in 7b781a8.

What changed from the PR branch:

  • reapplied the narrow Slack mention metadata fix on current main
  • kept the prior review fix: no AI-facing free-text guidance outside the existing stripped Conversation info (untrusted metadata) block
  • kept source priority as explicit bot/subteam, command bypass, mention-pattern, then implicit thread
  • added changelog and preserved contributor credit via Co-authored-by

Slack dependency/code-path check:

  • local Slack SDK types confirm this path is fed by Slack message / app_mention events plus threaded payload fields such as thread_ts and parent_user_id
  • the installed Slack dependency does not provide a richer typed parser for inline <@U...> / <!subteam^S...> mrkdwn tokens, so the fix keeps parsing local to the Slack plugin and continues to validate subteam membership through the existing Slack usergroups API path

Validation:

  • pnpm test extensions/slack/src/monitor/message-handler/prepare.test.ts src/auto-reply/reply/inbound-meta.test.ts
  • node scripts/run-oxlint.mjs --tsconfig config/tsconfig/oxlint.extensions.json extensions/slack/src/monitor/message-handler/prepare.ts extensions/slack/src/monitor/message-handler/prepare.test.ts
  • node scripts/run-oxlint.mjs --tsconfig config/tsconfig/oxlint.core.json src/auto-reply/reply/inbound-meta.ts src/auto-reply/reply/inbound-meta.test.ts src/auto-reply/templating.ts
  • pnpm exec oxfmt --check --threads=1 CHANGELOG.md extensions/slack/src/monitor/message-handler/prepare.ts extensions/slack/src/monitor/message-handler/prepare.test.ts src/auto-reply/reply/inbound-meta.ts src/auto-reply/reply/inbound-meta.test.ts src/auto-reply/templating.ts
  • git diff --check HEAD~1..HEAD

Note: broad pnpm check:changed passed conflict-marker, changelog, duplicate, core typecheck, core-test typecheck, extension typecheck, extension-test typecheck, and core lint lanes, then failed in broad lint:extensions on pre-existing unrelated no-base-to-string findings in Discord, memory-core, and synology-chat tests.

Thanks @tmimmanuel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

channel: slack Channel integration: slack proof: supplied External PR includes structured after-fix real behavior proof. size: M triage: low-signal-docs Candidate: docs-only change looks low signal; maintainer review needed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Slack: preserve explicit mention targets and distinguish implicit thread wakes from bot mentions

3 participants