Stacked on #91438 — review the increment (df7c31a106…a4477208de, 55 commits): the foundation is #91438. The full diff includes #91438's commits only until that PR lands; this branch then rebases to exactly the increment. A commit→feature map is below.

Summary

Builds on the msteams voice provider (#91438) along six lines:

• Correctness / security — fixes the realtime self-answer (echo) bug, makes the bridge handshake replay-proof, enforces group etiquette and verbal interrupts deterministically (in code at the audio egress, not model-trusted), and fixes outbound call-backs that spoke to a still-ringing phone (and the context-bleed that caused).
• Call UX — verbal interrupts ("stop" / wake-phrase forms, English and Arabic), notify call-backs (deliver → hang up, un-clipped), greet-by-name from the roster, bilingual Arabic/English, DTMF / IVR ("press 1 to…").
• CVI — retroactive vision ("what did the earlier slide say?"), scene-change + camera/screen ambient vision, captioned/slideshow visual sharing as a picture-in-picture overlay over the avatar, a "thinking" expression, per-thread sessions.
• Meeting productivity — opt-in end-of-meeting minutes posted to the Teams chat/thread, on-demand "summarize the meeting" mid-call, a Word-openable minutes document, and per-person attribution from unmixed meeting audio.
• Chat — a Teams "Ask OpenClaw about this" message action routes a selected chat message to the agent; opt-in voice-message transcription folds spoken clips into the agent's turn.
• Governance — opt-in audit-log channel mirroring agent replies and DLP-aware outbound redaction.

A post-feature self-review hardening pass then audited the whole increment and fixed everything it found (2 critical, 13 bugs, 1 security finding, structural refactors) — see "Review hardening" below.

⚠️ Behavior changes

• realtime.suppressInputDuringPlayback default flipped off → on: default-off was the live self-answer bug — the bot answered its own echo on every realtime deployment. Set false to restore; tune via realtime.echoSuppressionWindowMs / realtime.echoBargeInRms.
• Outbound call-backs now honor outbound.defaultMode (default notify = state the result, then hang up after the audio drains) and speak only once the callee answers (recording-active), not when the bridge connects — on both the realtime and streaming paths. Previously they greeted on connect — i.e. while the phone was still ringing — so the result was lost and the idle model could surface unrelated context. Set "conversation" to keep the lingering behavior.
• msteams.meetingRecap (default off) — when on, the bot posts meeting minutes to the caller's Teams chat (1:1) or the meeting thread (group) at call end. The bot posting unprompted is gated behind this opt-in.
• msteams.bilingual (default off) — when on, the realtime model is pinned to detect/mirror Arabic↔English and translate on request. The deterministic layers (verbal interrupts, visemes) understand Arabic regardless.
• channels.msteams.transcribeVoiceMessages (default off) — when on, inbound Teams voice clips are transcribed (STT cost) and folded into the agent's message text.
• channels.msteams.auditChannel (unset by default) — when set to a conversation id, agent replies are mirrored there for compliance review.
• channels.msteams.dlp.enabled (default off) — when on, built-in categories and customPatterns are redacted from every outbound surface (block messages, streamed replies, edits, adaptive cards). Because per-token streaming cannot be redacted retroactively (a secret completing across chunk boundaries would already be on screen), DLP also downgrades streaming.mode: "partial" to "progress": the live card shows working status and the redacted full reply lands on it in place.

🛡️ Security & correctness hardening

• Single-use HMAC upgrade tuples (replay guard) — a captured handshake replayed within the HMAC window could open a ghost session after the live call ended. (callId, timestamp, signature) tuples are single-use; a legitimate reconnect carries a fresh timestamp; only verified tuples are recorded (no unauthenticated map growth) and entries expire with the timestamp's own validity window (a future-dated handshake from worker clock skew can't outlive its record). Tested: connect OK → exact replay 401 → fresh-timestamp reconnect OK.
• Deterministic group-call gate (realtime) — every finished caller turn is evaluated with the same shouldRespondToGroupTurn core the streaming path uses; an unaddressed meeting turn's reply audio is dropped at the egress. 1:1 never gates.
• Echo guard on a playout clock — the realtime model streams faster than realtime, so last-send time is not play-time; the guard and the notify drain both key on a playout estimate (accumulated chunk durations, collapsed on barge-in). Regression test: burst 10 s of audio → echo still suppressed at +2 s. Opening-greeting hardening (live-reported): until the caller's first real turn the guard allows no barge-in, so the bot's own greeting echoing back on a speakerphone can't make the model interrupt itself and re-greet in a loop while the caller is silent.
• Greet-on-answer (outbound call-backs) — the greeting (the delivered result) now fires on recording-active (the callee answered), not on bridge-connect; this both delivers the result at the right time and closes the idle window where the model would otherwise free-associate from its context.

📞 Call experience

• Deterministic verbal interrupts — "stop" / "wait" / "hold on" / "never mind" handled in code: streaming cuts playback and suppresses the reply; realtime flushes the worker queue. Whole-utterance match with filler and configured wake-phrase stripping — "⟨name⟩, stop" cuts as instantly as bare "stop"; "stop by the store" never triggers. Unicode-aware: Arabic interrupts ("توقف", "خلاص", …) cut identically, with tashkeel-safe normalization (same \p{L} approach as the group gate).
• Roster greeting — the model greets the caller by first name (from the Teams roster) and addresses participants by the per-utterance speaker label.
• Bilingual Arabic/English (opt-in) — detect, mirror, switch-with-caller, translate on request; the viseme estimator also maps Arabic graphemes so lip-sync isn't RMS-only in Arabic.
• DTMF / IVR — in-band keypad tones from the caller are surfaced to the realtime model as a user message ("the caller pressed 1"), so the agent can run "press 1 to…" flows. (Worker companion forwards the tone.)
• Notify call-backs deliver-and-hang-up without clipping — drain-aware (playout clock) + notifyHangupDelaySec grace; a failed hangup falls back to a direct teardown.

🎥 CVI extensions

• Retroactive vision (frame history) — a ring of 16 scene-change keyframes per call; look_at_screen scope:"history" attaches up to 6 timestamped, attributed keyframes to one budgeted consult. No eager OCR-per-keyframe.
• Scene-change-driven ambient vision — frames reach the realtime model on arrival; camera + screen simultaneously, per-source dedup + labels.
• show_to_caller captions + slideshow + PiP overlay — caption strip on the tile; multi-image results play as a paced slideshow; images ride as a bottom-right inset over the live avatar (display.image mode:"overlay") so the bot stays visible. Additive protocol; older renderers fall back to fullscreen.
• "Thinking" avatar expression while a tool runs; the reply re-cues the real emotion.
• Real lip-sync timing — the elevenlabs provider's telephony synthesis uses /with-timestamps and surfaces per-character alignment (optional field through textToSpeechTelephony); msteams speech.marks now carry each sound's actual start time instead of an even spread, falling back to the estimator when alignment is absent. Audio never depends on timing: a failing /with-timestamps endpoint degrades to plain synthesis.
• Per-thread sessions — sessionScope: "per-thread" keys the agent session by the Teams meeting/chat thread; parallel conversations stay isolated. PSTN falls through to per-phone.

📝 Meeting productivity

• End-of-meeting recap (opt-in msteams.meetingRecap) — key points / decisions / action items posted to the caller's Teams chat (1:1) or into the meeting thread (group) when the call ends.
• On-demand minutes — "summarize the meeting" mid-call posts the running minutes to chat without ending the call (post_meeting_minutes tool).
• Word-openable minutes artifact — the recap attaches a formatted document the caller can open in Word.
• Per-person attribution — with unmixed meeting audio the worker tags each turn with the speaker, so minutes and transcripts read "Sara: …" instead of one mixed stream.

💬 Chat & governance

• "Ask OpenClaw about this" message action — a Teams message-context command (composeExtension/submitAction) quotes the selected chat message into a prompt and dispatches it through the normal message path, so the answer posts back into the conversation. (Requires the Teams app manifest to declare the command: composeExtensions > commands, type: "action", context: ["message"], id: "askOpenClaw".)
• Voice messages (opt-in transcribeVoiceMessages) — inbound Teams audio clips are transcribed and folded into the agent's turn as a quoted block; a clip that fails transcription keeps its attachment placeholder.
• Audit-log channel (opt-in auditChannel) — agent replies are mirrored to a named Teams conversation for compliance review. The mirror fires at the delivery choke point, so natively-streamed replies (the DM default) are in the trail too, not only block deliveries; the loop guard keeps the audit channel's own traffic out.
• DLP outbound redaction (opt-in dlp.enabled) — built-in categories plus custom patterns are redacted with a configurable placeholder, on every outbound surface: block messages, streamed replies (see Behavior changes), message edits, and adaptive cards (deep-redacted per string value). The audit excerpt is redacted before truncation so the cap can never slice a secret into an unrecognizable fragment.

🔎 Review hardening (post-feature self-audit)

After the feature work, a full self-review of the increment found and fixed — each with a regression test:

• 2 critical — realtime mode never wired the CallManager (the stale-call reaper force-ended connected outbound callbacks ~2 min in); the msteams safe-allowlist inbound default was dead code (msteams+streaming rejected ALL inbound calls in production config). (c38d360642)
• 1 security — HMAC replay records now expire at ts+window, not now+window, closing a replay window for future-dated (clock-skewed) handshakes. (bd5dbc06f3)
• 4 state-leak paths, one design fix — a single releaseCallState owner that every teardown funnels through: host-initiated closes no longer suppress onSessionEnd (dead calls reported in-progress forever); a realtime provider WS drop no longer strands the caller on an un-hangup-able call; manager hangups release vision/recording state (~1–2 MB/call); a declined outbound call no longer leaves its CallRecord active against maxConcurrentCalls. (15f5efbe2e)
• 9 individual bugs — deliver-to-ringing on the streaming path; custom-tool calls stalling the model turn; unbounded/misattributed transcript coalescing; the message action being mention-gated to silence in group chats; history-look poisoning the live vision cache; the minutes tool skipping the recording gate + the task tool acking a no-op; empty-aadId cross-caller session bleed; lost-and-budget-burning ambient vision pushes; voice-clip placeholder mishandling. (cb29dcae1b)
• Structural refactors so this class of bug has fewer places to hide — withConsultGuards (one copy of the recording-gate/deps/error/filler boilerplate for all agent-running tools), emitCallEnded, a single hoisted consult session key, the tool surface split into msteams-realtime-tools.ts, and the msteams runtime no longer building the webhook-plane realtime handler Teams never uses. (9305586f1a)
• Bilingual honored by the deterministic layers — verbal interrupts match Arabic (Unicode normalization + Arabic phrase set) and the viseme map covers Arabic graphemes. (eace301545)
• 4 governance bypasses (the review's most severe findings) — DLP/audit only covered the block-delivery text path. Now: native streaming redacted (token streaming downgrades to progress under DLP — append-only streams can't retract a secret that completes across chunks); the audit mirror moved to the delivery choke point so streamed replies are in the trail; message edits and adaptive cards redact (closing the "edit the secret in" / "put it in a card" loopholes); the secret pattern catches segmented sk-proj-/sk-ant- keys and the audit excerpt redacts before truncating. (4bf60c2850)
• Poll-card DLP redaction (Codex review follow-up) — the ClawSweeper Codex pass caught a remaining card surface: sendPollMSTeams sent the poll Adaptive Card unredacted (the S3 fix only covered the generic adaptive-card path), so a secret in a poll question or option label reached the wire. The poll's agent-supplied question and option strings are now redacted before the card is built — visible text only, leaving the internal option indices and pollId intact so vote matching is unaffected. (fe2f789935)
• DTMF recording gate (Codex review follow-up) — DTMF keypad input was forwarded to the realtime model before the call's recording status was active, bypassing the Media Access API recording gate that governs audio/video/transcripts/consults. notifyDtmf now drops keypresses until recording is active (regression test included). (63f7f30e88)

Commit → feature map (increment beyond #91438)

Each feature ships with unit tests (matcher/positive-negative units, display-path, history-ring, group-gate egress drop, replay 401/reconnect, recap prompt, DTMF mapping, message-action extraction, teardown/leak regressions, Arabic interrupt/viseme cases, etc.).

✅ Live validation (2026-06-12, real Teams calls)

Foundation flows (base voice/video, group calls, background-task chat delivery, outbound callback) were separately live-proven on the #91438 foundation PR — see its Real-behavior-proof section; this PR’s diff includes those commits until #91438 lands.

Re-validated end-to-end on this branch's exact head against a production Teams tenant (gateway from source, companion media worker as a Windows service), in two calls:

• Inbound 1:1 call (~7.5 min) — roster greeting by name; no echo self-answer with the reply on open speakers; barge-in; deterministic verbal interrupts in English, wake-phrase form, and Arabic ("توقف"); bilingual Arabic↔English mirroring + on-request translation; openclaw_agent_consult with the "thinking" avatar expression held across six tool runs (worker log); emotion cues (Happy/Neutral transitions) tracking reply tone; screen-share vision (look_at_screen, scene-change awareness, retroactive keyframe history — two VBSS share windows in the worker log); show_to_caller rendered two images as a PiP overlay with caption, paced as a slideshow with exactly the coded hold times (4500 ms non-final / 5000 ms final).
• Outbound notify call-back (placed by an in-call background task): outbound call placed → Established → recording-active gate opened → result spoken only after answer (greet-on-answer) → notify-mode auto-hangup after the audio drained — complete, un-clipped delivery, clean teardown, no leaked call state.
• Meeting productivity — on-hangup recap fired (posting meeting recap) and the minutes landed in the Teams chat; in-call background task delivered its answer to chat.
• Both calls tore down cleanly (connection closed with no lingering state) — the review-hardening teardown paths behaving live.
• The validation also caught that the governance keys were unconfigurable (stale generated channel metadata + a Windows-silent regen script) — fixed in 90b7805cfd; the gateway now starts with dlp/transcribeVoiceMessages/auditChannel configured.

Chat & governance (same day, follow-up session):

• Audit mirror on streamed replies (S2) — with auditChannel set to a dedicated group chat, every streamed DM reply produced a 🧾 Reply for <sender> in <conversation>: <excerpt> line in the audit chat; the bot's own reply inside the audit chat produced no self-mirror (loop guard).
• DLP on the wire (S1) — a DM reply containing an email rendered as …([REDACTED:email]) in the Teams client: the model emitted the address, the redaction layer scrubbed it on the streamed path — the exact bypass this PR closes. (The secret category never reached the wire live because the model itself refused to repeat even a dummy sk-proj-… token — twice; the category patterns are unit-tested and the wire path is category-agnostic.)
• Message action (Fix path traversal vulnerability in media server #10) — Ask OpenClaw about this on a DM message dispatched the quoted-message prompt and produced a substantive reply in the conversation. (The group/mention-gate case is unit-tested.)
• Voice-message transcription (Feature Request: Group Chat Mention Detection #13) — not live-testable in this tenant: Teams desktop's mic-only "video clip" is a stream attachment that delivers no media to bots (the bot correctly reported only the text label arrived), and the tenant's mobile client has no voice-clip button. Unit-tested; the graceful no-media handling was observed live.
• Mention gating — unmentioned messages in the group chat were correctly ignored; a mentioned message was answered.

Real lip-sync timing (2026-06-13, follow-up): the ElevenLabs /with-timestamps alignment path (0ef6f8e570) was validated on a live Teams call with the SVG face avatar and debug logging. The greeting's mouth track was driven by real per-character timing, not the even-spread estimator:

04:53:23 MsteamsProvider: session.start callId=22005380-…
04:53:24 MsteamsProvider: recording.status 22005380-… = active
04:53:2x MsteamsProvider: expression cue 'neutral' for 22005380-…        ← debug level active
04:53:2x MsteamsProvider: speech.marks 60 visemes (aligned) for 22005380-…   ← real ElevenLabs alignment
04:53:36 [worker] Call status updated to Terminated - The conversation has ended
04:53:36 [worker] OpenClaw bridge closed for call 22005380-… (call-ended); disposed 0 audioMediaBuffers

(aligned), not (estimated) — the with-timestamps endpoint succeeded with no fallback, and the 60 marks carry each character's actual start time. The caller-hangup teardown was clean on both sides (Graph Terminated → bridge closed → disposed, no leak). Full evidence: docs/proofs/LIVE-TEST-92081.md in the companion worker repo.

Real behavior proof

• Behavior or issue addressed: End-to-end behavior of this PR on real Microsoft Teams calls and chats: realtime voice (echo self-answer fix, deterministic verbal interrupts incl. Arabic, roster greeting, bilingual), CVI vision (look_at_screen, scene-change ambient push, retroactive history, show_to_caller PiP/caption/slideshow), outbound notify call-back (greet-on-answer, drain-aware auto-hangup), meeting minutes, and governance (audit-channel mirror on streamed replies, DLP redaction on the wire).

• Real environment tested: Production Microsoft 365 tenant (PCFC, Dubai) with the Teams desktop client; OpenClaw gateway run from source at this branch's head on a Windows Server VM; the companion app-hosted media worker running as a Windows service; Azure OpenAI realtime voice model; ElevenLabs STT/TTS.

• Exact steps or command run after this patch: pnpm openclaw gateway run on the branch head with msteams.dlp.enabled, auditChannel, meetingRecap, and bilingual configured → placed a 1:1 Teams voice call to the bot and exercised greeting, echo, interrupts ("stop" / "assistant, stop" / "توقف"), Arabic conversation, screen-share vision, "show me a screenshot", and "summarize the meeting" → said "look up the time in Tokyo and call me back", hung up, and answered the bot's call-back → ran the chat pass: DM questions including "reply with exactly: contact our support team at [email protected]", with an "Audit Log" group chat configured as the audit channel.

• Evidence after fix: Copied (redacted) gateway runtime log from the live calls:

  06:18:54 MsteamsProvider: realtime session.start callId=24005480-… caller.aadId=9a78…
  06:18:54 MsteamsProvider: recording.status 24005480-… = active (realtime)
  06:26:22 MsteamsRealtime: posting meeting recap for 24005480-…
  06:26:32 MsteamsProvider: outbound call placed callId=2b005180-… -> 9a78…   (the caller)
  06:26:34 MsteamsProvider: outbound realtime session.start … attached to internal e4ad…
  06:26:38 MsteamsProvider: recording.status 2b005180-… = active (realtime)  ← greet-on-ANSWER trigger
  06:26:52 MsteamsProvider: notify-mode auto-hangup for 2b005180-… after delivery
  

  Worker runtime log excerpt: [display] showing image for 4500 ms (mode=overlay, caption=True) followed by 5000 ms — the show_to_caller slideshow pacing constants observed on the wire. Copied live output from the Teams client: the bot's streamed DM reply rendered as Noted — thanks, Alaaeldin. I've got the support contact ([REDACTED:email])., and the audit chat received 🧾 Reply for Eng. Alaaeldin Mohamed Elhenawy in a:180Gt…: … mirror lines — one per streamed DM reply.

• Observed result after fix: Caller greeted by name; the bot never answered its own echo; all three interrupt forms cut playback instantly; vision answered correctly across two screen-share windows including "what did the FIRST screen say"; the captioned PiP slideshow displayed; the call-back spoke only after the callee answered, delivered the complete un-clipped result, and hung itself up; recap + minutes landed in the Teams chat; the email in a reply reached the client as [REDACTED:email]; every streamed DM reply was mirrored to the audit channel and the bot's own reply inside the audit chat produced no self-mirror.

• What was not tested: Voice-message transcription live (this tenant exposes no audio voice-message capability on either Teams client — the desktop mic-only "video clip" delivers no media to bots, and the bot's graceful no-media reply WAS observed live); the DLP secret category on the wire (the model refused to repeat even a stated-dummy sk-proj-… token, twice — the email category proves the same redaction path); the group-chat form of the message action; DTMF (no dialpad on VoIP 1:1 calls); group/meeting calls with a second human participant.

Foundation recap (from #91438, included in this diff until it lands)

The msteams provider: inbound Teams voice + video vision (look_at_screen, streaming auto-attach, realtime continuous vision via RealtimeVoiceBridge.sendImage), group/meeting awareness (per-participant attribution + speak-only-when-addressed), outbound "call me back" with voicemail fallback, avatar rendering drivers (expression cues, viseme lip-shape, show_to_caller), caller allowlist (closed by default), recording-status gate on every media-derived path, HMAC-authenticated bridge WebSocket, SSRF-guarded outbound. Live-proven on 1:1 and 3-party group Teams calls.

Notes

• Several features (DTMF tone capture, unmixed meeting audio, avatar rendering) have a companion in the application-hosted media worker; the protocol additions are backward-compatible (older workers ignore unknown fields).
• The avatar art is a worker-side choice (not an OpenClaw config key): the companion ships two styles — an SVG face with full lip-sync/viseme shapes, expressions, and blink (AvatarImagePath=avatar, the default), and a Lottie loop with speech-reactive zoom (AvatarImagePath=jarvis) — plus explicit file paths and a procedural fallback. The expression / speech.marks / display.image messages this PR emits drive whichever style is configured.
• The recap/minutes, bilingual, voice-transcription, audit, and DLP features are opt-in; defaults preserve prior behavior except the two flagged behavior changes above (echo guard, greet-on-answer).

Acknowledgements

Contributed by the team at the Ports, Customs and Free Zone Corporation (PCFC) — Dubai, United Arab Emirates — bringing Microsoft Teams voice & video (the Conversational Video Interface) to OpenClaw for public-sector use. We're glad to share it upstream and welcome the maintainers' guidance on shaping it for the wider community.