feat(voice-call): add Microsoft Teams voice provider (OpenClawTeamsBridge)#89724
feat(voice-call): add Microsoft Teams voice provider (OpenClawTeamsBridge)#89724ahenawy wants to merge 10 commits into
Conversation
openclaw-barnacle
Bot
added
channel: voice-call
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
Codex review: needs maintainer review before merge. Reviewed June 19, 2026, 5:02 AM ET / 09:02 UTC. Summary PR surface: Source +2191, Tests +1610, Docs +139. Total +3940 across 24 files. Reproducibility: not applicable. as a bug reproduction; source inspection confirms current main rejects Review metrics: 2 noteworthy metrics.
Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Risk before merge
Maintainer options:
Next step before merge
Security Review detailsBest possible solution: Have maintainers choose one canonical Teams direction and explicitly accept the bundled-vs-external provider home plus the external bridge trust contract before landing any Teams voice PR. Do we have a high-confidence way to reproduce the issue? Not applicable as a bug reproduction; source inspection confirms current main rejects Is this the best way to solve the issue? Unclear as the best product solution: the patch is a plausible implementation, but VISION and the open supersets require maintainer choice between bundled voice foundation, broader CVI, or external plugin ownership. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against dba291ed3559. Label changesLabel justifications:
Evidence reviewedPR surface: Source +2191, Tests +1610, Docs +139. Total +3940 across 24 files. View PR surface stats
Security concerns:
Acceptance criteria:
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
|
openclaw-barnacle
Bot
removed
the
triage: needs-real-behavior-proof
|
Addressed the review feedback:
(Edited to fix character encoding.) |
openclaw-barnacle
Bot
added
the
proof: supplied
clawsweeper
Bot
added
rating: 🧂 unranked krab
|
Update: the full live Teams audio round-trip is now proven on a real call; the PR body "Real behavior proof" section is updated accordingly. Captured from a real inbound Microsoft Teams call (OpenClaw 2026.6.2, provider Gateway lifecycle log: Agent session trajectory for the same call ( The disfluencies ("uh", "Mm-hmm") confirm live ASR; the (Edited to fix character encoding.) |
clawsweeper
Bot
added
proof: sufficient
openclaw-barnacle
Bot
removed
the
proof: sufficient
P1 review points - addressedHardening pushed (commit
Tests: 45 passing across provider + transport + config; |
clawsweeper
Bot
added
the
proof: sufficient
openclaw-barnacle
Bot
removed
the
proof: sufficient
clawsweeper
Bot
added
proof: sufficient
|
Thanks for the thorough re-review, and for confirming the proof. Two findings are genuinely maintainer-policy calls, so I would value your steer before building further:
In parallel I am addressing the code findings that apply regardless of the boundary decision:
I will push these shortly and follow your lead on the two policy items above. |
openclaw-barnacle
Bot
removed
the
proof: sufficient
clawsweeper
Bot
added
the
proof: sufficient
openclaw-barnacle
Bot
removed
the
proof: sufficient
Closed the last author-side finding: msteams setup/status webhook-exposurePushed With this, all author-side code findings from the reviews are addressed, the live realtime-delegation + recording-gate proof is in the PR body, and CI is green (the earlier |
clawsweeper
Bot
added
proof: sufficient
openclaw-barnacle
Bot
removed
the
proof: sufficient
clawsweeper
Bot
added
the
proof: sufficient
openclaw-barnacle
Bot
removed
the
proof: sufficient
clawsweeper
Bot
added
rating: 🌊 off-meta tidepool
🗺️ Roadmap — where Microsoft Teams support is headingThis PR is Phase 1 of a larger Teams presence for OpenClaw. Sharing the full trajectory so the bundled-vs-plugin home and the worker/security-boundary decisions can be weighed against where this is going. Every phase reuses the same authenticated worker bridge and session model this PR establishes — so the security contract accepted here is the foundation the later phases build on. Phase 1 — Inbound voice (this PR) Phase 2 — Outbound voice · "call me when it's done" (built; follow-up PR) Phase 3 — Video calls 🎥 (in progress in the worker) Phase 4 — OpenClaw animated avatar · lip-synced 🧑🚀 (active R&D) The endgame is a full see-and-be-seen Teams assistant: it hears you, talks back, calls you when work is done, watches your screen, and shows a lip-synced face while doing it — all over one authenticated bridge. Contributed by the Ports, Customs and Free Zone Corporation (PCFC), Dubai 🇦🇪 — building the "Virtual Employee" Teams assistant in production. |
Adds a 5th voice-call provider, `msteams`, that bridges Microsoft Teams call
audio into the existing voice-call machinery (CallManager, realtime
transcription, response generation, telephony TTS) via an external Windows
worker over an HMAC-authenticated WebSocket (inbound-only, PCM 16 kHz).
Transport & security:
- HMAC-SHA256 authenticated WebSocket upgrade (x-openclawteamsbridge-timestamp /
-signature over `${ts}.${callId}`), constant-time compare, replay window.
- Binds loopback by default (msteams.bindAddress); 64 KB max payload; total +
per-IP connection caps; pre-start idle reaper; callId integrity check.
- sharedSecret is SecretRef-compatible (registered in the credential matrix).
Two modes:
- Agent (streaming): caller audio -> realtime STT -> OpenClaw agent (tools) ->
TTS. Early-audio buffered until STT connects; STT connect failure tears the
call down; runtime fails fast if streaming is enabled without a usable STT.
- Realtime (speech-to-speech): caller bridged to a realtime voice provider, which
delegates real work to the OpenClaw agent via openclaw_agent_consult (inline)
and openclaw_agent_task (owner-policy background task that delivers the result
to the caller's Teams chat). 16k<->24k resampling; barge-in.
Recording-status gate (Microsoft Media Access API): media-derived data is not
processed/persisted before Teams recording status is active. The streaming path
drops raw audio + transcripts; the realtime path refuses consult/task and does
not retain pre-recording transcripts. requireRecordingStatus (default true) is
operator-overridable.
setup/status exempts msteams from the public-webhook-exposure check (it uses the
bridge listener, not the webhook plane). Docs: voice-call README provider
section. Restores packages/plugin-sdk/src/testing.ts to its canonical single
re-export (an upstream docs pass had tripped the package-contract guardrail).
Tests cover the transport (HMAC/caps/recording protocol), provider audio loop,
realtime consult/async + recording gate, runtime fail-fast, and webhook-exposure
exemption.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…o option Addresses the two author-side review findings on the inbound provider: - [P1] Teams allowlists now match AAD callers. isAllowlistedCaller matches by phone number (digits) OR by exact caller id (case-insensitive), so a Teams aadId is no longer phone-normalized into a non-matching value. The msteams paths (realtime isInboundCallAllowed + streaming shouldAcceptInbound) pass the raw caller id; carrier paths still match by phone. Tests cover AAD-id matching. - [P2] Wire the documented realtime echo option. realtime.suppressInputDuringPlayback is now a real config field (schema + manifest + threaded into the msteams realtime deps), matching the README; it was previously documented but only an internal default. oxlint/oxfmt/tsgo prod+test clean; allowlist + webhook + config suites green; base-config-schema unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Completes the allowlist fix. Runtime matching already accepted AAD caller ids, but config + manifest validation still required E.164 for every allowFrom entry, so `inboundPolicy: "allowlist"` was unusable for Teams (a GUID failed validation before runtime ever saw it). allowFrom entries now accept an E.164 phone number OR a Teams AAD object id (GUID) in both the Zod schema and the plugin manifest; README documents the AAD-id form. Carrier providers are unchanged (E.164). Tests cover the schema accepting phone + GUID (either case) and rejecting non-E.164/non-GUID entries. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Addresses the reviewer's inbound-posture decision: msteams previously defaulted inboundPolicy to "open", accepting every authenticated Teams caller out of the box. It now defaults to an allowlist-oriented policy — with an empty allowFrom no caller is accepted until the operator opts callers in (the allowlist accepts AAD object ids) or sets inboundPolicy: "open" explicitly. No unsafe default; explicit policies are honored unchanged. README + tests updated. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
[P3] Keep manifest provider metadata in sync — the provider enum includes "msteams" but the human-readable description still only listed Twilio/Telnyx/ Plivo. Mention Microsoft Teams voice so the manifest metadata matches. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
[P1/P2] The public voice-call docs described only Twilio/Telnyx/Plivo/mock, so the new msteams provider shipped undocumented. Add it to the summary and provider list and add a "Microsoft Teams voice (msteams)" section covering the external Windows-worker WebSocket bridge, streaming vs realtime modes, the safe allowlist default + AAD-id allowFrom, the recording-status gate, and the HMAC/loopback security model, linking to the extension README for the full protocol. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…ty in session keys Two source-review defects: - Socket-close teardown: a started Teams session that dropped without an explicit session.end frame (worker crash, network loss, hangup) only released local connection tracking — onSessionEnd never fired, so the provider/manager call record leaked until the stale-call reaper. The ws "close" handler now delivers onSessionEnd (reason "socket-closed") for a started-but-not-ended session, guarded against double-delivery when an explicit session.end preceded it. - Session-key caller identity: when a Teams caller has no aadId, `from` fell back to the shared literal "teams", so every anonymous caller collided into one session key (cross-caller memory bleed under sessionScope: per-phone) and would match an allowlist as a generic caller. Fall back to a per-call-unique `teams:<callId>` instead; the AAD id is still preferred when present. Tests cover abrupt-close onSessionEnd, no double-fire after session.end, no fire before session.start, and the per-call-unique session key. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…arget A realtime openclaw_agent_task delivers its result to the caller's Teams chat (message tool -> user:<aadId>). When the bridge session has no caller.aadId (anonymous caller), handleAsyncTask still acked "I'll message you on Teams" and ran the task — but runAsyncTask had no delivery target, so the promised result was silently dropped. Refuse the task up front when caller.aadId is absent, returning a message that tells the model to offer to answer on the call instead. Regression test covers the no-aadId refusal (no false delivery ack). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…rd Teams worker docs - The two new socket-close tests used `(resolve) => setTimeout(resolve, 100)`, whose arrow body implicitly returns the timer handle — oxlint no-promise-executor-return. Use a block body so nothing is returned. This was failing both check-lint and check-additional-extension-bundled (lint:extensions:bundled). - Docs: describe the Teams bridge as a "Windows-side bridge worker" rather than "an external Windows worker (not part of this repo)". Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
… posture Reviewer follow-up: the setup examples showed `inboundPolicy: "open"` as the headline, which undercuts the safe default. The README and public-docs snippets now use `inboundPolicy: "allowlist"` with a sample AAD `allowFrom` entry, and note `"open"` only as the explicit accept-all alternative. Matches the code/ schema default (allowlist) and the now-corrected PR description. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
Consolidating the Microsoft Teams CVI voice/video work into a single refreshed PR (rebased onto current upstream); the foundation PR #91438 stays open. Closing as superseded - its content/summary is carried into the consolidated PR. |
2 participants
Summary
What problem does this PR solve?
voice-callextension supports Twilio, Telnyx, Plivo, and a mock provider, but has no way to handle Microsoft Teams calls. Teams audio cannot reach the existing webhook/media-stream plane because Teams calls are delivered through Graph Calling signalling + theMicrosoft.Skype.Bots.MediaAudioSocket, which only run on .NET/Windows.msteams, that bridges Teams call audio into the existing voice-call machinery (CallManager, realtime transcription, response generation, telephony TTS).Why does this matter now?
What is the intended outcome?
provider: "msteams"and OpenClaw will accept per-call WebSocket connections from an external Windows worker ("OpenClawTeamsBridge" / AzureBot), authenticate them, and run the full inbound->STT->agent->TTS->outbound loop with barge-in - mirroring the streaming Twilio provider but speaking PCM 16 kHz instead of mu-law 8 kHz.What is intentionally out of scope?
Skype.Bots.MediaAudioSocket) is a separate component, not included in this repo. This PR only adds the OpenClaw-side provider + WebSocket transport it connects to.initiateCall.What does success look like?
provider: "msteams"binds an HMAC-authenticated WebSocket server; a live Teams call routed through the external worker produces a transcript, an agent reply, and synthesized speech back to the caller, with barge-in.What should reviewers focus on?
msteams-media-stream.ts(HMAC verification, replay window, payload cap, and loopback-by-default bind).config.ts: requiredport+ SecretRef-compatiblesharedSecret, thefromNumberexemption, andinboundPolicydefaulting to a safe "allowlist".runtime.ts: the media server starts via an awaitedprovider.start()(a failed bind aborts init) and stops on teardown so the port is released.generateVoiceResponse,CallManager.speak, telephony TTS) rather than duplicating webhook logic.Linked context
Which issue does this close?
Closes #
Which issues, PRs, or discussions are related?
Related #
Was this requested by a maintainer or owner?
Real behavior proof (required for external PRs)
voice-callhad no Microsoft Teams provider -provider: "msteams"was rejected as unsupported. This PR adds the provider, the HMAC-authenticated WebSocket transport, and the inbound->STT->agent->TTS->outbound pipeline.voice-callprovidermsteams; agentmicrosoft-foundry/gpt-5.4; ElevenLabs realtime STT (scribe_v2_realtime,pcm_16000) + ElevenLabs TTS; an external Windows worker bridged a real inbound Microsoft Teams call.pnpm buildnode openclaw.mjs config validate-> printsConfig validnode openclaw.mjs gateway run --force(a) Gateway lifecycle log during the call (private identifiers redacted):
sessionKey: voice:<redacted>, modelmicrosoft-foundry/gpt-5.4) - the inbound caller audio was transcribed by ElevenLabs STT and answered by the agent; each{"spoken":...}reply is what was synthesized back over the bridge as TTS the caller heard:gpt-5.4agent reply -> ElevenLabs TTS played back to the caller, across multiple turns over ~70 s. The speech disfluencies ("uh", "Mm-hmm") and natural turn-taking confirm these are live ASR transcripts (not typed input); the{"spoken":...}payloads are exactly what the provider sends to TTS. Session lifecycle (connection open/session.start/connection closed) is verified in the gateway log.provider: "msteams"producedUnsupported voice-call provider: msteams.After-fix proof: realtime delegation + recording-status gate (latest commits)
Captured on real inbound Microsoft Teams calls using the realtime (speech-to-speech) path (
realtime.enabled, provideropenai,toolPolicy: "owner"), with the recording gate at its default (requireRecordingStatus: true). Private identifiers redacted; call ids shortened.Recording gate refuses, then opens. With no recording signal, asking the assistant to do work was refused out loud ("recording isn't active") and the gateway logged no
recording.status. Once the external worker reports recording active on call establish, the gateway opens the gate:Inline consult (
openclaw_agent_consult). With the gate open, the caller asked a question; the model spoke the "working on it" filler and the OpenClaw agent ran and replied through the realtime model -- confirmingrealtime model -> openclaw_agent_consult -> agent -> spoken reply.Background task (
openclaw_agent_task) -> Teams chat delivery. The caller said "take care of local time in the background and ping me on Teams." The model invoked the task, spoke the ack ("I'll message you on Teams"), the caller hung up, the agent ran the task in the background, and delivered the result to the caller's Teams chat:The two messages received in the caller's Teams chat:
The agent did real work (read the actual machine clock --
05:28:13, matching the run time), not a canned reply.Pre-recording transcripts are not retained:
recordTranscriptdrops turns while the gate is closed, so the consult context only ever contains recording-active text (regression test inmsteams-realtime.test.ts).Tests and validation
Which commands did you run?
Result:
node openclaw.mjs config validate->Config valid.What regression coverage was added or updated?
extensions/voice-call/src/providers/msteams.test.ts- provider lifecycle: session.start -> CallManager registration, transcript -> response, barge-in (assistant.cancel), playTts framing/pacing, hangup/teardown.extensions/voice-call/src/msteams-media-stream.test.ts- WebSocket server: HMAC upgrade verification (accept/reject), replay-window rejection,session.start/session.end/audio.frame/pingparsing, outboundsend/close, and inbound payload-cap rejection (oversized frame -> close1009, added for GHSA-vw3h-q6xq-jjm5).What failed before this fix, if known?
provider: "msteams"was rejected as an unsupported voice-call provider; there was no transport for Teams audio.If no test was added, why not? N/A - unit tests added for both the provider and the transport.
Risk checklist
Did user-visible behavior change?
Yes (additive, opt-in)- adds a newmsteamsprovider option; existing Twilio/Telnyx/Plivo/mock behavior is unchanged. For msteams specifically,inboundPolicydefaults to a safe"allowlist"(Teams is inbound-first): no caller is accepted until the operator opts callers in (the allowlist accepts AAD object ids) or sets"open"explicitly.Did config, environment, or migration behavior change?
Yes- addsvoice-callconfig underprovider: "msteams"(msteams.{port,bindAddress,path,sharedSecret}, plusstreamingandttsblocks).msteams-scoped config/validation behavior inconfig.ts:msteamsis exempt from thefromNumberrequirement (it never places outbound PSTN calls).inboundPolicydefaults to a safe"allowlist"formsteamswhen unset.provider: "msteams",validateProviderConfignow requiresmsteams.portandmsteams.sharedSecret(so a misconfigured runtime fails validation instead of starting but never binding).msteams.pathhas a schema default (/voice/msteams/stream).msteams.sharedSecretaccepts the sameSecretInputshape astwilio.authToken(string or${ENV}/secret ref), resolved viaresolveMsteamsSharedSecretand registered inconfigContracts.secretInputs.No data migration.
Did security, auth, secrets, network, or tool execution behavior change?
Yes- introduces a WebSocket server (operator-configured port) that accepts connections from the external worker.What is the highest-risk area? The WebSocket transport (
msteams-media-stream.ts) - it listens on a network port and accepts external connections carrying call audio.How is that risk mitigated?
127.0.0.1unless the operator setsmsteams.bindAddressto a specific trusted-network interface - it is never exposed on all interfaces implicitly.timestamp + callIdusing a shared secret, compared in constant time (safeEqualSecret).zoddiscriminated union.SecretInput(supports${ENV}/secret refs), required by config validation, and never logged.Current review state
What is the next action?
What is still waiting on author, maintainer, CI, or external proof?
msteamsships bundled invoice-callor as a separate optional plugin (see Product boundary).Which bot or reviewer comments were addressed?
msteams.bindAddress; it no longer implicitly binds all interfaces.provider: "msteams"now fails validation withoutmsteams.port+msteams.sharedSecret; the secret isSecretInput/SecretRef-compatible and registered inconfigContracts.secretInputs(so it no longer starts a runtime that cannot accept calls safely).maxPayloadon inbound frames; regression test (oversized frame -> close1009).Product boundary (maintainer decision)
VISION.mdsets a high bar for bundling optional/commercial integrations, while the docs statevoice-callowns call transport. This PR is written to be additive and opt-in and to keep that decision easy either way:provider: "msteams"; default behavior of Twilio/Telnyx/Plivo/mock is unchanged, and nothing activates unless an operator configures it.ws,zod, and existing plugin-sdk helpers) and no commercial SDK - Teams-specific Graph/Skype.Bots.Mediacode lives entirely in the out-of-repo Windows worker.providers/msteams.ts,msteams-media-stream.ts,msteams-tts.ts,msteams.runtime.ts) with only thin, guarded hooks in shared files, so it can be lifted into a separate optional plugin if maintainers prefer that over bundling.Author is happy to follow maintainer preference on bundled-vs-external packaging.
Files changed (feature scope)
extensions/voice-call/src/providers/msteams.tsmsteamsprovider: explicit awaitedstart()(binds the WS server; replaces constructor fire-and-forget), inbound audio -> STT, response generation, TTS playback with paced PCM framing, barge-in, CallManager integration,stop()for teardown.extensions/voice-call/src/providers/msteams.test.tsawait provider.start(); connect over127.0.0.1).extensions/voice-call/src/msteams-media-stream.tsbindAddress; session.start/end/audio.frame/ping protocol; 64 KB payload cap; outbound send/close.extensions/voice-call/src/msteams-media-stream.test.ts127.0.0.1).extensions/voice-call/src/msteams-tts.tsextensions/voice-call/src/msteams.runtime.tsextensions/voice-call/src/runtime.tsbindAddress+ resolvedsharedSecret; starts the WS server as an awaited runtime-init step and registersprovider.stop()in the resource lifecycle (bind failures surface; port released on teardown).extensions/voice-call/src/config.tsMsteamsConfigSchema(port/bindAddress/path(default)/sharedSecretasSecretInput);resolveMsteamsSharedSecret; msteams exempt fromfromNumber;inboundPolicydefaults to a safe"allowlist"; requiresmsteams.port+msteams.sharedSecret.extensions/voice-call/openclaw.plugin.jsonmsteams.bindAddressandsharedSecret(string/object); registersmsteams.sharedSecretinconfigContracts.secretInputs.