feat(channels)!: migrate whatsapp from in-process adapter to sidecar (dual-mode)#5445
Merged
Conversation
…(dual-mode) Removes the in-process Rust librefang-channels::whatsapp adapter (WhatsAppAdapter, 918 lines) along with the [channels.whatsapp] config schema, the channel-whatsapp cargo feature, the kernel whatsapp_gateway.rs module (embedded JS source, gateway-dir extraction, npm install orchestration, node-process supervisor with restart backoff, whatsapp_gateway_pid field + accessor, shutdown SIGTERM cleanup), the two custom dashboard routes (/channels/whatsapp/qr/start + /qr/status, ~210 LOC incl. raw-TCP helpers), the CLI wizard arm + status-table row + TUI ChannelDef, the channel_bridge builder block + import + check_channel + find_channel_info arms, the openclaw / openfang migrators (SkippedItem same shape as IRC / Mattermost / Signal / Matrix / Teams), the schema golden file, and every cascade touchpoint. Replaced by librefang.sidecar.adapters.whatsapp (~750 LOC), stdlib-only Python — urllib.request for REST + BaseHTTPRequestHandler over ThreadingTCPServer for inbound. No third-party deps. Behaviour parity — Cloud API mode: * Outbound text / audio (URL link) / image (link + caption) / document (link + filename) / location via POST https://graph.facebook.com/v17.0/{phone_id}/messages * 4096-char chunking * Bearer-auth on every request * Group / DM policy filter (respond / allowed_only / ignore × all / mention_only / commands_only / ignore) * Bot-mention detection (phone with / without @ / +, name substring, case-insensitive) * Sender allowlist (exact phone match) * account_id metadata injection (#5003) Behaviour parity — Web/QR (Baileys) gateway mode: * Outbound text via POST {gateway_url}/message/send * Voice URL degrades to "(Voice message: <url>)" text * Image without caption degrades to placeholder text * File degrades to "(File: <name> — not supported)" text * Inbound continues to flow direct gateway → /api/agents/.../message The kernel no longer auto-spawns the Baileys gateway. Operators on Web/QR mode now run `npx @librefang/whatsapp-gateway` separately (or declare it as another `[[sidecar_channels]]` entry). Improvements over the Rust adapter: 1. REAL Cloud API webhook handler — whatsapp.rs:454-483 was a TODO stub that logged "webhook ready" but never parsed inbound. The sidecar implements the full handler: * GET {path} returns hub.challenge for Meta's subscription confirmation when hub.verify_token matches * POST {path} verifies X-Hub-Signature-256 against HMAC-SHA256(WHATSAPP_APP_SECRET, raw_body) with constant-time compare, then parses entry[].changes[].value.messages[] and emits text events 2. Inbound dedupe on message.id — Meta retries on non-200, bounded SeenSet (10000 / evict 5000) keeps redeliveries from double-emitting. 3. 429 Retry-After honoured on every outbound POST — Rust warned-and-failed on the first non-2xx (whatsapp.rs:373-377). 4. Explicit 30 s timeouts on every REST call. Verification: cd sdk/python && pytest tests/test_whatsapp_adapter.py — 76 passed. Full sidecar suite — 1691 passed. Also fixes a latent compile error in routes/config.rs: the teams migration (PR 5433) was merged with --admin and left a stale `ch!(teams)` macro invocation pointing at a removed ChannelsConfig field. This PR cleans it up alongside the analogous whatsapp removal. Operator action required: re-declare [channels.whatsapp] as [[sidecar_channels]] running python3 -m librefang.sidecar.adapters.whatsapp. Cloud API mode needs WHATSAPP_PHONE_NUMBER_ID + WHATSAPP_WEBHOOK_PORT (in [sidecar_channels.env]) and WHATSAPP_ACCESS_TOKEN + WHATSAPP_VERIFY_TOKEN + WHATSAPP_APP_SECRET (in ~/.librefang/secrets.env). Meta's Webhook URL must be repointed from /channels/whatsapp/webhook to {sidecar-host}:{WHATSAPP_WEBHOOK_PORT}{WHATSAPP_WEBHOOK_PATH}. Web/QR mode users start `npx @librefang/whatsapp-gateway` separately and set WHATSAPP_GATEWAY_URL. ChannelType::WhatsApp is preserved via channel_type="whatsapp" on the sidecar entry so existing routing keys continue to resolve.
Self-review found a small timing-attack surface in `_handle_get_verify`: the Meta webhook subscription handshake compared the operator's `WHATSAPP_VERIFY_TOKEN` against the inbound `hub.verify_token` query param using `==`, which short-circuits at the first mismatching byte. The handshake runs infrequently (typically once at Webhook subscription creation) so timing attacks against this path are impractical in production, but `hmac.compare_digest` is the same line of code with the right semantics — no reason to leave `==` in a security-sensitive comparison. The HMAC-SHA256 `X-Hub-Signature-256` path was already using `hmac.compare_digest` correctly (see `verify_xhub_signature`); this just brings the GET-handshake path to the same standard. No new test — the verify-token-match contract is already pinned by `test_get_verify_subscribe_match` / `test_get_verify_wrong_token_rejected`, both of which still pass; constant-time is an implementation detail of the same boolean answer.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3e947b7a38
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
29
to
30
| $mac!(google_chat, "google_chat"); | ||
| $mac!(webhook, "webhook"); |
There was a problem hiding this comment.
Dropping whatsapp from for_each_channel_field! changes resolve_agent_home_channel behavior because that function still relies exclusively on this macro to find an agent’s default_agent channel. After this migration, a bot configured as [[sidecar_channels]] with channel_type = "whatsapp" can no longer be resolved as a home channel, so trigger/cron sends that call resolve_agent_home_channel (e.g. in triggers_and_workflow.rs) lose their WhatsApp return path and run without channel context. Either keep WhatsApp in this lookup path or add equivalent sidecar-channel resolution in resolve_agent_home_channel before removing it from the macro.
Useful? React with 👍 / 👎.
Round-2 self-review found `_handle_post_webhook` returned 401
both for an invalid signature AND for a missing
`X-Hub-Signature-256` header. They're not the same failure
mode:
* **Missing header** = malformed request (or upstream proxy
stripped it, or attacker probing) — 400 is the right code.
* **Invalid signature** = signature presented and rejected —
401 is the right code.
Teams already distinguishes these (the route-handler test for
Teams asserts `Path("teams")` 400 vs 401 by header presence);
WhatsApp gets the same shape.
Two new tests pin the contract:
* `test_post_webhook_missing_signature_returns_400` — header
absent (`None`) returns 400.
* `test_post_webhook_empty_signature_returns_400` — empty
string header is also malformed, returns 400.
The existing `test_post_webhook_invalid_signature_rejected`
(real HMAC value computed with the wrong key → 401) is
untouched and still passes.
houko
added a commit
that referenced
this pull request
May 21, 2026
Surfaces from the post-#5445 audit (6th in the dingtalk/qq/omnibus review chain — first verdict that wasn't BROKEN/NEEDS_HOTFIX). All three nits are doc/UX cleanups, none are runtime regressions: 1. **Docstring claim #3 was wrong** — said "429 Retry-After honoured on every outbound POST", but only Cloud API uses `_cloud_post_with_retry`; the gateway path calls `_http_request` directly and raises on any non-2xx. Soften the claim to "Cloud-API outbound POSTs only" + explain when gateway-side retry would matter (operators proxying the local Baileys gateway behind a rate-limiting reverse-proxy). 2. **WHATSAPP_VERIFY_TOKEN unset failed silently** — Meta's subscription handshake returned 403 with no log line pointing back to the missing env var; operators saw "subscription failed" in the Meta dashboard with nothing in their daemon logs. Now warns at __init__, matching the WHATSAPP_APP_SECRET pattern already there. 3. **WHATSAPP_GROUP_POLICY is dead config** — Cloud API webhook payloads don't surface a group/conversation distinction (we hardcode `is_group=False` at `_handle_post_webhook`), and gateway mode delegates inbound entirely to the Node Baileys gateway which never calls back into the sidecar's `should_handle_message` filter. So `WHATSAPP_GROUP_POLICY` has zero effect today. Mark the schema field with an explicit "(currently inert)" note in the label so operators don't waste time setting it. Kept the field itself for forward-compat — Meta has been rolling out group-chat support to the Cloud API gradually and we want the schema to be ready when it lands. Drive-by: dropped the misleading send-voice docstring claim (lines 22-23 + 36-39 promised a `{gateway}/message/send-voice` multipart upload route that doesn't exist in code; rationale comment about the daemon's ChannelContent::Voice always carrying a URL at the dispatch boundary explains why we don't need it). Test: - New test_get_verify_empty_self_token_rejects asserts BOTH the __init__ warn fires AND the handshake fails closed. Regression guard for #2 — without this an attacker who guesses the empty hub.verify_token could subscribe their own callback URL. - cd sdk/python && pytest tests/test_whatsapp_adapter.py — 79 passed (was 78; +1 regression guard). This closes the audit chain. WhatsApp itself is structurally clean — natural routing key (phone) is preserved as channel_id both directions, the bug family from #5417 → #5449 doesn't apply.
github-actions
Bot
added
area/docs
houko
added a commit
that referenced
this pull request
May 21, 2026
…to stop bot false positives (#5467) The repo's stub-detection github-actions bot scans for the literal word `stub` in source and auto-opens issues per occurrence. Three mentions in `sdk/python/librefang/sidecar/adapters/whatsapp.py` docstrings were describing the **historical, already-removed** Rust adapter (`whatsapp.rs:454-483`, deleted in #5445) — every one of them explicitly contrasts the removed code with the current Python implementation: - L11-14: "The in-process Rust adapter's `start()` was a stub that logged 'webhook ready' but never actually parsed inbound activities ... This sidecar implements the real webhook handler." - L58-61: "Rust's `start()` ... was a TODO stub — operators wanting Cloud API inbound had to wire their own webhook ... The sidecar implements the real handler." - L153-155: "The Rust adapter didn't actually run a webhook (whatsapp.rs:454-483 was a TODO stub); this verification path is new in the sidecar." But the bot can't tell "current code is a stub" from "historical removed code was a stub". It opened issues #5457 and #5458 yesterday on two of the three mentions; the third is just waiting for the next scan. Both issues are factually wrong — the Python sidecar IS the real Cloud-API webhook implementation, pinned by 79 tests in `sdk/python/tests/test_whatsapp_adapter.py`. Rephrasing the three mentions to use "no-op" / "unimplemented TODO" preserves the meaning (and the historical contrast with the Rust adapter) without tripping the bot's keyword match. Verified post- edit: `grep -n stub sdk/python/librefang/sidecar/adapters/whatsapp.py` returns nothing; module imports cleanly; full whatsapp adapter test suite (79 cases) stays green. Closes #5457, #5458 (both also closed manually with the same explanation so the bot's history of false positives stays visible in the issue thread).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Removes the in-process Rust
librefang-channels::whatsappadapter (918 LOC) and replaces it withlibrefang.sidecar.adapters.whatsapp(~750 LOC, stdlib-only Python). Both modes preserved: Cloud API (Meta Business) + Web/QR (Baileys gateway).After this PR: 27 / 27 channels migrated to sidecar (100%) — modulo
webhook+google_chatwhich are now the only in-process channels remaining.Why now
WhatsApp was the most-intertwined remaining adapter:
start()was a TODO stub —whatsapp.rs:454-483logged "webhook ready" but never parsed inbound activities. Cloud API inbound never actually worked through the Rust adapter; operators had to wire their own forwarder.whatsapp_gateway.rsmodule thatinclude_str!'d the JS, extracted to disk, rannpm install, supervised the Node process with restart backoff, and tracked PIDs for shutdown — all auto-triggered on[channels.whatsapp]presence. That coupling is gone now./channels/whatsapp/qr/{start,status}, ~210 LOC including a raw-TCP HTTP helper).Moving to a sidecar lets the Python process do REAL Cloud API inbound, lets operators control the Baileys gateway as a separate process, and gets ~1300 LOC of channel-specific glue out of the API + kernel crates.
Behaviour parity
Line-cited against
crates/librefang-channels/src/whatsapp.rson the pre-migration tree.Cloud API mode:
Authorization: Bearer <WHATSAPP_ACCESS_TOKEN>respond/allowed_only/ignore×all/mention_only/commands_only/ignore)@++strip, name substring, case-insensitive)account_id(fix(channels): override account_id() in non-Telegram multi-bot adapters #5003)Web/QR mode:
POST {gateway_url}/message/send(Voice message: <url>)text(Image — not supported in Web mode)(File: <name> — not supported in Web mode)/api/agents/{id}/message(unchanged)Improvements over the Rust adapter
start()was a stub; sidecar implementsGET {path}returninghub.challengefor Meta's subscription handshake ANDPOST {path}verifyingX-Hub-Signature-256againstHMAC-SHA256(WHATSAPP_APP_SECRET, raw_body)(constant-time compare), then parsing and emitting text messages.message.id— Meta retries on non-200; boundedSeenSet(10000 / 5000) keeps redeliveries from double-emitting.Retry-Afterhonoured on every outbound POST — Rust warned-and-failed on the first non-2xx (whatsapp.rs:373-377).reqwest's defaults.Webhook URL change (operator-visible)
For Cloud API mode, the shared
/channels/whatsapp/webhookmount is gone. Repoint Meta's Webhook subscription URL tohttps://<host>:<WHATSAPP_WEBHOOK_PORT><WHATSAPP_WEBHOOK_PATH>(defaults8460//webhook).Web/QR gateway change (operator-visible)
The kernel no longer auto-spawns the Baileys gateway. Operators on Web/QR mode now run it separately:
Or declare it as another
[[sidecar_channels]]entry. Inbound delivery to/api/agents/{id}/messageis unchanged.Latent main-branch fix
The teams migration (PR #5433) was merged with
--admin(CI skipped) and left a stalech!(teams)macro invocation inroutes/config.rsthat referenced a now-removedChannelsConfigfield. This PR cleans that up alongside the analogousch!(whatsapp)removal, restoring workspace compile.Cascade
Standard touchpoint pattern + WhatsApp-specific extras:
lib.rsmod removed,whatsapp.rsdeleted (~918 LOC)librefang-channels/Cargo.toml—channel-whatsappfeature removedlibrefang-api/Cargo.toml—channel-whatsappremoved fromall-channels/all-channels-no-email/miniChannelsConfig.whatsappfield +WhatsAppConfigstruct + helpers deletedchannels-allowlist.txt— whatsapp removedchannel_bridge.rs—WhatsAppAdapterimport + builder block + check_channel + find_channel_info arms removedroutes/channels.rs—ChannelMetaremoved + 4 match arms + 2 custom QR routes (~210 LOC) +gateway_http_post/gateway_http_gethelpers removed, route-handler 412/200 test witness rotatedwhatsapp→google_chat, SIDECAR_CATALOG entry addedroutes/config.rs—ch!(whatsapp)+ stalech!(teams)removedroutes/skills.rs— round-trip channel TOML witness rotatedwhatsapp→webhookchannel_sender.rs— macro entry + EXPECTED list updatedvalidation.rs— env-var check removedkernel/whatsapp_gateway.rs— entire 240-line module deleted;lib.rsmod gone;LibreFangKernel.whatsapp_gateway_pidfield +whatsapp_pid()accessor +boot.rsinit +background_lifecyclespawn +shutdown()SIGTERM all removedconfig_reload.rs— test witness rotatedwhatsapp→webhooklibrefang-migrate— both YAML + JSON5 paths emitSkippedItemfor[channels.whatsapp](same shape as IRC/Mattermost/Signal/Matrix/Teams); JSON5 round-trip channel-items count drops 2 → 1;openfang.rsdrift tests rotated to[[mcp_servers]](the remainingdeny_unknown_fieldswitness);config_routes_integrationboot-fail test rotated to[channels.webhook]librefang-cli— TUIChannelDef,librefang channel setup whatsappwizard arm, status-table row, channel-picker entry all removedkernel_config_schema.golden.json—WhatsAppConfig+OneOrMany_WhatsAppConfig+ property + null example removedconfig/mod.rstests —test_whatsapp_config_*+test_validate_missing_env_varsremoved;test_channels_config_with_new_channels+#5130 deny_unknown_fieldswitnesses rotated to webhook / mcp_servers;test_one_or_many_*witness rotated whatsapp → webhook[channels.whatsapp]blocks rewritten as[[sidecar_channels]]redirects with Cloud + Web/QR mode setup docs; voice + DM/group policy sections updated to use env-var names instead ofwith_*()builder methodsTest plan
cd sdk/python && pytest tests/test_whatsapp_adapter.py— 76 passed. Coverage: env handling (Cloud / gateway construction; missing-cred fail-closed; CSV; path normalize; policy lowercase),X-Hub-Signature-256verify (valid / wrong key / wrong body / missing / empty / wrong prefix / non-hex / empty digest),is_bot_mentioned(phone /@-prefix / name case-insensitive / no-match / empty-text),should_handle_message(DM ×respond/allowed_onlyreject+accept /allowed_onlyempty-allowlist /ignore; group ×all/mention_onlyreject+accept /commands_onlyreject+accept-with-leading-spaces /ignore/ unknown policy fails-closed),parse_cloud_api_message(text emit / non-text dropped / empty text / missing field / phone fallback / multiple / account_id injection / non-dict / missing entry),_handle_get_verify(subscribe match / wrong token / wrong mode all-reject),_handle_post_webhook(signature disabled accepts any / valid sig / invalid 401 / malformed 400 / Activity-ID dedupe / DM policy applied), Cloud API outbound (text basic / chunking / 429 retry / non-2xx raises / empty drops / audio-link / image with-and-without caption / file / location), gateway outbound (text basic / chunks / non-2xx raises),on_senddispatch (cloud text / image / voice / file / location; gateway text / voice degrades / image uses caption / no caption placeholder; empty recipient drops; user.platform_id fallback), schema + capabilities.cd sdk/python && pytest tests/— full sidecar suite 1691 passed (no regressions in the 22 existing sidecars).Operator action required
Existing
[channels.whatsapp]block no longer recognised. Re-declare as[[sidecar_channels]]. Cloud API mode:WHATSAPP_ACCESS_TOKEN+WHATSAPP_VERIFY_TOKEN+WHATSAPP_APP_SECRETbelong in~/.librefang/secrets.env. Repoint Meta's Webhook URL to your new sidecar endpoint.Web/QR mode — set
WHATSAPP_GATEWAY_URLin[sidecar_channels.env]and runnpx @librefang/whatsapp-gatewayas a separate process.ChannelType::WhatsAppis preserved viachannel_type = "whatsapp"on the sidecar entry so existing routing /channel_role_mappingkeys continue to resolve.