Skip to content

Add WhatsApp approval QA scenarios#87782

Merged
kevinslin merged 2 commits into
openclaw:mainfrom
kevinslin:dev/kevinlin/whatsapp-approval-qa-scenarios
May 28, 2026
Merged

Add WhatsApp approval QA scenarios#87782
kevinslin merged 2 commits into
openclaw:mainfrom
kevinslin:dev/kevinlin/whatsapp-approval-qa-scenarios

Conversation

@kevinslin

@kevinslin kevinslin commented May 28, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

  • Add WhatsApp QA Lab equivalents for the Slack native approval scenarios: whatsapp-approval-exec-native and whatsapp-approval-plugin-native.
  • Enable the scenarios through top-level session approval forwarding and verify the native pending/resolved WhatsApp approval messages.
  • Keep the new native approval scenarios explicit-only so omitted or empty WhatsApp QA selections still run the standard canary/pairing/mention set.
  • Document the WhatsApp QA lane, credential env vars, scenarios, and artifacts.

Verification

  • node scripts/run-vitest.mjs extensions/qa-lab/src/live-transports/whatsapp/whatsapp-live.runtime.test.ts extensions/qa-lab/src/live-transports/whatsapp/whatsapp-boundary.test.ts extensions/qa-lab/src/live-transports/shared/live-transport-scenarios.test.ts extensions/whatsapp/src/approval-auth.test.ts extensions/whatsapp/src/approval-handler.runtime.test.ts extensions/whatsapp/src/approval-native.test.ts extensions/whatsapp/src/approval-reactions.test.ts
    • 7 files passed, 57 tests passed after rebasing onto current origin/main.
  • OPENCLAW_TSGO_HEAVY_CHECK_LOCK_HELD=1 node scripts/run-tsgo.mjs -p test/tsconfig/tsconfig.extensions.test.json --incremental false
    • Passed.
  • OPENCLAW_OXLINT_SKIP_LOCK=1 OPENCLAW_OXLINT_SHARD_HEARTBEAT_MS=0 node scripts/run-oxlint-shards.mjs --only extensions
    • Passed.
  • CI=true pnpm_config_confirm_modules_purge=false pnpm_config_minimum_release_age=0 OPENCLAW_HOME=/private/tmp/openclaw-whatsapp-approval-qa-home-prebuild node scripts/run-node.mjs qa whatsapp --help
    • Rebuilt the private QA CLI and printed WhatsApp QA help. Known Baileys optional jimp warning was treated as external and the command exited 0.
  • git diff --check
    • Exited 0.
  • WhatsApp QA Lab live run:
    • Output: .artifacts/qa-e2e/whatsapp-approval-env-20260528-1310/whatsapp-qa-summary.json
    • Counts: total 2, passed 2, failed 0, skipped 0.

Real behavior proof

Behavior addressed: WhatsApp now has QA Lab coverage equivalent to slack-approval-exec-native and slack-approval-plugin-native, proving native WhatsApp exec and plugin approval prompts can be requested, observed, resolved, and matched.

Real environment tested: Local OpenClaw worktree with the WhatsApp QA Lab lane, real pre-linked WhatsApp Web QA credentials sourced from the Convex credential payload in memory, and mock-openai provider mode.

Exact steps or command run after this patch: Ran the private QA CLI against --scenario whatsapp-approval-exec-native --scenario whatsapp-approval-plugin-native with --credential-source env, after mapping the active Convex WhatsApp credential payload into OPENCLAW_QA_WHATSAPP_* variables in the child process without printing the payload.

Evidence after fix: .artifacts/qa-e2e/whatsapp-approval-env-20260528-1310/whatsapp-qa-summary.json reports total: 2, passed: 2, failed: 0, skipped: 0.

Copied live output from the redacted summary artifact:

{
  "counts": {
    "total": 2,
    "passed": 2,
    "failed": 0,
    "skipped": 0
  },
  "scenarios": [
    {
      "id": "whatsapp-approval-exec-native",
      "status": "pass",
      "details": "exec approval whatsapp-qa-exec-a2151c2f-0225-4f13-b5ef-387bde27837b resolved allow-once in 838ms"
    },
    {
      "id": "whatsapp-approval-plugin-native",
      "status": "pass",
      "details": "plugin approval plugin:72620973-7ebb-42cd-97ae-935287bd46ca resolved allow-once in 494ms"
    }
  ]
}

Observed result after fix: whatsapp-approval-exec-native passed with allow-once resolution in 838ms, and whatsapp-approval-plugin-native passed with allow-once resolution in 494ms. whatsapp-qa-observed-messages.json includes pending and resolved matched messages for both scenarios.

What was not tested: The full all-scenario WhatsApp lane was not rerun after this patch; the live run was scoped to the two new approval scenarios.

@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation extensions: qa-lab size: L maintainer Maintainer-authored PR labels May 28, 2026
@clawsweeper

clawsweeper Bot commented May 28, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed May 28, 2026, 6:09 PM ET / 22:09 UTC.

Summary
Adds two explicit WhatsApp QA Lab native approval scenarios, their runtime selection/tests, and WhatsApp QA documentation.

PR surface: Source +418, Tests +79, Docs +42. Total +539 across 3 files.

Reproducibility: not applicable. This PR adds QA coverage rather than reporting a current-main bug. The contributor did provide a high-confidence live WhatsApp QA run for the two new scenarios.

Review metrics: 1 noteworthy metric.

  • QA Scenarios Added: 2 explicit-only scenarios added. The change extends WhatsApp approval coverage while keeping the default WhatsApp QA lane stable.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Next step before merge

  • No automated repair remains; the protected maintainer label and normal landing checks are the remaining action.

Security
Cleared: Cleared: the patch adds QA runner/docs/tests only, reuses existing safe archive handling and redacted artifacts, and does not change dependencies, workflows, lockfiles, or credential storage.

Review details

Best possible solution:

Land after maintainer approval and current checks, preserving the explicit-only WhatsApp approval scenarios alongside the existing Slack equivalents.

Do we have a high-confidence way to reproduce the issue?

Not applicable: this PR adds QA coverage rather than reporting a current-main bug. The contributor did provide a high-confidence live WhatsApp QA run for the two new scenarios.

Is this the best way to solve the issue?

Yes: the implementation follows the existing Slack native approval QA pattern, keeps approval scenarios explicit-only, and adds focused tests plus docs for the new lane behavior.

AGENTS.md: found and applied where relevant.

Codex review notes: model gpt-5.5, reasoning high; reviewed against 4287cd2e6e35.

Label changes

Label justifications:

  • P2: This is a normal-priority QA coverage improvement for a bounded live transport lane, with limited blast radius.
  • rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): Sufficient: the PR body includes copied live output from a redacted WhatsApp QA summary showing both new approval scenarios passing in a real WhatsApp QA lane.
  • proof: sufficient: Contributor real behavior proof is sufficient. Sufficient: the PR body includes copied live output from a redacted WhatsApp QA summary showing both new approval scenarios passing in a real WhatsApp QA lane.
Evidence reviewed

PR surface:

Source +418, Tests +79, Docs +42. Total +539 across 3 files.

View PR surface stats
Area Files Added Removed Net
Source 1 432 14 +418
Tests 1 79 0 +79
Docs 1 42 0 +42
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 3 553 14 +539

What I checked:

Likely related people:

  • @steipete: The current WhatsApp QA runtime, docs, and WhatsApp native approval handling appear to have been introduced in b5d90ae, and shortlog shows the heaviest history on the affected files. (role: feature-history owner; confidence: high; commits: b5d90ae4ec39; files: extensions/qa-lab/src/live-transports/whatsapp/whatsapp-live.runtime.ts, extensions/whatsapp/src/approval-handler.runtime.ts, docs/concepts/qa-e2e-automation.md)
  • Aamir Jawaid: Recently touched the WhatsApp QA runtime in 04c2982 while updating adjacent live-transport patterns. (role: recent adjacent contributor; confidence: medium; commits: 04c29825356f; files: extensions/qa-lab/src/live-transports/whatsapp/whatsapp-live.runtime.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed May 28, 2026
View reviewed changes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 76a2d4ecef

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread extensions/qa-lab/src/live-transports/whatsapp/whatsapp-live.runtime.ts
@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 automation 🚨 May affect CI, automerge, proof capture, label sync, or maintainer automation. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels May 28, 2026
@kevinslin

kevinslin commented May 28, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented May 28, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@clawsweeper clawsweeper Bot added rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. merge-risk: 🚨 automation 🚨 May affect CI, automerge, proof capture, label sync, or maintainer automation. labels May 28, 2026
@kevinslin
kevinslin force-pushed the dev/kevinlin/whatsapp-approval-qa-scenarios branch from 0975ad2 to 49743d2 Compare May 28, 2026 21:54
@openclaw-barnacle openclaw-barnacle Bot added commands Command implementations extensions: elevenlabs labels May 28, 2026
@kevinslin
kevinslin force-pushed the dev/kevinlin/whatsapp-approval-qa-scenarios branch from 49743d2 to 083bf94 Compare May 28, 2026 22:03
@openclaw-barnacle openclaw-barnacle Bot removed commands Command implementations extensions: elevenlabs labels May 28, 2026
@kevinslin kevinslin self-assigned this May 28, 2026
@kevinslin
kevinslin merged commit 359c31b into openclaw:main May 28, 2026
95 of 99 checks passed
@kevinslin
kevinslin deleted the dev/kevinlin/whatsapp-approval-qa-scenarios branch May 28, 2026 22:29
@github-actions github-actions Bot mentioned this pull request May 28, 2026
Open
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 29, 2026
@kevinslin
Add WhatsApp approval QA scenarios (openclaw#87782)
0b8deaa 
* test(qa): add WhatsApp approval scenarios

* fix(qa): keep WhatsApp approval scenarios explicit
steipete pushed a commit that referenced this pull request May 29, 2026
@kevinslin @steipete
Add WhatsApp approval QA scenarios (#87782)
bee0a31 
* test(qa): add WhatsApp approval scenarios

* fix(qa): keep WhatsApp approval scenarios explicit
SYU8384 pushed a commit to SYU8384/openclaw that referenced this pull request Jun 3, 2026
@kevinslin
Add WhatsApp approval QA scenarios (openclaw#87782)
7032c90 
* test(qa): add WhatsApp approval scenarios

* fix(qa): keep WhatsApp approval scenarios explicit
sablehead pushed a commit to sablehead/openclaw that referenced this pull request Jun 10, 2026
@kevinslin
Add WhatsApp approval QA scenarios (openclaw#87782)
606410c 
* test(qa): add WhatsApp approval scenarios

* fix(qa): keep WhatsApp approval scenarios explicit
@clawsweeper clawsweeper Bot mentioned this pull request Jul 1, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chatgpt-codex-connector chatgpt-codex-connector[bot] chatgpt-codex-connector[bot] left review comments

Assignees

@kevinslin kevinslin

Labels

docs Improvements or additions to documentation extensions: qa-lab maintainer Maintainer-authored PR P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. size: L status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@kevinslin