Problem

Moltbot processes external content from multiple channels (WhatsApp, Telegram, email, GitHub, web pages) and tools. This external content could contain prompt injection attacks that attempt to:

• Exfiltrate sensitive data (API keys, config files, credentials)
• Execute unauthorized commands
• Bypass trust boundaries between users/channels

Currently there's no automated way to verify the agent resists these attacks across the various ingress paths.

Proposal

An E2E security test harness that:

1. Sends poisoned payloads through real channel ingress (not just direct gateway calls)
2. Uses LLM-as-judge (Claude Opus) to evaluate whether attacks succeeded - pattern matching can't reliably detect nuanced compliance
3. Covers multiple attack surfaces: channel metadata (forwarded messages, vCards, group descriptions), CLI tool responses (curl, gh, gog), browser content

Note: These tests require an Anthropic API key for the LLM judge. The repo won't have a key configured - contributors run tests locally with their own key.

Key Components

• chat.ingress gateway method to test real Telegram/WhatsApp parsing pipelines
• CLI mocks that return poisoned responses to test tool-based injection
• LLM judge with structured output for pass/fail verdicts
• HTML report generator for triage

Implementation Approach

Happy to break this into smaller PRs:

1. Harness infrastructure - gateway client, LLM judge, assertions
2. Channel ingress testing - chat.ingress method + Telegram/WhatsApp tests
3. CLI mocks - curl, gh, gog, browser-cli with poisoned payloads
4. Reporting - HTML/JSON report generation

Prior Art

Have a working implementation on feat/security-test-harness branch (~1100 lines). Closed PR #3923 due to size - seeking alignment before resubmitting as smaller chunks.

Questions for Maintainers

• Is there an ongoing effort for security testing that this duplicates or could integrate with?
• Does this align with security testing priorities?
• Preference on PR breakdown?