Social Triage Agent

A multi-tenant social-inbox triage agent. It pulls pending comments and DMs from Metricool, classifies each one with an LLM, drafts a brand-voice reply for the ones that warrant it, and outputs a categorized, action-ready markdown queue — so the right human handles the right message fast.

Built for a regulated (healthcare) use case, so the design's centerpiece is a deterministic safety layer that overrides the LLM: anything mentioning a medication, symptom, or diagnosis is force-routed to a human doctor, no matter what the model thought.

Why it's interesting

• Multi-model routing, per task. Classification runs on Claude Haiku (fast, cheap, safety-critical); drafting runs on DeepSeek (cheap, good Spanish) — both through the OpenRouter gateway via one OpenAI-compatible client. Cheap model where volume lives, careful model where safety lives.
• LLM + deterministic guardrail. The model gives a nuanced read, then a hardcoded keyword layer runs after it and forces the safe category if a medical or abusive term fired. It's not a config flag — a future contributor can't accidentally disable it. False positives (over-routing to the doctor) are cheap; false negatives (auto-replying about a drug dosage) are catastrophic.
• Prompt caching. The large classifier system prompt (taxonomy + keyword lists) is marked cache_control: ephemeral, cutting ~90% of per-call cost within the cache window.
• Fails safe. Invalid JSON from the model falls back to the most conservative bucket so a human still sees the message.

Architecture

flowchart TB
    MC["Metricool API<br/>pending comments / DMs"] --> PULL
    subgraph APP["social-triage-agent"]
      direction TB
      PULL["pull.py<br/>per-brand loop"] --> CLS
      CLS["Classifier — Claude Haiku<br/>(cached system prompt, via OpenRouter)"] --> KW
      KW["Deterministic guardrail layer<br/>MEDICAL_KEYWORDS / ABUSIVE_KEYWORDS<br/>runs AFTER the LLM, forces safe category"] --> ROUTE
      ROUTE{"category?"}
      ROUTE -- "LEAD / NEGATIVE / FAQ" --> DRAFT["Drafter — DeepSeek<br/>brand-voice reply (via OpenRouter)"]
      ROUTE -- "MEDICAL / HARASSMENT / POSITIVE / OTHER" --> SKIP["no draft<br/>(route to human / hide / like / ignore)"]
      DRAFT --> REPORT
      SKIP --> REPORT
      REPORT["report.py<br/>categorized markdown queue"] --> ALERTS["alerts.py<br/>email on red/yellow items"]
    end
    REPORT --> OUT["output/YYYY-MM-DD-&lt;client&gt;-pending-queue.md"]

Loading

Categories

Category	What it is	Suggested action
HARASSMENT_OR_ABUSIVE	Insults, defamation, slurs, trolls	Hide via the platform UI (not delete) — trolls don't escalate when they don't realize they've been silenced.
NEGATIVE_SENTIMENT	Legitimate complaint or dissatisfaction	Draft reply for human approval — acknowledge, offer to take private.
MEDICAL_CONTENT_REQUIRES_DOCTOR	Any symptom, medication, condition, dosage, treatment, lab result	Route to the doctor personally. The engine never auto-replies.
LEAD_OPPORTUNITY	"How do I book?", "Where are you?", "How much?" without medical specifics	Draft reply for human approval.
FAQ_AUTOREPLIABLE	Hours, location, parking, phone — pure FAQ	Safe template reply (Phase 2 actions this; Phase 1 queues).
POSITIVE	Thanks, compliment, supportive emoji	Auto-like, no reply needed.
OTHER	Tags, off-topic, spam	Ignore.

Healthcare guardrails

community_manager/classifier.py holds two keyword sets — MEDICAL_KEYWORDS (Mounjaro, Ozempic, testosterona, dosage, …) and ABUSIVE_KEYWORDS (Spanish/English insults). After the LLM classifies a thread, _keyword_overrides() runs a word-boundary match; if a medical term fired, the category is forced to MEDICAL_CONTENT_REQUIRES_DOCTOR and the action to route_to_doctor, regardless of what the model returned. Same idea for abuse → HARASSMENT_OR_ABUSIVE. Deterministic, always-on, and impossible to silently disable.

Quick start

python -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
pip install -r requirements.txt

cp .env.example .env
# Fill in: METRICOOL_USER_TOKEN, METRICOOL_USER_ID, OPENROUTER_API_KEY

# Edit community_manager/config.py to point CLIENTS at your own Metricool brand ids,
# then run for a client slug:
python pull.py --client example-clinic

Output goes to output/YYYY-MM-DD-<client>-pending-queue.md.

CLI flags

Flag	What it does
--client <slug>	Which client to process (required). See community_manager/config.py.
--no-draft	Skip the DeepSeek drafting step. Faster + cheaper if you only want classification.
--dry-run	Print summary to stdout but don't write the report file.
--no-alert	Skip email alerts even if SMTP is configured.
--reset-seen	Wipe the dedup seen-set (re-alerts on every pending thread).

Adding a client

1. Confirm the brand(s) are connected to your Metricool account.
2. Add a Client entry in community_manager/config.py → CLIENTS. List every Metricool blog_id that should roll up to that client.
3. (Optional) Update community_manager/drafter.py → DRAFTER_SYSTEM_PROMPT to encode the brand's voice.
4. Run: python pull.py --client <new-slug>.

Tech stack

• Python 3.10+
• LLM gateway: OpenRouter via the OpenAI-compatible SDK — one key, many providers
• Models: anthropic/claude-haiku-4-5 (classification), deepseek/deepseek-chat (drafting)
• Prompt caching: cache_control: ephemeral on the classifier system prompt (Anthropic-honored via OpenRouter)
• Metricool: REST API via requests
• Config: python-dotenv
• No DB in Phase 1a — output is markdown; dedup is a local JSON seen-set

Roadmap

• Phase 1a (this repo): manual run, classify + draft + markdown report + email alerts.
• Phase 1b: cron scheduling + webhook fan-out (WhatsApp / task tracker).
• Phase 2: whitelisted auto-actions (auto-like, template FAQ replies), DM inbox, review sources, a database-backed event log.

Files

social-triage-agent/
├── README.md
├── .env.example
├── requirements.txt
├── deploy.sh                         # idempotent Linux venv + cron installer
├── pull.py                           # CLI entry point
└── community_manager/
    ├── config.py                     # client + brand registry, settings loader
    ├── metricool.py                  # REST client + CommentThread dataclass
    ├── classifier.py                 # Haiku classifier + deterministic keyword overrides
    ├── drafter.py                    # DeepSeek draft generator
    ├── report.py                     # markdown rendering
    ├── alerts.py                     # email alerter (Gmail SMTP)
    └── seen.py                       # local dedup seen-set

License

MIT — see LICENSE.