Consent Guardian

A consent-aware trust-&-safety layer for intimate chat.

The promise of a dating or intimate-chat platform is judgment-free but safe. The hard version of that isn't an NSFW filter — a filter would censor the consensual intimacy that is the product. Consent Guardian protects boundaries without policing desire, and runs on the chat infrastructure these platforms already use (e.g. Stream Chat).

The idea in one paragraph

The same explicit message can be welcome in one conversation and a violation in another. The deciding factor is consent context, not the words. Consent Guardian keeps a per-conversation consent state machine and runs a two-layer enforcement pipeline over it: a fast, on-device/edge classifier on every message, and a Claude judge that fires only on the ~small slice Layer 1 can't resolve. Interventions are graded and framed as care, not policing, and intimate media is classified on-device — the image never reaches the server.

Architecture

message ─▶ Layer 1: fast classifier (every message, cheap)
                │   contextual labels from consent state:
                │   welcome_explicit / unsolicited_explicit / pressure /
                │   boundary_cross / threat_abuse
                ▼
           [ escalate? ]  ── no ──▶ intervention
                │ yes
                ▼
           Layer 2: Claude judge (Sonnet 5)  ── reasons about consent CONTEXT,
                │   returns verdict + severity + rationale + intervention
                ▼
           intervention (graded, care-framed)
                │
                ▼
           update consent state machine, persist

• Consent state machine (src/guardian/consent/) — getting_to_know → explicit_opted_in, plus standing media consent and stated boundaries. Opting in takes a mutual signal; a single boundary pulls the brakes. This is the part that distinguishes welcome from unwelcome — a dumb filter can't.
• Layer 1 (src/guardian/layer1/) — transparent lexicon classifier (a small on-device model in production) + an on-device NSFW media gate. Scores concern in context: explicit text between opted-in adults is welcome_explicit (~0 concern); the same text pre-opt-in is unsolicited_explicit.
• Layer 2 (src/guardian/layer2/judge.ts) — a Claude judge that reasons about the message against the consent state. Swappable: ClaudeJudge in prod, HeuristicJudge for offline/keyless runs.
• Interventions (src/guardian/interventions.ts) — pre-send nudge → media consent handshake (image held, blurred, until the recipient opens it) → one-tap boundary → T&S escalation for threats only.

Quickstart

Requires Node ≥ 20.

npm run setup             # installs root + dashboard deps
cp .env.example .env      # key left empty = offline judge; add ANTHROPIC_API_KEY for the real one

# run the labeled eval WITHOUT a key (Layer-1 baseline):
npm run eval:offline

# run the eval with the real Claude judge:
npm run eval

# run the service:
npm run dev

# …or run the service + the live dashboard together (http://localhost:3010):
npm run dev:all

Try the flip (the whole idea, in three curls)

# 1. Unsolicited explicit, no prior opt-in → intervene
curl -s localhost:4000/message -H 'content-type: application/json' -d '{
  "conversationId":"c1","senderId":"alex","recipientId":"sam","text":"send nudes"
}' | jq '{intervention: .intervention.type, labels: .layer1.labels}'
# → { "intervention": "sender_nudge", "labels": ["unsolicited_explicit"] }

# 2. Sam opts into explicit conversation
curl -s localhost:4000/consent/optin -H 'content-type: application/json' \
  -d '{"conversationId":"c1","userId":"sam","granted":true}' >/dev/null

# 3. The IDENTICAL message again → welcome. Only the consent context changed.
curl -s localhost:4000/message -H 'content-type: application/json' -d '{
  "conversationId":"c1","senderId":"alex","recipientId":"sam","text":"send nudes"
}' | jq '{intervention: .intervention.type, labels: .layer1.labels}'
# → { "intervention": "none", "labels": ["welcome_explicit"] }

Note that talk consent ≠ media consent: even after step 2, an intimate photo is still held behind the consent handshake until Sam separately grants standing media consent (POST /consent/media).

Point your chat provider's new-message webhook (e.g. Stream Chat's message.new) at POST /stream/webhook to run the Guardian on the chat infra you already use.

Dashboard

A Next.js console (dashboard/) for the live demo. Send messages as Alex, change what Sam has consented to, and watch the same message flip from intervene to welcome as the consent context changes — plus a metrics strip driven by the eval and a live view of the consent state machine.

npm run dev:all          # Guardian on :4000 + dashboard on http://localhost:3010

The dashboard is a pure client of the Guardian service (proxies /api/guardian/* → :4000; set GUARDIAN_URL if you change the service PORT), which is itself the architecture story: the Guardian is a service; the console just consumes it.

The metric that matters

The eval reports welcome false-positive rate — how often the Guardian would intervene on consenting adults. That number, not "messages blocked", is the one a system like this must be optimized against: over-blocking welcome intimacy kills the product and disproportionately harms marginalized kink/queer communities. The eval is deliberately hostile to shortcuts: scenarios include multi-turn conversations (consent is earned through the pipeline, not just seeded) and state-pollution assertions — a verdict can be "right" and still fail the scenario if the message silently recorded a bogus boundary. judge fire rate is the cost lever (it reads 50% here because the eval set is adversarial by construction; organic traffic is mostly benign and resolves in Layer 1).

Design principles

• Context over content. The system answers "did the recipient consent to this?", not "is this explicit?" — the only question that separates welcome intimacy from a violation at scale, and the one a content filter can't ask.
• Optimize the product-protecting metric. Welcome false-positive rate, with an eval harness to prove it — over-blocking is the failure mode that kills the product, so it's the number the system is tuned against.
• A systems answer, not a feature. Two-layer enforcement + on-device privacy
  • a consent state machine, designed to fail open and to be cheap on the 99% of benign traffic that never needs the judge.

A companion design doc (docs/RFC.md) covers rollout (shadow → nudge-only → enforce, with a kill switch), failure modes, and the equity risk of over-blocking — the production thinking behind the code.

Layout

src/
  config.ts                     env + thresholds
  types.ts                      shared domain types
  guardian/
    pipeline.ts                 two-layer orchestrator
    consent/stateMachine.ts     consent transitions (the differentiator)
    consent/store.ts            pluggable persistence (Firestore-ready)
    layer1/textClassifier.ts    fast contextual classifier
    layer1/nsfw.ts              on-device media gate (score only)
    layer2/judge.ts             Claude judge + offline heuristic stub
    interventions.ts            graded, care-framed interventions
  server.ts                     Express API + Stream Chat webhook adapter
  eval/                         labeled scenarios + metrics harness
dashboard/                      Next.js live console (the demo UI)
docs/RFC.md                     design doc

All safety copy is illustrative.