Skip to content

About

Catch fabricated RAG citations before they reach the user: a zero-token verbatim gate + a burden-of-proof judge, framework-agnostic.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

6 stars

Watchers

0 watching

Forks

Latest commit

Β 

History

47 Commits

Folders and files

Repository files navigation

verbatim-citation-gate

Checks that a quote an AI cites really appears word for word in the source document, before the answer reaches the user.

Who it's for: developers whose chatbot or RAG system (a bot that answers from your documents) shows citations. What you get: a free, instant verdict per quote (found, not found, or quoted from the wrong document), plus an optional second check by a model of your choice.

Quick start

  1. Install Python 3.9 or newer.
  2. Run pip install "git+https://github.com/tonydzi/verbatim-citation-gate".
  3. Run the short example under "Quickstart - 30 seconds" in Details below. Or, from a clone of the repo, run pip install -e . and then python examples/demo.py (the demo fails with ModuleNotFoundError if the package is not installed).

Done when: a made-up quote prints not_found and a real quote prints found. No API key or internet is needed for this first check.

Want an AI agent to do it for you? Paste this to Claude Code / Codex / Cursor: "Read the 'For AI agents' section of https://github.com/tonydzi/verbatim-citation-gate and set it up for me."

For AI agents

Preconditions

  • Python 3.9 to 3.13, git. The gate has zero dependencies. Not on PyPI: install from the repository.
  • The optional judge stage needs a model you already use. Ask the human which model and where its API key lives. Never invent a key.

Install

pip install "git+https://github.com/tonydzi/verbatim-citation-gate"
# from a clone, with the test extra:
pip install -e ".[test]"

Configure

  • Stage 1 needs nothing: quote_gate(quote, doc_id, docs) where docs is a dict of doc id to full text.
  • Stage 2 is optional: pass llm_call=<function(system, user) -> str> to audit_citation(...). Wire it to the human's own model client. See examples/quickstart.py.
  • Unparseable judge output fails closed: it never counts as support.

Verify

python -c "from verbatim_citation_gate import quote_gate; d={'a':'Body weight was unchanged in both arms.'}; print(quote_gate('Body weight was unchanged in both arms.','a',d), quote_gate('Dramatic weight loss.','a',d))"
# expected: found not_found
pytest -q          # from a clone, after the test extra: 45 passed

Without installing, from a clone: PYTHONPATH=src python examples/demo.py (PowerShell: $env:PYTHONPATH="src"; python examples/demo.py) prints, among other lines, 4 citations audited - 3 rejected for free, 1 reached the model and exits 0. Without PYTHONPATH and without installing it exits 1 with ModuleNotFoundError. Both Verify commands can fail: the one-liner prints something other than found not_found if the gate is broken, and pytest -q reports failures; pytest needs pip install -e ".[test]" first (or PYTHONPATH=src python -m pytest -q tests).

Uninstall / rollback

pip uninstall verbatim-citation-gate

Rules for the agent

  • Do not add dependencies to the gate and do not loosen the fail-closed rule.
  • Keep API keys out of the repo, out of commits and out of logs.
  • A found verdict only means the words exist in the document. It does not mean the claim is true.
  • Publishing to PyPI or bumping the version needs the human's approval.

Details

tests python license docs

Catch fabricated RAG citations before they reach the user. A two-stage, framework-agnostic auditor for quote-style citations. The deterministic half is src/verbatim_citation_gate/gate.py: zero dependencies, zero tokens. The model half is src/verbatim_citation_gate/judge.py, and plugs into whatever LLM your pipeline already speaks.

Four cited claims audited: a fabrication, a frankenquote and a misattributed quote are rejected for zero tokens; only the one real quote reaches the model.

Quickstart β€” 30 seconds, no API key

pip install "git+https://github.com/tonydzi/verbatim-citation-gate"
from verbatim_citation_gate import quote_gate

docs = {"veltranib-rct": "... Body weight was unchanged in both arms."}

quote_gate("Body weight was unchanged in both arms.", "veltranib-rct", docs)  # 'found'
quote_gate("Veltranib produced dramatic weight loss.", "veltranib-rct", docs)  # 'not_found'

That is stage 1, and it is the whole offline half: no key, no network, no tokens. The run in the animation above is examples/demo.py β€” python examples/demo.py reproduces it, model calls counted by the demo itself. Stage 2 (the judge) needs a model and is one argument away β€” see Use below.

πŸ“– Docs: https://tonydzi.github.io/verbatim-citation-gate/ β€” the two stages, the API, the verdicts, and every known limit with its issue.

RAG systems love to cite. The problem is how they fail β€” a quote that reads as authoritative and word-for-word can still be one of three things, each with its own case in tests/test_gate.py:

  • fabricated β€” a plausible sentence that appears in no source (test_fabrication_is_not_found),
  • a frankenquote β€” every word is real, but the sentence was stitched together and never actually written (test_frankenquote_is_not_found), or
  • misattributed β€” a real quote lifted from a different document (test_misattributed_real_quote_from_wrong_doc).

A judge model, asked "does this quote support this claim?", waves all three through: they read as fluent and supportive. The fix is to ask the cheaper, prior question first, deterministically β€” that is src/verbatim_citation_gate/gate.py.

The two stages

                 β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
 claim + quote β†’ β”‚ 1. verbatim gate (free)  β”‚ β†’ not_found / misattributed   (rejected, 0 tokens)
   + doc id      β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                               β”‚ found
                               β–Ό
                 β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                 β”‚ 2. skeptical judge (LLM) β”‚ β†’ supports / partial / unrelated / contradicts
                 β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Stage 1 β€” the verbatim gate, all of it in src/verbatim_citation_gate/gate.py. Pure re + substring matching over a normalized form: case, smart quotes, dashes and whitespace folded, numbers and % preserved (test_normalize_preserves_numbers_and_percent in tests/test_gate.py). It cannot be sweet-talked, and every fabrication it rejects costs zero model calls β€” proven by test_audit_fabrication_never_calls_judge in tests/test_gate.py. Frankenquotes fail because the match is contiguous, not bag-of-words, and a real quote from the wrong document is surfaced as misattributed rather than collapsed into not_found β€” the two need different fixes upstream (src/verbatim_citation_gate/gate.py).

Stage 2 β€” the skeptical judge, all of it in src/verbatim_citation_gate/judge.py. For quotes that do exist, a burden-of-proof prompt decides whether the passage actually establishes the claim. All three rules below live in that file's prompt:

  1. Default-refute (judge.py) β€” the verdict starts at unsupported; the quote must earn supports, and ties break against the claim.
  2. Outside knowledge is inadmissible (judge.py) β€” a claim can be true in the world and still unsupported by this source.
  3. Full-strength support (judge.py) — same population, direction, magnitude and certainty, or the verdict caps at partial (subgroup→everyone, correlation→causation, "may"→"does").

If the judge's output does not parse it fails closed (src/verbatim_citation_gate/judge.py): an unverifiable citation never counts as support.

Install

Not on PyPI yet β€” install from the repository:

pip install "git+https://github.com/tonydzi/verbatim-citation-gate"

# from a clone, with the test extra:
pip install -e ".[test]" && pytest -q

Pure stdlib: the gate pulls in nothing, and the judge takes whatever llm_call you already have.

Use

from verbatim_citation_gate import audit_citation

docs = {"veltranib-rct": "... Body weight was unchanged in both arms."}

# Stage 1 only β€” no model, no key, fully offline:
audit_citation("Body weight did not change.", "veltranib-rct",
               "Body weight was unchanged in both arms.", docs)   # -> "found"

# Both stages β€” supply any model as llm_call(system, user) -> str:
def llm_call(system, user):
    return client.messages.create(              # Anthropic shown; any vendor works
        model="claude-haiku-4-5", system=system, max_tokens=1024,
        messages=[{"role": "user", "content": user}],
    ).content[0].text

audit_citation("Body weight did not change.", "veltranib-rct",
               "Body weight was unchanged in both arms.", docs, llm_call=llm_call)
# -> "supports" | "partial" | "unrelated" | "contradicts" | "not_found" | "misattributed"

llm_call is deliberately the smallest possible contract β€” (system, user) β†’ text β€” so the judge wires to Claude, GPT, Gemini, Mistral, Cohere, or a local Qwen without adapters. OpenAI, Cohere, and other one-liners are in examples/quickstart.py.

Why the gate goes first

The number you page on for a citation auditor is recall of unfaithful citations β€” the fraction of bad citations you actually catch. The gate lifts that recall for free on the three failure modes a judge is worst at (fabricated, franken, misattributed), because those are exactly the ones that look supportive. What reaches the model is only the genuinely ambiguous case: a real quote whose sufficiency is a judgment call.

Scope

For citations expressed as character offsets, use the same fail-closed gate via span_gate(doc_id, start, end, docs). Offsets are Python string indices and use the half-open interval [start, end). Pass chunk=(chunk_start, chunk_end) when the retriever returned one chunk; spans outside it are rejected rather than silently clipped. An optional quote= must match inside the resolved span:

from verbatim_citation_gate import span_gate

span_gate("report", 10, 42, docs, quote="the cited text", chunk=(0, 100))

Contributions wiring the gates into specific frameworks (Haystack, LlamaIndex, LangChain, Cohere, Qwen-Agent, …) are welcome.

Roadmap

Now β€” v0.1.0. The deterministic gate (gate.py) and the burden-of-proof judge (judge.py), 24 tests green, install from git. Known limits are open issues, not footnotes β€” read them before you rely on it, especially #1.

Next, in the order we would take them:

What Why it matters Issue
Minimum-evidence rule a one-word quote currently returns found #4
Citations by doc id / offset the gate presumes quote-style citations #9
PyPI package so pip install verbatim-citation-gate is real #8

Shipped from this table β€” all three fixed by outside contributor Guan Tong, who claimed them off the issue list and had the first PR open within the hour: Unicode-aware normalization (#1 β€” before it, the gate's core promise was silently off in every non-Latin script), corpus normalization cached (#2), and the py.typed marker (#3).

Every noticeable change ships as a new release, so the release feed is the honest record of how far along this is β€” not the commit graph.

What "working" means here β€” the metrics we actually track

Stars are free and prove nothing. From 2026-08 this project is measured on three numbers, reviewed monthly, target horizon 4–8 months:

  1. Weekly unique clones β€” people pulling the code, not applauding it.
  2. Independent consumers β€” repos or teams that would have to replace this gate if it disappeared. This is the only number that really matters.
  3. Repeat outside contributors β€” engineers who came back a second time. First one already exists; that fact is why this repo became the lab's flagship.

AI contributors

This project is built by a human + AI team, and the git log says so: Claude writes most of the code, Codex and Grok review it, Gemini feeds the research. Each is credited on a commit only if its output changed that commit's content β€” no decorative credits. Lab-wide policy, one source for every repo: AI-CONTRIBUTORS.md.

License

MIT Β© Palo Alto AI Research Lab

About & contact

Built and battle-tested at Palo Alto AI Research Lab β€” a fleet of Claude Code machines running 24/7 as a second brain and synthetic cofounder. This gate was extracted after it survived production, not written as a demo.

Questions, war stories, or you want to run this on your own fleet:

Contributors welcome β€” and there is a queue

The queue is visible: verbatim-citation-gate β€” roadmap β€” Now (an open PR exists), Next (scoped, free to take), Later (deferred, with the reason on the card), Shipped β€” and Shipped now starts with v0.1.0.

Issues labelled accepted are scoped, free to take, and nobody is on them. Comment "claiming this" β€” no permission needed β€” and it is yours for 7 days. New here? Start with good first issue.

You keep the copyright to your code. No CLA, no assignment, ever β€” your contribution goes in under this repo's existing license, the same terms as ours. We answer every issue and PR within 48 hours, including "no, and here is why"; our silence is our bug, so ping the thread.

Full deal: CONTRIBUTING.md


🧩 One piece of a working system

This repository is one piece lifted out of a live operation: one engineer running a fleet of machines, built with Claude as implementation collaborator; the machines reach consensus with each other and wake the human only for money or the irreversible. It was extracted after it survived production, not written as a demo β€” and it runs on its own: nothing here phones home to the rest.

See how the whole thing fits together β†’ SYSTEM.md

Its closest neighbours in the gates layer: verdict-contract Β· claim-check Β· verified-ops-starter

READ THIS WITH AI

One click and an agent reads the repo, pulls out the patterns and helps you apply them to your own work.

Codex - open ChatGPT - open Claude - open

Copy the prompt (works in any agent: Gemini, Grok, a local model, your own CLI)
Read this repo: https://github.com/tonydzi/verbatim-citation-gate (β€œverbatim-citation-gate” - Catch fabricated RAG citations before they reach the user: a zero-token verbatim gate + a burden-of-proof judge, framework-agnostic). Work out what problem it actually solves, pull out the reusable patterns and help me apply them to my own setup. Start by asking what I am working on.

β€” TonyDzi, Palo Alto AI Research Lab Β· second brain, agent coordination, persistent memory: github.com/tonydzi

About

Catch fabricated RAG citations before they reach the user: a zero-token verbatim gate + a burden-of-proof judge, framework-agnostic.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

6 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages