pdfdiff — Git-like PDF Version Diff Engine

A deterministic, coordinate-shift-proof PDF comparison engine implemented for this codebase (Python 3.13, PyMuPDF stack — no new dependencies required).

The critical requirement it solves:

OLD:  A B C D E          NEW:  A B NEW CONTENT C D E

Because NEW CONTENT was inserted, the layout engine moved C, D, E to completely different coordinates. The changeset says:

A = unchanged     B = unchanged
NEW CONTENT = ADDED
C = unchanged     D = unchanged     E = unchanged

It never says C/D/E = modified because their coordinates changed. Content identity decides what changed; coordinates only locate the change.

---

What's in this repo

Path	What
pdfdiff/	The diff engine package (pipeline stages)
pdfdiff/reasons.py	Deterministic WHY-explanations for every change (attached to the changeset, shown in the PDF/reports/viewer)
api/	FastAPI service: upload, compare, cached changesets, annotated PDF, page renders
viewer/	Side-by-side web viewer with clickable highlight overlays + reason tooltips
tests/	28 automated tests: 24 engine scenarios + 4 API integration
results/	Changeset JSON + Markdown + HTML + side-by-side annotated PDF for the two real PDFs in this folder
FINAL_DIFF_SIDE_BY_SIDE.pdf	Ready-to-open side-by-side diff of the two shipped PDFs — every change numbered on the pages and explained in the reasons panel
docs/	Architecture, algorithm, data-model and design-decision documentation

The two PDFs in this directory (45-page and 46-page versions of the same real-estate sale agreement) are the primary real-world test case: docs/REAL-PDF-RESULTS.md walks through every change the engine found in them.

Quick start

# CLI: full comparison of the two real PDFs (text pipeline only)
python -X utf8 -m pdfdiff "Final Neo AOS Copy for Regular and Combo.pdf" \
                          "Final Neo AOS Copy for Regular and Combo1.pdf" \
                          -o results --json --markdown --html

# side-by-side annotated PDF (default layout): each sheet has the OLD page
# on the left (red = deleted) and the NEW page on the right
# (green = added, yellow = modified, blue = moved, purple = format).
# Every change gets a numbered callout and a Git-style review entry:
#   - removed OLD text, + added NEW text, > moved text, = format-only text
# followed by a deterministic WHY explanation. Long entries wrap instead
# of being clipped.
python -X utf8 -m pdfdiff "old.pdf" "new.pdf" --visual --annotated-pdf --console

# classic interleaved layout (old page, then new page) instead:
python -X utf8 -m pdfdiff "old.pdf" "new.pdf" --annotated-pdf --layout sequential

# OCR for scanned pages (opt-in; uses the installed PaddleOCR backend)
python -X utf8 -m pdfdiff "scan1.pdf" "scan2.pdf" --ocr --json

A ready-made side-by-side diff of the two PDFs shipped in this repo is at FINAL_DIFF_SIDE_BY_SIDE.pdf (46 sheets, one per logical page pair). The self-contained source PDFs and all related outputs are organized under results/final-neo-aos/; see results/README.md.

# Service: uvicorn api.service:app --port 8000
# then:
#   POST /documents                    (upload PDF)
#   POST /documents/{id}/versions      (upload version PDF)
#   POST /documents/{id}/compare       {"oldVersionId": "...", "newVersionId": "..."}
#   GET  /comparisons/{id}             -> changeset JSON
#   GET  /comparisons/{id}/annotated.pdf
#   GET  /comparisons/{id}/report.md
#   open /viewer?comparisonId=...      -> side-by-side viewer

Tests

python -X utf8 -m pytest tests/ -q      # 28 tests, ~31s

The suite covers all 12 required scenarios including the critical coordinate-shift test (test_paragraph_inserted_coordinate_shift — asserts both that the fixture really moves C/D/E and that they are reported unchanged), duplicate-text context matching, 100-page performance, and the false-negative rules (10%→20%, ₹10,000→₹20,000, January→February, 2025→2026, approved→rejected).

Pipeline

OLD PDF ──extract──▶ Document model ─┐
                                     ├─▶ page alignment ─▶ regions ─▶
NEW PDF ──extract──▶ Document model ─┘     (fingerprint + NW)         │
                                                                      ▼
     ┌──────────────── word-stream diff (content identity) ◀──────────┘
     │            (SequenceMatcher over normalized word keys — the
     │             Git-line-level equivalent for regions)
     ├──▶ grouped changes (Git-style hunks)
     ├──▶ char-level diff inside modified words
     ├──▶ style diff on equal runs (FORMAT_CHANGED)
     ├──▶ movement detection (TEXT_MOVED / PAGE_MOVED)
     ├──▶ images / drawings / annotations / form fields diff
     └──▶ visual pixel diff (opt-in, noise-guarded)
                    │
                    ▼
          CANONICAL CHANGESET (JSON)
                    │
     ┌──────────────┼─────────────────┐
     ▼              ▼                 ▼
   JSON         annotated PDF      viewer overlays

Full detail: docs/ARCHITECTURE.md, docs/ALGORITHM.md, docs/DATA-MODEL.md, docs/DESIGN-DECISIONS.md, docs/TEST-REPORT.md.

Dependencies

Zero new dependencies — the engine uses libraries already installed in this environment. Rationale in docs/DESIGN-DECISIONS.md:

Library	Role	Why
pymupdf 1.28	PDF extraction (words + bboxes + font/style), rendering, annotation	Best-in-class word-level spans with exact glyph boxes and real font metadata; renders pages for visual diff; AGPL
pillow + numpy	render buffering / pixel math	standard
opencv (optional)	connected-component blob analysis for visual regions	graceful numpy-grid fallback when absent
paddleocr (optional, opt-in)	OCR for scanned pages	already installed; pytesseract fallback supported
fastapi + uvicorn	service layer	already installed
difflib (stdlib)	sequence alignment core	deterministic, zero-dep; wrapped by our own shingle/WP similarity layers

Configuration

Every threshold is a named field of pdfdiff.config.DiffConfig (tolerances, movement rules, grouping distances, visual-diff noise floors, highlight colors). See the dataclass for the full annotated list; override per request via the CLI flags or programmatically.