driftsnap

Your orchestrator says the task succeeded. It has no idea the table came back with 40% fewer rows, a column that went all-null, or a region that collapsed to a single value because a join fanned out.

driftsnap profiles a table and tells you what changed since last time. One query, no agent, no service, no dependencies.

pip install driftsnap[duckdb]

---

Use it in CI

driftsnap capture duckdb:///warehouse.db sales -o snapshots/sales.json   # after each run
driftsnap diff snapshots/sales.json snapshots/sales.new.json             # exit 1 on drift

ERROR: row_count_drop - 1000 -> 800 rows (-20.0%)
ERROR: null_rate_spike [region] - nulls 1.0% -> 50.0%
WARN: distinct_collapse [customer_id] - distinct values 4210 -> 1 (-100.0%)

Commit the snapshot files next to your models and the diff becomes a reviewable artifact: the PR shows exactly how the data moved, not just that the DAG went green.

Use it in Python

import duckdb
from driftsnap import snapshot, diff, Thresholds

conn = duckdb.connect('warehouse.db')
before = snapshot(conn, 'sales')
# ... pipeline runs ...
after = snapshot(conn, 'sales')

result = diff(before, after, Thresholds(row_count_drop=0.05))
if not result.ok:
    raise RuntimeError(result.report())

Scope a check to the partition you just wrote instead of the whole table:

snapshot(conn, 'sales', where="sold_on = DATE '2026-09-06'")

What it checks

Check	Default	Severity
row_count_drop	more than 10% of rows gone	error
row_count_spike	off	warn
null_rate_spike	a column's null rate rose by 5 points	error
distinct_collapse	distinct values halved while rows held	warn
column_added / column_removed / type_changed	any	error

Every threshold is a constructor argument or a CLI flag. --allow-schema-change turns schema movement into warnings for tables where evolution is expected.

Engines

The library takes any DB-API 2.0 connection whose engine exposes information_schema — DuckDB, Postgres, Trino, Snowflake, MySQL. Iceberg and Delta tables work by registering them in DuckDB (iceberg_scan, delta_scan) or by pointing at Trino/Spark SQL.

conn.execute("CREATE VIEW sales AS SELECT * FROM iceberg_scan('s3://lake/sales')")
snapshot(conn, 'sales')

The driftsnap CLI ships DSN wiring for DuckDB and Postgres only. For anything else, call snapshot(connection, table) from Python — that path is engine-agnostic.

Design notes

• One query per snapshot. Columns come from information_schema, stats come from a single aggregate scan. No per-column round trips.
• No dependencies. Snapshots are plain JSON, so they diff in git, land in S3, and stay readable in five years.
• Non-scalar columns are described, not profiled. Structs, lists, maps and blobs get a type and a null count; min/max/distinct are skipped rather than guessed at.
• where is inlined into SQL. Never build it from untrusted input.

Development

pip install -e ".[dev]"
pytest

License

MIT