Consider an opt-in in-memory index for uniqueness constraints #130

Consider an opt-in in-memory index for models that declare uniqueness constraints.

Context

This is the last remaining recommendation from a finding that is otherwise closed. The original report was that Unique() silently turns writes from O(1) into O(table size) with a full storage read per document — a 55–89× slowdown that nothing warned about. Two of its three recommendations have landed:

• Batch case fixed. insert_many() checks the whole batch against a single table scan. Measured at HEAD: a batch of 100 into a table with one Unique() field costs 2 storage reads (it was 102).
• Cost documented. docs/usage/models.md now carries the note in the project's own voice — "Enforcement costs a table scan. TinyDB has no indexes, so an enforcing write reads the whole table and compares it document by document — O(documents) … If a single write is slow, the table has outgrown what TinyDB is for."

What is left

A single-document write still costs one extra full scan. Measured at HEAD:

So a loop of insert() calls into a growing table is still quadratic, where a single insert_many() is linear.

The idea

An opt-in in-memory index for models that declare constraints, invalidated on write. That fits the single-process/single-thread contract the library already requires, and would make uniqueness roughly free for the loop-of-inserts case.

Why this is low priority

• The documented answer — "use insert_many() for bulk loads" — already covers the case that hurt most, and it is now genuinely fast.
• The cost note tells users where the ceiling is, honestly.
• Adding an index is real machinery: invalidation, correctness under bind(), interaction with validate_writes=False, and a new failure mode if the index and the table ever disagree. It buys speed on a path the docs already steer people away from.

Filed so the idea is not lost, not because it is next.

Reproduction

from typing import Annotated

from tinydb import TinyDB
from tinydb.storages import MemoryStorage

from tinydantic import TinydanticModel, Unique
from tinydantic.tinydb.middlewares import ProfilingMiddleware

storage = ProfilingMiddleware(MemoryStorage)
db = TinyDB(storage=storage)


class Plain(TinydanticModel, database=db):
    v: str


class Uniq(TinydanticModel, database=db):
    v: Annotated[str, Unique()]


storage.reset()
Plain(v="a").insert()
print("Plain.insert():", storage.read_count, "storage reads")

storage.reset()
Uniq(v="a").insert()
print("Uniq.insert(): ", storage.read_count, "storage reads")

Run with python repro.py.

Actual output:

Plain.insert(): 2 storage reads
Uniq.insert():  3 storage reads

One extra full read per constrained single-document write — so a loop of insert() calls into a growing table stays quadratic, while a single insert_many() is linear.

Expected: with an opt-in index, a constrained insert() costs what an unconstrained one costs. This is the idea being filed, not a defect: the documented answer (insert_many() for bulk loads) already covers the case that hurt most.

Environment: tinydantic 0.4.0 at 049f261, Python 3.10.20, TinyDB 4.9.0, pydantic 2.13.4, macOS. Run 2026-08-13, using the ProfilingMiddleware added in #141.

Re-measured at e7d0ed0 on 2026-08-12. Originally recommendation 3 of finding H3 in the 2026-08-06 internal review; recommendations 1 and 2 are done.

Assisted-by: Claude:claude-opus-5