FindQuery.count()/exists() scan and validate everything; pagination cost undocumented #122

FindQuery.count() and exists() materialize and validate every match; pagination cost is undocumented.

What happens

FindQuery._execute() calls search() — which validates every matching document into a model instance — then sorts, then slices. The window is applied after all the work, and count()/exists() throw that work away.

Evidence

5,000 documents, MemoryStorage (best case — no file I/O at all), mean of 5 runs:

Fetching the last page of 10 costs 102% of fetching all 5,000. find(cond).count() is 373× Model.count(cond) for the same answer. find(cond).exists() is 1,470× Model.contains(cond) — a full scan and full validation to answer a yes/no question.

Why it matters

The pagination recipe in docs/usage/find.md is explicitly aimed at HTTP handlers ("an HTTP handler would map it to 404"), and fastapi.md links pagination to request handling. On Pattern A (async def with inline calls) every page request blocks the event loop for the full scan.

The underlying cost is inherent to TinyDB — there are no indexes, and sorting on validated instances is a deliberate, well-argued choice. But two things are fixable:

What to do

1. Short-circuit when no sort is set. exists() needs only the first match and can delegate to Model.contains(cond); count() with no skip/limit can delegate to Model.count(cond). Both are contained changes with no semantic effect. With a sort set, or with a window, the current path is still required.
2. Document the pagination cost where the recipe lives. find.md currently carries no cost note. Something in the voice the rest of the docs use for trade-offs: "A chain materializes and validates every matching document before applying the window, so page 50 costs the same as page 1. That is fine for the table sizes TinyDB targets; if a single page is slow, the table has outgrown TinyDB."

Reproduction

The measurement harness behind the table above: 5,000 documents, MemoryStorage, mean of 5 runs.

import timeit

from tinydb import TinyDB
from tinydb.storages import MemoryStorage

from tinydantic import TinydanticModel

db = TinyDB(storage=MemoryStorage)


class Doc(TinydanticModel, database=db):
    n: int


Doc.insert_many([Doc(n=i) for i in range(5000)])
cond = Doc.n >= 0  # matches every document

calls = {
    'find().sort("n").skip(4990).limit(10).all()':
        lambda: Doc.find().sort("n").skip(4990).limit(10).all(),
    "find().all()": lambda: Doc.find().all(),
    "find(cond).count()": lambda: Doc.find(cond).count(),
    "Doc.count(cond)": lambda: Doc.count(cond),
    "find(cond).exists()": lambda: Doc.find(cond).exists(),
    "Doc.contains(cond)": lambda: Doc.contains(cond),
}
for label, call in calls.items():
    print(f"{label:44} {timeit.timeit(call, number=5) / 5 * 1000:8.3f} ms")

Run with python repro.py.

Actual output:

find().sort("n").skip(4990).limit(10).all()     5.061 ms
find().all()                                    5.225 ms
find(cond).count()                              4.117 ms
Doc.count(cond)                                 0.019 ms
find(cond).exists()                             3.614 ms
Doc.contains(cond)                              0.011 ms

Absolute times are machine-dependent and the ratios move with the condition used, but the shape holds: the last page of 10 costs what all 5,000 cost, find(cond).count() is ~217× Model.count(cond), and find(cond).exists() is ~329× Model.contains(cond).

Expected: with no sort and no window set, exists() stops at the first match and count() delegates to Model.count(cond), so both land in the same order of magnitude as their model-level equivalents. The pagination cost is inherent and stays — but documented where the recipe lives.

Environment: tinydantic 0.4.0 at 4d373d1, Python 3.10.20, TinyDB 4.9.0, pydantic 2.13.4, macOS. Run 2026-08-13.

Re-measured at e7d0ed0 on 2026-08-12. Originally finding M5 of the 2026-08-06 internal review.

Assisted-by: Claude:claude-opus-5