You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit a30e1e0
Browse filesBrowse the repository at this point in the historyBrowse files
docs(upstream): link every limitation to its tracking issue
Backfills the requirement this branch introduced. The limitations
page documented friction that had no issue behind it, so the page
stated a rule its own entries did not satisfy.
Eight issues now cover the seven limitation sections plus the one
private-API use with no section of its own (no public export for
ModelMetaclass). Granularity follows the friction, not the table
rows: QueryInstance._hash belongs to the id-condition limitation
that forces it, and Table._update_table to the update_many() gap, so
neither private-API row gets an issue of its own.
The issues are deliberately thin. They link to the docs section
rather than restating it, because the page is the artifact that
ships — versioned with mike, checked by strict mode, and written to
be read standalone by an upstream maintainer, which makes it the
ready-to-send proposal already. An issue body cannot be any of
those, and nothing reviews it when the code it describes moves. What
the issues carry instead is the part a page cannot: status,
priority, relationships, and the decision about whether anything
reaches upstream.
Also records the mkdocstrings decision at the config line it governs.
merge_init_into_class stays false: enabling it labels the source
block with the inherited __init__ and leaks a .venv path into
published docs, and the upstream report was closed with the project
in maintenance mode. Without the comment the next reader sees a
disabled feature we appear to want and turns it back on.
Closes#131
Assisted-by: Claude:claude-opus-5
**Limitation.** TinyDB evaluates every query condition against the raw document _body_ mapping — the value in the `{doc_id: body}` table dict — never against a [`Document`](https://tinydb.readthedocs.io/en/latest/api.html#tinydb.table.Document) carrying `doc_id`. All five evaluation sites behave this way: `Table.search()`, `Table.get(cond=...)`, and the updater loops inside `Table.update(cond=...)`, `Table.update_multiple()`, and `Table.remove(cond=...)`. (`Table._update_table()`'s docstring notes that skipping the `Document` wrap is a deliberate optimization.) Only `Table.__iter__` yields `Document` instances. Consequently, no query object — however it is constructed — can express "document id equals 1": the id is structurally invisible to the condition, even though it sits right there as the dict key in every evaluation loop.
24
26
25
27
**Why it matters to an ODM.** Every mainstream document ODM lets users query by the model's id attribute (Beanie and ODMantic translate `Model.id` to MongoDB's `_id`; Firestore special-cases `FieldPath.documentId()` because — exactly like TinyDB — its document key is not a body field). `tinydantic` maps its `id` field to `doc_id`, so supporting `User.get(User.id == 1)` required a translation layer: id-bearing conditions are detected in every model method and executed via `doc_id=`/`doc_ids=` operations or by iterating the table (the one API that yields `Document`s). See `src/tinydantic/_query.py` and the id-condition branches in `src/tinydantic/_model.py`.
@@ -34,6 +36,8 @@ Existing queries and third-party `QueryLike` objects pay nothing (the `getattr`
**Limitation.**`Table.update(fields, cond, doc_ids)` and `Table.remove(cond, doc_ids)` both accept a `doc_ids` selector, but `Table.update_multiple(updates)` accepts only `(fields, cond)` pairs. Combined with the evaluation limitation above, there is no way to run a batched update that targets documents by id — neither via a condition (conditions cannot see ids) nor via an explicit selector (the parameter does not exist).
38
42
39
43
**Why it matters to an ODM.**`update_many()` is TinyDB's only batched write — the whole batch runs in one atomic read-modify-write cycle (`Table._update_table()`), which is the reason to use it over looping `update()`. `tinydantic` translates id conditions to `doc_ids=` operations where a public landing spot exists, but `update_many()` offers none, so supporting id conditions there (without sacrificing the batch's single-write atomicity) required the private-API use recorded in the [registry below](#tinydb-private-api-usage-registry).
@@ -45,6 +49,8 @@ Existing queries and third-party `QueryLike` objects pay nothing (the `getattr`
45
49
46
50
### `Table.update(doc_ids=…)` and `Table.remove(doc_ids=…)` skip missing ids silently
**Limitation.** As of TinyDB 4.9.0 ([#591](https://github.com/msiemens/tinydb/issues/591)), `Table.update(fields, doc_ids=…)` and `Table.remove(doc_ids=…)` filter the requested ids down to those present in the table, operate on that subset, and return only the ids they actually touched. An id that does not exist is not reported in any way. Before 4.9.0 the same call raised a bare `KeyError` partway through the updater — which was its own problem (an uncurated exception, raised after some documents had already been mutated in the working copy).
49
55
50
56
**Why it matters to an ODM.** The new behavior makes a mixed batch a **partial write that reports success**: `update(fields, doc_ids=[1, 999])` on a table without document 999 updates document 1, returns `[1]`, and leaves the caller to notice that the returned list is shorter than the one they passed. A typo'd id in a batch is silently a different operation than the one requested. That is precisely the silent-wrong failure mode `tinydantic` exists to eliminate, and the returned-list-length check that would catch it is exactly the kind of bookkeeping users do not write.
@@ -57,6 +63,8 @@ Existing queries and third-party `QueryLike` objects pay nothing (the `getattr`
57
63
58
64
### Document ids are stringified before reaching storages
**Limitation.** Before table data is handed to the storage layer, document ids are converted to strings (`{str(doc_id): doc}` in `Table._update_table()`), because the reference JSON storage requires string keys. Storages and middlewares therefore never see the native int ids, and serialized output sorts ids lexicographically (`"10"` before `"2"`). See the upstream discussion [msiemens/tinydb#466](https://github.com/msiemens/tinydb/discussions/466).
61
69
62
70
**Why it matters.** Human-readable storage output (a design goal of `tinydantic`'s YAML storage) lists documents in confusing lexicographic order. `tinydantic` ships `SortIntDocIDsMiddleware` (`src/tinydantic/tinydb/middlewares.py`) purely to undo the stringification — it converts keys back to ints, pre-sorted numerically — and that middleware has to pass ints where the `Storage` protocol declares strings, an acknowledged hack.
@@ -65,6 +73,8 @@ Existing queries and third-party `QueryLike` objects pay nothing (the `getattr`
65
73
66
74
### Query objects answer `bool()` and `in` silently
**Limitation.**`QueryInstance` defines no `__bool__` and no `__len__`, so every condition is truthy — `bool(where("name") == "Alice")` is `True`, and so is the same expression for a value no document holds. `Query` additionally defines `__getitem__` (the alternate spelling for nested keys) but no `__iter__`, so Python's legacy sequence protocol makes `x in Query().name` iterate the query and report `True` for any `x`. A non-string path step is read as a callable to apply (`_generate_test`'s runner), so `Query().tags[0] == "red"` raises internally, is swallowed by the runner's `except (KeyError, TypeError)`, and matches nothing.
69
79
70
80
**Why it matters.**`tinydantic` presents these objects as its primary query API, so `if User.name == requested:` reads like an existence check, passes review, and is permanently true. `tinydantic` therefore ships `GuardedQuery`/`GuardedCondition` (`src/tinydantic/_query.py`), which raise `QueryTypeError` for all three. Because TinyDB constructs conditions with `QueryInstance(...)` directly — a `Query` subclass cannot change what its own comparisons return — the guard is applied by reassigning `__class__` on the object TinyDB just built. That uses no private names and preserves the test function and hashval (so guarded conditions still compare, hash, and cache identically), but it does depend on `QueryInstance` remaining a plain, `__slots__`-free class. Each of the ~15 public condition builders is overridden to apply it; a builder added by a future TinyDB would return an unguarded condition, degrading to today's behavior rather than breaking.
@@ -77,8 +87,8 @@ Every approved use of a TinyDB internal/private API in `tinydantic` is recorded
77
87
78
88
| TinyDB internal | Used by | Status | Reason | Upstream change that would remove it |
79
89
| --- | --- | --- | --- | --- |
80
-
|`QueryInstance._hash` (read-only attribute access) |`tinydantic._query.has_id_condition()` — detecting id conditions inside composed queries |**In use** (approved 2026-08-02; shipped with the original id-query work) | Composing queries (`&`/`\|`/`~`) produces plain `QueryInstance` objects, so a custom condition type cannot survive composition — the hashval tree is the only place a marker does. The access is read-only via `getattr(cond, "_hash", None)` and degrades loudly, never silently: if a future TinyDB renames the attribute, bare id conditions are still detected by `isinstance`, and undetected compositions raise `DocumentIDConditionError` when TinyDB's evaluator runs them. | The `needs_doc_id`/`DocId` evaluator improvement above (composition would propagate a public flag); alternatively, a public accessor for a query's hash tree. |
81
-
| `Table._update_table(updater)` | `TinydanticModel._run_write_cycle()` — all `update()`/`update_many()` writes (which validate each matched document's merged result unless `validate_writes=False`), plus the id-condition paths of `remove()` and `upsert()` | **In use** (approved 2026-07-13 for id-condition writes; scope extended 2026-08-02 to all validated update writes) | The only way to select write targets by id inside one atomic read-modify-write cycle, and the only way to validate-then-write atomically: `update_many()` has no `doc_ids` parameter, conditions cannot see ids (the two limitations above), and a public two-pass validate-then-update has a read-modify-write race between passes. The custom updater evaluates every condition against `Document(body, doc_id)` wrappers inside upstream's own read → mutate → write → cache-clear lifecycle, validates merged bodies before the write, applies mutations copy-on-write (so an aborted or validation-failed cycle leaks nothing, even on `MemoryStorage`, whose `read()` shares body dicts by reference), and skips the storage write entirely when nothing matched. Benchmarked ~23% faster than the two-pass public-API alternative on a 5,000-document JSONStorage table (one full file read saved per write). | Either `update_many()` improvement above plus an atomic validate-hook, or the `needs_doc_id`/`DocId` evaluator change combined with a public batched read-modify-write API. |
90
+
|`QueryInstance._hash` (read-only attribute access) |`tinydantic._query.has_id_condition()` — detecting id conditions inside composed queries |**In use** (approved 2026-08-02; shipped with the original id-query work). Tracking: [#133](https://github.com/tinydantic/tinydantic/issues/133)| Composing queries (`&`/`\|`/`~`) produces plain `QueryInstance` objects, so a custom condition type cannot survive composition — the hashval tree is the only place a marker does. The access is read-only via `getattr(cond, "_hash", None)` and degrades loudly, never silently: if a future TinyDB renames the attribute, bare id conditions are still detected by `isinstance`, and undetected compositions raise `DocumentIDConditionError` when TinyDB's evaluator runs them. | The `needs_doc_id`/`DocId` evaluator improvement above (composition would propagate a public flag); alternatively, a public accessor for a query's hash tree. |
91
+
| `Table._update_table(updater)` | `TinydanticModel._run_write_cycle()` — all `update()`/`update_many()` writes (which validate each matched document's merged result unless `validate_writes=False`), plus the id-condition paths of `remove()` and `upsert()` | **In use** (approved 2026-07-13 for id-condition writes; scope extended 2026-08-02 to all validated update writes). Tracking: [#134](https://github.com/tinydantic/tinydantic/issues/134) | The only way to select write targets by id inside one atomic read-modify-write cycle, and the only way to validate-then-write atomically: `update_many()` has no `doc_ids` parameter, conditions cannot see ids (the two limitations above), and a public two-pass validate-then-update has a read-modify-write race between passes. The custom updater evaluates every condition against `Document(body, doc_id)` wrappers inside upstream's own read → mutate → write → cache-clear lifecycle, validates merged bodies before the write, applies mutations copy-on-write (so an aborted or validation-failed cycle leaks nothing, even on `MemoryStorage`, whose `read()` shares body dicts by reference), and skips the storage write entirely when nothing matched. Benchmarked ~23% faster than the two-pass public-API alternative on a 5,000-document JSONStorage table (one full file read saved per write). | Either `update_many()` improvement above plus an atomic validate-hook, or the `needs_doc_id`/`DocId` evaluator change combined with a public batched read-modify-write API. |
82
92
83
93
### Resolved upstream in TinyDB
84
94
@@ -112,6 +122,8 @@ All observations in this section were verified against Pydantic 2.11 (the minimu
112
122
113
123
### `model_config` merges across bases in "last wins" order, not MRO
**Limitation.** Pydantic builds a model's effective [`model_config`][pydantic.BaseModel.model_config] by walking the bases left to right and letting each one overwrite what came before, so the _last_ base to set a key wins. Python's MRO says the _first_ one does. The two orderings are exact opposites, and nothing warns when they disagree ([pydantic#9992](https://github.com/pydantic/pydantic/issues/9992), open, carrying the v3 milestone):
116
128
117
129
```python
@@ -143,6 +155,8 @@ The cost of the workaround is real but small: `tinydantic` config does not show
143
155
144
156
### `protected_namespaces` replaces the default instead of extending it
**Limitation.**`protected_namespaces` is an ordinary `ConfigDict` key, so setting it on a subclass _replaces_ the inherited value rather than adding to it. A library that wants to reserve its own prefix has to restate Pydantic's defaults alongside it, and there is no public constant to restate them from — the value lives in `pydantic._internal._config.config_defaults`. Forgetting to restate them is silent: nothing warns that a model just gave up namespace protection.
147
161
148
162
```python
@@ -177,8 +191,8 @@ Every approved use of a Pydantic internal/private API in `tinydantic` is recorde
177
191
178
192
| Pydantic internal | Used by | Status | Reason | Upstream change that would remove it |
179
193
| --- | --- | --- | --- | --- |
180
-
|`pydantic._internal._model_construction.ModelMetaclass` (name only, under `TYPE_CHECKING`) |`TinydanticModelMetaclass`, the metaclass of `TinydanticModel` (`src/tinydantic/_model.py`) |**In use** (shipped with the original model work; recorded here 2026-08-08) |`tinydantic` must run at class-creation time — to capture `database=`/`table_name=` class keywords, resolve `__tinydantic_config__`, and detect shadowed fields — which requires subclassing `BaseModel`'s metaclass. Pydantic exports no public name for it. The exposure is limited to type-checking: the import sits in an `if TYPE_CHECKING` block and the runtime branch resolves the same class as `type(BaseModel)`, so no Pydantic internal is imported at runtime and moving the module would not break an installed `tinydantic`. | A public re-export, e.g. `pydantic.ModelMetaclass`, so the type-checking import can name a supported path. |
181
-
|`pydantic._internal._config.config_defaults["protected_namespaces"]` (test-only) |`tests/model/test_model_config.py` — asserting `tinydantic`'s reservations still cover Pydantic's defaults |**In use** (added with the `protected_namespaces` fix above) | The default has no public accessor, and the alternative to reading it is hard-coding the same tuple in the test that is supposed to catch it drifting. Test-only, so no shipped code depends on it; if Pydantic moves the attribute the test fails loudly rather than silently passing. | The `DEFAULT_PROTECTED_NAMESPACES` constant suggested above. |
194
+
|`pydantic._internal._model_construction.ModelMetaclass` (name only, under `TYPE_CHECKING`) |`TinydanticModelMetaclass`, the metaclass of `TinydanticModel` (`src/tinydantic/_model.py`) |**In use** (shipped with the original model work; recorded here 2026-08-08). Tracking: [#140](https://github.com/tinydantic/tinydantic/issues/140)|`tinydantic` must run at class-creation time — to capture `database=`/`table_name=` class keywords, resolve `__tinydantic_config__`, and detect shadowed fields — which requires subclassing `BaseModel`'s metaclass. Pydantic exports no public name for it. The exposure is limited to type-checking: the import sits in an `if TYPE_CHECKING` block and the runtime branch resolves the same class as `type(BaseModel)`, so no Pydantic internal is imported at runtime and moving the module would not break an installed `tinydantic`. | A public re-export, e.g. `pydantic.ModelMetaclass`, so the type-checking import can name a supported path. |
195
+
|`pydantic._internal._config.config_defaults["protected_namespaces"]` (test-only) |`tests/model/test_model_config.py` — asserting `tinydantic`'s reservations still cover Pydantic's defaults |**In use** (added with the `protected_namespaces` fix above). Tracking: [#139](https://github.com/tinydantic/tinydantic/issues/139)| The default has no public accessor, and the alternative to reading it is hard-coding the same tuple in the test that is supposed to catch it drifting. Test-only, so no shipped code depends on it; if Pydantic moves the attribute the test fails loudly rather than silently passing. | The `DEFAULT_PROTECTED_NAMESPACES` constant suggested above. |
0 commit comments