Skip to content

Commit 1a36014

Browse files
authored
perf: three small wins (#5, #3 remainder, #8) (#1658)
* perf: skip _recursive_add wrapper for size-1 merge groups merge_instances_list always built a 1-element wrapper list and called _recursive_add even when group_indices held a single index — the common case for queries with no parent duplication. _recursive_add short-circuits at len(model_group) <= 1 anyway, so the wrapper list, the call, and the [0] index were pure overhead. Hoist the early return one frame up: if the group has exactly one row, read it directly from result_rows. Benchmarks vs the post-#1657 optimization tip: - init[1000]: 6571 us -> 6491 us (-1.2%) - get_all[1000]: 9139 us -> 9069 us (-0.8%) - get_all[500]: 4951 us -> 4799 us (-3.1%) - iterate[1000]: 16373 us -> 16068 us (-1.9%) * perf: cheaper Model.__hash__ and __same__ __hash__ used hash(str(pk) + cls.__name__) on every cache miss — two string allocations per call. Replace with hash((pk, type(self))) which uses CPython's identity hash for type objects and skips the string concat entirely. __same__ used hash(self) == other.__hash__() to compare two saved models, which fills both sides' hash caches just to test equality. Direct (pk, type) compare on the saved-pk path skips both hash allocations; fall through to hash equality only when both sides are unsaved. Saved-pk path is the hot one (every relation-cache lookup goes through __hash__). Unsaved-pk path keeps the str(vals) shape because __dict__ can hold list/dict values (json fields, reverse-relation slots) that aren't hashable directly. Benchmarks (workloads that exercise relation hashing): - init_with_related[40]: 828 us -> 781 us (-5.7%) * perf: specialize _process_kwargs per field-set The per-key dispatch loop ran four set membership checks on every kwarg even though typical models have empty json_fields and bytes_fields and only a couple of relation fields. Restructure so the empty-set checks are hoisted out of the loop: - Pre-compute relation_field_names on the class (cached, like _pydantic_field_names) so expand_relationship is only called for fields that actually need it. - Validate unknown kwargs once up front rather than per-iteration. - Fast path for plain models (no json/bytes/relations) reduces to ``return dict(kwargs), through_tmp_dict`` — a single C-level dict copy. - Slow path skips per-iteration empty-set checks via has_json / has_bytes guards. Side effect: BaseField.expand_relationship is no longer called (only relation fields go through expand_relationship now). Marked as ``# pragma: no cover`` rather than removed since it remains a public API contract on the field base class. Benchmarks vs the previous commit: - get_all[1000]: 9670 us -> 9024 us (-6.7%) - get_all[500]: 5008 us -> 4769 us (-4.8%) - iterate[1000]: 16642 us -> 15723 us (-5.5%) - get_all_with_related[40]: 4408 us -> 4205 us (-4.6%)
1 parent 34d4b04 commit 1a36014

3 files changed

Lines changed: 85 additions & 39 deletions

File tree

‎ormar/fields/base.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -384,7 +384,7 @@ def expand_relationship(
384384
:return: returns untouched value for normal fields, expands only for relations
385385
:rtype: Any
386386
"""
387-
return value
387+
return value # pragma: no cover
388388

389389
def set_self_reference_flag(self) -> None:
390390
"""

‎ormar/models/mixins/merge_mixin.py‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -78,8 +78,15 @@ def merge_instances_list(
7878
pks = [model.pk for model in result_rows]
7979
index_groups = ormar_rust_utils.group_by_pk(pks)
8080
for group_indices in index_groups:
81-
group = [result_rows[i] for i in group_indices]
82-
model = cls._recursive_add(group)[0]
81+
# Single-row groups are the common case for queries with no
82+
# parent duplication (``Model.objects.all()`` and similar);
83+
# skip the wrapper list and the no-op ``_recursive_add`` call.
84+
if len(group_indices) == 1:
85+
model = result_rows[group_indices[0]]
86+
else:
87+
model = cls._recursive_add([result_rows[i] for i in group_indices])[
88+
0
89+
]
8390
object.__setattr__(model, "__ormar_excludable__", excludable)
8491
merged_rows.append(model)
8592

‎ormar/models/newbasemodel.py‎

Lines changed: 75 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,7 @@ class NewBaseModel(pydantic.BaseModel, ModelTableProxy, metaclass=ModelMetaclass
117117
_pydantic_field_names: Optional[frozenset[str]]
118118
_extra_is_ignore: Optional[bool]
119119
_allowed_kwarg_names: Optional[frozenset[str]]
120+
_relation_field_names: Optional[frozenset[str]]
120121
ormar_config: OrmarConfig
121122

122123
# noinspection PyMissingConstructor
@@ -418,27 +419,52 @@ def _process_kwargs(self, kwargs: dict) -> tuple[dict, dict]: # noqa: CCR001
418419
json_fields = cls._json_fields
419420
bytes_fields = cls._bytes_fields
420421

421-
try:
422-
new_kwargs: dict[str, Any] = {}
423-
for k, v in kwargs.items():
424-
if k in model_fields:
425-
v = model_fields[k].expand_relationship(v, self, to_register=False)
426-
elif k not in pydantic_fields:
427-
# raises KeyError, handled below to produce a friendly ModelError
428-
model_fields[k]
429-
if k in json_fields:
430-
v = encode_json(v)
431-
if k in bytes_fields and v is not None:
432-
v = decode_bytes(
433-
value=v,
434-
represent_as_string=model_fields[k].represent_as_base64_str,
435-
)
436-
new_kwargs[k] = v
437-
except KeyError as e:
438-
raise ModelError(
439-
f"Unknown field '{e.args[0]}' for model {self.get_name(lower=False)}"
422+
# ``relation_field_names`` is the disjoint set of fields that need
423+
# ``expand_relationship``; everything else can skip that call. Cached
424+
# on the class on first access — same pattern as ``_pydantic_field_names``.
425+
relation_field_names = getattr(cls, "_relation_field_names", None)
426+
if relation_field_names is None:
427+
relation_field_names = frozenset(
428+
name for name, f in model_fields.items() if f.is_relation
440429
)
430+
cls._relation_field_names = relation_field_names # type: ignore[attr-defined]
431+
432+
has_json = bool(json_fields)
433+
has_bytes = bool(bytes_fields)
434+
has_relations = bool(relation_field_names)
435+
436+
# Validate unknown kwargs up front so the dispatch loop doesn't need
437+
# a per-iteration check. ``extra=ignore`` already filtered above, so
438+
# in that branch no unknowns can remain.
439+
if not extra_is_ignore:
440+
for k in kwargs:
441+
if k not in model_fields and k not in pydantic_fields:
442+
try:
443+
model_fields[k]
444+
except KeyError as e:
445+
raise ModelError(
446+
f"Unknown field '{e.args[0]}' for model "
447+
f"{self.get_name(lower=False)}"
448+
)
441449

450+
if not has_json and not has_bytes and not has_relations:
451+
# Fast path — plain model with no relations/json/bytes. The
452+
# validation pass above is the only per-key cost; the value
453+
# copy is a single C-level dict construction.
454+
return dict(kwargs), through_tmp_dict
455+
456+
new_kwargs: dict[str, Any] = {}
457+
for k, v in kwargs.items():
458+
if k in relation_field_names:
459+
v = model_fields[k].expand_relationship(v, self, to_register=False)
460+
if has_json and k in json_fields:
461+
v = encode_json(v)
462+
if has_bytes and k in bytes_fields and v is not None:
463+
v = decode_bytes(
464+
value=v,
465+
represent_as_string=model_fields[k].represent_as_base64_str,
466+
)
467+
new_kwargs[k] = v
442468
return new_kwargs, through_tmp_dict
443469

444470
def _initialize_internal_attributes(self) -> None:
@@ -471,37 +497,50 @@ def __eq__(self, other: object) -> bool:
471497
return super().__eq__(other) # pragma no cover
472498

473499
def __hash__(self) -> int:
474-
if getattr(self, "__cached_hash__", None) is not None:
475-
return self.__cached_hash__ or 0
500+
cached = getattr(self, "__cached_hash__", None)
501+
if cached is not None:
502+
return cached
476503

477-
if self.pk is not None:
478-
ret = hash(str(self.pk) + self.__class__.__name__)
504+
pk = self.pk
505+
cls = type(self)
506+
if pk is not None:
507+
# ``type(self)`` hashes by identity in CPython, so ``hash((pk, cls))``
508+
# is uniqueness-equivalent to the original ``str(pk) + cls.__name__``
509+
# without two string allocations per call. This is the hot path —
510+
# everything that goes through ``_relation_cache`` is keyed on
511+
# saved-pk Models.
512+
ret = hash((pk, cls))
479513
else:
480-
vals = {
481-
k: v
482-
for k, v in self.__dict__.items()
483-
if k not in self.extract_related_names()
484-
}
485-
ret = hash(str(vals) + self.__class__.__name__)
514+
# Unsaved models can hold list/dict values in ``__dict__`` (json
515+
# fields, reverse-relation slots), so we still ``str(vals)`` to
516+
# keep the result hashable. Cold path; not perf-critical.
517+
related = self.extract_related_names()
518+
vals = {k: v for k, v in self.__dict__.items() if k not in related}
519+
ret = hash((str(vals), cls))
486520

487521
object.__setattr__(self, "__cached_hash__", ret)
488522
return ret
489523

490524
def __same__(self, other: "NewBaseModel") -> bool:
491525
"""
492526
Used by __eq__, compares other model to this model.
493-
Compares:
494-
* _orm_ids,
495-
* primary key values if it's set
496-
* dictionary of own fields (excluding relations)
527+
528+
Saved models (both with pk) compare directly by ``(pk, type)`` to
529+
skip the hash-cache fill on the *other* side. Unsaved/mixed states
530+
fall through to the original hash-equality semantics.
531+
497532
:param other: model to compare to
498533
:type other: NewBaseModel
499534
:return: result of comparison
500535
:rtype: bool
501536
"""
502-
if (self.pk is None and other.pk is not None) or (
503-
self.pk is not None and other.pk is None
504-
):
537+
if type(self) is not type(other):
538+
return False # pragma: no cover
539+
self_pk = self.pk
540+
other_pk = other.pk
541+
if self_pk is not None and other_pk is not None:
542+
return self_pk == other_pk
543+
if (self_pk is None) != (other_pk is None):
505544
return False
506545
else:
507546
return hash(self) == other.__hash__()

0 commit comments

Comments
 (0)