Skip to content

Defining a Struct on a non-Struct base leaks that base's type dict #1198

Closed

Description

@Siyet

On Python 3.12, 3.13 and 3.14, every Struct class defined with a non-Struct base retains one reference to that base's type dict. When the base is created at runtime, its namespace and everything stored in it then stay alive for the life of the process. This covers ordinary mixins and user-defined subclasses of Generic[T]. A struct written directly on Generic[T] retains a reference to the type dict of typing.Generic itself, which costs a reference count rather than memory.

Environment: reproduced on msgspec 0.21.1 (PyPI wheels) and on current main, with CPython 3.12.13, 3.13.14 and 3.14.6 on macOS arm64. Not reproduced on CPython 3.10.20, 3.11.15 or 3.15.0b4.

Minimal reproduction:

import gc
import weakref

import msgspec


def define_struct_on_a_fresh_base():
    class Sentinel:
        pass

    sentinel = Sentinel()
    Mixin = type("Mixin", (), {"sentinel": sentinel})

    class Example(Mixin, msgspec.Struct):
        x: int

    return weakref.ref(sentinel)


refs = [define_struct_on_a_fresh_base() for _ in range(100)]
gc.collect()
print("base attributes still alive:", sum(r() is not None for r in refs), "of", len(refs))

On 3.12, 3.13 and 3.14 this prints base attributes still alive: 100 of 100. On 3.10 and 3.11 it prints 0 of 100, which is what I would expect everywhere.

On 3.15.0b4 it also prints 0 of 100, and that one is not msgspec. The missing release is still there, but 3.15 empties the dict of an unreachable heap type even when that dict has outside references: holding a class namespace through an ordinary reference, dropping the class and collecting leaves the keys intact on 3.13 and 3.14 and leaves an empty dict on 3.15. What remains there is one empty dict per definition rather than the whole base namespace.

How much is retained depends on what the base holds, since the whole namespace survives. Storing a 20 KB payload on each freshly created mixin and defining one struct per mixin, tracemalloc on 3.14.6 reports 4,534,666 bytes still traced after 200 iterations and a gc.collect(), against 85,592 bytes once the reference is released and 36,888 bytes for a control that creates the same mixins without defining a struct. The exact counts shift a little between CPython versions.

This only matters where the base type itself is created at runtime. A mixin defined at import time is alive for the process anyway, so the retained reference costs nothing beyond a count.

Cause: MS_GET_TYPE_DICT expands to PyType_GetDict on 3.12+, which returns a strong reference, while the comment above the macro in src/msgspec/_core.c calls the result borrowed. The only call site, in structmeta_collect_base, releases it on neither the success path nor the error path that rejects a base defining __init__ or __new__. Before 3.12 the macro reads tp_dict directly, which is borrowed, so nothing needs releasing there. The macro was introduced in #745.

A fix with regression tests is ready, and I will open a PR referencing this issue.

Activity

  1. added a commit that references this issue on Sep 29, 2026
    6d3443f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions