Skip to content

Every Struct class definition leaks its annotations dict #1193

Closed

Description

@Siyet

Every successful Struct class definition that carries annotations retains one reference to the annotations dict built for that class. The dict and everything it references stay alive for the life of the process, even after the struct type itself has been collected. A class that carries a string annotation beginning ClassVar or typing.ClassVar, with or without a subscript, additionally retains one reference to the __dict__ of the module it was defined in.

This matters only where struct types are created repeatedly, for example one type per request or per schema, or in a long-running process that builds types in a loop. A fixed set of types defined at import time retains a bounded amount of memory.

Environment: reproduced on msgspec 0.21.1 (PyPI wheels) with CPython 3.11.15 and 3.14.6 on macOS arm64, and on current main.

Minimal reproduction:

import gc
import tracemalloc
import weakref

import msgspec


def define_struct_type():
    class Annotation:
        pass

    class Example(msgspec.Struct):
        x: Annotation

    return weakref.ref(Annotation)


refs = [define_struct_type() for _ in range(100)]
gc.collect()
print("annotation types still alive:", sum(r() is not None for r in refs), "of", len(refs))


def grow(n):
    gc.collect()
    tracemalloc.start()
    for i in range(n):
        msgspec.defstruct(f"Temp{i}", [("a", int), ("b", str), ("c", float)])
    gc.collect()
    current, _ = tracemalloc.get_traced_memory()
    tracemalloc.stop()
    return current


print("traced memory retained after 10000 struct definitions:", grow(10_000), "bytes")

Output on 0.21.1 and on current main, CPython 3.11.15 (the exact byte count varies slightly between Python versions):

annotation types still alive: 100 of 100
traced memory retained after 10000 struct definitions: 1844656 bytes

The same script with a plain class, a dataclass, a NamedTuple or a TypedDict in place of the struct prints 0 of 100, and that is what I would expect here. A struct with no annotations at all does not retain anything.

Cause: in structmeta_collect_fields (src/msgspec/_core.c) both annotations and module_ns are owned references that are released only on the failure paths. The success path returns without releasing either. The annotations half dates from #852, which made the reference owned but added the matching Py_DECREF(annotations) only on the failure paths; before that the reference was borrowed. The module_ns half has been there since #281 added ClassVar support.

A fix with regression tests is ready, and I will open a PR referencing this issue.

Activity

  1. added a commit that references this issue on Sep 29, 2026
    7930780
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions