Skip to content

Assignments to function/attribute calls are stripped from .pyi output #60

Closed

Description

@mdboom

We encountered this issue while migrating cuda-core from stubgen-pyx 0.2.6 to 0.2.19.

An LLM-generated summary is below:

Summary

When stubgen-pyx encounters a module-level assignment whose RHS is a
function/attribute call (e.g. X = namedtuple("X", (...)), X = int(some_c_enum)),
it strips the assignment out of the generated .pyi entirely
(strip_artifacts.py::_is_runtime_assignment), even for public names and even
with --include-private. This has three compounding, user-visible effects:

  1. Public names silently disappear from the stub. If another module does
    from pkg.repro import Point, the generated stub for the importing module
    still emits that import, but Point is simply gone from repro.pyi -- the
    import now points at nothing, which mypy flags as an import error.
  2. Dangling references in other, still-emitted assignments. If a dropped
    name is referenced inside a dict/list/tuple literal that itself survives
    (because such literals aren't classified as "runtime assignments"), the
    literal is copied into the stub verbatim, including the now-undefined name.
    mypy then reports "Name is not defined".
  3. Invalid ... annotations for TypedDict-style struct fields. Separately
    (but for a related reason -- an annotation referencing a name that's absent
    from the stub, here a C function-pointer ctypedef from a cdef extern
    block), trim_not_defined.py replaces the undefined annotation with a bare
    ast.Constant(...) rather than Any. strip_artifacts.py's own
    _AnnotationRewriter uses Any for the equivalent case just a few frames
    away in the same pipeline, so the two undefined-name-repair passes disagree,
    and the ...-annotation is invalid Python typing syntax outside of a
    Callable[..., X]/tuple context -- mypy rejects it with
    error: Unexpected "..." [misc].

Minimal repro

pkg/repro.pyx:

from collections import namedtuple

# (1) Public, Call-valued module assignment: silently dropped from the stub,
# even though it's part of the module's public surface.
Point = namedtuple("Point", ("x", "y"))

# (2) Private, Call-valued module assignment: also dropped ...
_UNIT = int(1)

# ... but this dict literal (kept verbatim, since dict literals aren't
# "runtime assignments") still references it, leaving a dangling name.
LOOKUP = {"a": _UNIT}

cdef extern from "repro.h":
    ctypedef int (*Callback)(int)

    ctypedef struct Foo:
        Callback cb
$ stubgen-pyx pkg --continue-on-error --include-private
2026-08-17 11:17:22,130 - INFO - stubgen-pyx v0.2.19
2026-08-17 11:17:22,286 - WARNING - Replaced undefined name 'Callback' with '...'
2026-08-17 11:17:22,290 - INFO - Successfully converted 1 file(s)

pkg/repro.pyi (generated):

# This file was generated by stubgen-pyx v0.2.19 from pkg/repro.pyx

from typing import TypedDict

LOOKUP = {'a': _UNIT}

class Foo(TypedDict):
    cb: ...

Note:

  • Point is entirely absent (no trace at all) -- any downstream module doing
    from pkg.repro import Point now gets a broken stub-only import.
  • LOOKUP survived, but _UNIT (referenced inside it) did not -- _UNIT is
    undefined in the stub, so mypy reports
    error: Name "_UNIT" is not defined [name-defined].
  • Foo.cb: ... is not valid as a plain annotation -- mypy reports
    error: Unexpected "..." [misc].

Versions: stubgen-pyx 0.2.19, Cython 3.2.9, Python 3.14, mypy 1.20.0.

Where this comes from

  • conversion/strip_artifacts.py::_is_runtime_assignment treats any
    module-level ast.Assign whose value is an ast.Call or ast.Attribute
    (and isn't a TypeVar(...) call) as removable "runtime-only" state, via
    _keep_node. This runs regardless of whether the target name is public,
    and regardless of --include-private.
  • Other module-level statements (e.g. dict/list/tuple literals) that
    reference a name removed this way are not rewritten -- only
    ast.Name/ast.Attribute nodes that occur in annotation positions get
    swept up by _AnnotationRewriter (which correctly substitutes Any).
    Plain Assign values are never passed through that rewriter, so dangling
    references in kept literals are left untouched.
  • Separately, conversion/trim_not_defined.py::_NotDefinedRemover runs
    earlier(?) in the pipeline and also repairs "undefined name in annotation"
    situations, but substitutes ast.Constant(...) (bare Ellipsis) instead of
    Any. This is inconsistent with strip_artifacts.py's own repair (Any)
    and produces syntactically-invalid-for-mypy annotations whenever the
    undefined name occurs as a bare field/variable annotation rather than inside
    a Callable[...].

Suggested fixes

  1. In _is_runtime_assignment, don't drop Assign nodes whose target is
    referenced elsewhere in the kept AST (or, more simply, don't drop
    Assign nodes at all when their target name is used by any other kept
    statement -- compute this after the initial pass, similar to how
    strip_artifacts.py already tracks defined).
  2. At minimum, when a Call/Attribute-valued assignment is dropped, also
    rewrite (or drop) any surviving statement that references the removed
    name, the same way _AnnotationRewriter does for annotations.
  3. Make trim_not_defined.py's undefined-name substitution use Any (like
    strip_artifacts.py does) instead of bare ast.Constant(...), at least
    for AnnAssign/annotation contexts outside of Callable[...]. ... is
    only valid as a type-expression fragment inside Callable[..., X],
    tuple[X, ...], etc. -- never as the entire annotation.

Impact

We hit all three of these while working through the .pyi output for a
larger Cython package (cuda_core in NVIDIA/cuda-python) migrating from
stubgen-pyx 0.2.6 -> 0.2.19. We were able to work around (1) and (2) locally
by adding explicit type annotations (turning Assign into AnnAssign, which
_is_runtime_assignment doesn't touch), but that's not viable for (3) -- the
struct fields have named C function-pointer ctypedefs that have no
reasonable Python-level annotation we can add ourselves.

Activity

  1. jon-edward commented on Aug 18, 2026

    @jon-edward
    Owner

    strip_artifacts postprocessing pass definitely needs to be reworked, thank you for reporting this. I’ll work on fixing the behavior

  2. jon-edward commented on Aug 18, 2026

    @jon-edward
    Owner

    I think the c?import cython/cpython.* (and from) should probably be discarded through the stubgen_pyx.conversion.Converter by something similar to _is_cxx_import

    Then, we can remove the strip_artifacts step completely and make trim_not_defined more robust to Any over ... and undefined decorator deletion. I really don't like dangling names being handled independently by two steps as it introduces conflicting behavior

    I’ll work on this over the next couple days (work is pretty busy these days, sorry!), just wanted to get my thoughts out there

  3. jon-edward commented on Aug 21, 2026

    @jon-edward
    Owner

    @mdboom This should be fixed with v0.2.20, and now produces the following output for your sample:

    from collections import namedtuple
    from typing import Callable, TypeAlias, TypedDict
    
    Point = namedtuple('Point', ('x', 'y'))
    _UNIT = int(1)
    LOOKUP = {'a': _UNIT}
    Callback: TypeAlias = Callable[[int], int]
    
    class Foo(TypedDict):
        cb: Callback

    Would you mind verifying that all is good for the cuda-core usage and closing if satisfactory? I opted for _typeshed.Incomplete over typing.Any to distinguish between the two cases for the trim_not_defined pass

  4. mdboom commented on Aug 25, 2026

    @mdboom
    ContributorAuthor

    I think this issue can be closed. I ran into a different issue upgrading to v0.2.20, reported in #69.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions