Skip to content

Heinsenbug seen against numpy + python-build-standalone builds on Linux #934

Open

Description

@neutrinoceros

I'm seeing a spurious warning from a C extension in numpy, that only appears when running a pre-built interpreter from python-build-standalone (as far as I can tell).

# t.py
from astropy.coordinates import Angle
from astropy.units import degree

angle = Angle("1.0", unit=degree)
angle.to_string()

run with uv run python -Werror t.py

Traceback (most recent call last):
  File "/home/clm/dev/gh/astropy/astropy/t.py", line 5, in <module>
    angle.to_string()
    ~~~~~~~~~~~~~~~^^
  File "/home/clm/dev/gh/astropy/astropy/astropy/coordinates/angles/core.py", line 387, in to_string
    result = format_ufunc(self.to_value(unit))
  File "/home/clm/dev/gh/astropy/astropy/.venv/lib/python3.14/site-packages/numpy/lib/_function_base_impl.py", line 2518, in __call__
    return self._call_as_normal(*args, **kwargs)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/clm/dev/gh/astropy/astropy/.venv/lib/python3.14/site-packages/numpy/lib/_function_base_impl.py", line 2511, in _call_as_normal
    return self._vectorize_call(func=func, args=vargs)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/clm/dev/gh/astropy/astropy/.venv/lib/python3.14/site-packages/numpy/lib/_function_base_impl.py", line 2600, in _vectorize_call
    outputs = ufunc(*args, out=...)
RuntimeWarning: invalid value encountered in do_format (vectorized)

It's a bit opaque since both numpy and astropy's internal logic is involved, and I'm 99% sure astropy isn't actually needed to reproduce (I'll be trying to eliminate this bit shortly), but here are a couple key details:

  • astropy.coordinates.Angle.to_string runs an internal closure that's produced from np.vectorize
  • my understanding is that this warning normally corresponds to an invalid (nan) is received. Here the only value that is actually passed to the vectorized closure is np.ndarray(np.float64(1.0), dtype=object)

I discovered this in CI, and I'm able to reproduce this locally under the following conditions:

  • run on Linux: I can't reprod on macOS
  • run with a release-build of numpy (either a wheel or a local build with default settings): the warning disappears when numpy is compiled in debug mode
  • all versions of Python I tested are affected
  • but only builds from python-build-standalone (that I use via uv): if I build my own interpreter locally, or if I switch to actions/setup-python in CI, no warning is emitted.

Furthermore, my CI started failing with this warning on dec 20. My last successful job ran on dec 18. I'm surprised to see that uv didn't get a release since dec 9. Maybe my mental model for how python-build-standalone is used in uv is just wrong; I was working under the assumption no changes here would affect uv before it was mirrored (upgraded) there.

I'm very much out of my depth and currently out of ideas about where the problem might be coming from exactly, but I only see it with this project's relocatable builds of Python so this is where I'm starting my quest for help. I'll attempt to minimize my reproducer now.

Activity

  1. jjhelmus commented on Jan 6, 2026

    @jjhelmus
    Contributor

    I was able to reproduce this issue on Linux with Python 3.14.2. Have you been able to minimize this at all to something smaller? astropy/astropy#18989 mentions the same RuntimWarning.

  2. neutrinoceros commented on Jan 7, 2026

    @neutrinoceros
    Author

    I was hoping it would take only a couple minutes but this didn't turn out the way I thought, so I haven't found the time to do it yet. Thanks for making the connection with this astropy issue ! clearly I was aware of it at some point but didn't connect the dots. I further note that astropy's CI is using uv via tox-uv but not uv-managed python interpreters. The plot thickens

  3. jjhelmus commented on Jan 7, 2026

    @jjhelmus
    Contributor

    I was able to get this to a much smaller reproducer that only uses NumPy.
    Produces a RuntimeWarning only on python-build-standalone on Linux.

    import numpy as np
    
    
    def func(_):
        # np.<func> is required
        foo = np.sign(1.0)
        # an f-string is required
        bar = f"{foo}"
        return '1'
    
    # otypes is required
    vfunc = np.vectorize(func, otypes="U")
    result = vfunc(1.0)
    print(result)
    $ uvx --with numpy --managed-python --python 3.14 python -Werror bug.py
    Traceback (most recent call last):
      File "/home/jhelmus/debugging/pbs/934_numpy/bug.py", line 13, in <module>
        result = vfunc(1.0)
      File "/home/jhelmus/.cache/uv/archive-v0/M5DN8HT84PasGhJ694PWp/lib/python3.14/site-packages/numpy/lib/_function_base_impl.py", line 2518, in __call__
        return self._call_as_normal(*args, **kwargs)
               ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
      File "/home/jhelmus/.cache/uv/archive-v0/M5DN8HT84PasGhJ694PWp/lib/python3.14/site-packages/numpy/lib/_function_base_impl.py", line 2511, in _call_as_normal
        return self._vectorize_call(func=func, args=vargs)
               ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/jhelmus/.cache/uv/archive-v0/M5DN8HT84PasGhJ694PWp/lib/python3.14/site-packages/numpy/lib/_function_base_impl.py", line 2599, in _vectorize_call
        outputs = ufunc(*args, out=...)
    RuntimeWarning: invalid value encountered in func (vectorized)
    
    $ uvx --with numpy --no-managed-python --python 3.14 python -Werror bug.py
    1
    
  4. neutrinoceros commented on Jan 7, 2026

    @neutrinoceros
    Author

    excellent, that's just what I was aiming for ! Can you clarify what interpreter you're using here ?

  5. jjhelmus commented on Jan 7, 2026

    @jjhelmus
    Contributor

    For the pbs interpreter I'm using the most recent build of 3.14.2 (Dec 17 2025). For comparison I'm using a conda environment with python 3.14.2 from conda-forge. The conda environment does not produce a runtime warning. Both are on a Ubunutu 24.04 system with an x86-64 CPU.

    I was able to remove the use of np.vectorize and use frompyfunc directly to produce the ufunc:

    import numpy as np
    
    def func(_):
        # call to something in numpy, resets error state?
        np.array([1,2,3])
        # a floating point scalar, float32 works but np.array([1], dtype='f8') does not
        foo = np.float64(1.0)
        # an f-string is required
        bar = f"{foo}"
        return '1'
    
    ufunc = np.frompyfunc(func, nin=1, nout=1)
    result = ufunc(1.0)
    print(result)

    numpy/numpy#27760 points out that error reporting in ufuncs created via vectorize (but also likely frompyfunc) is inconsistent.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions