Skip to content

[Python][C++] Python 3.15 Builds Fail due to PyWeakref_GET_OBJECT Removal #47823

New issue
New issue
Closed
#48027
Closed
[Python][C++] Python 3.15 Builds Fail due to PyWeakref_GET_OBJECT Removal#47823
#48027
Assignees
paultiq
Labels
Component: PythonType: bug
Milestone
23.0.0

Description

@p-vdp
p-vdp
opened on Oct 14, 2025

Describe the bug, including details regarding any error messages, version, and platform.

Python v3.15.0a1 is out, and after building it from source I'm trying to build pyarrow from apache-arrow-22.0.0-rc0.

I'm following the same steps to build pyarrow that I've used successfully on Python 3.14.0.

Here's the error output:

234.8       /root/builds/arrow/python/pyarrow/src/arrow/python/extension_type.cc:
234.8       In member function ‘PyObject* arrow::py::PyExtensionType::GetInstance()
234.8       const’:
234.8       /root/builds/arrow/python/pyarrow/src/arrow/python/extension_type.cc:167:20:
234.8       error: ‘PyWeakref_GET_OBJECT’ was not declared in this scope
234.8         167 |   PyObject* inst = PyWeakref_GET_OBJECT(type_instance_.obj());
234.8             |                    ^~~~~~~~~~~~~~~~~~~~
234.8       gmake[2]: *** [CMakeFiles/arrow_python.dir/build.make:160:
234.8       CMakeFiles/arrow_python.dir/pyarrow/src/arrow/python/extension_type.cc.o]
234.8       Error 1
234.8       gmake[2]: *** Waiting for unfinished jobs....
234.8       gmake[1]: *** [CMakeFiles/Makefile2:126:
234.8       CMakeFiles/arrow_python.dir/all] Error 2
234.8       gmake: *** [Makefile:136: all] Error 2
234.8       error: command '/usr/bin/cmake' failed with exit code 2

I'm running this in Docker on Ubuntu 24.04, it's available as the next image here.

Component(s)

Python, C++

Activity

  1. p-vdp
    added
    Type: bug
    on Oct 14, 2025
  2. github-actions
    added
    Component: Python
    on Oct 14, 2025
  3. p-vdp
    changed the title [-][Python] Building `pyarrow` fails on apache-arrow-22.0.0-rc0 and Python 3.15.0a1: `'PyWeakref_GET_OBJECT’ was not declared`[/-] [+][Python] Building `pyarrow` fails on apache-arrow-22.0.0-rc1 and Python 3.15.0a1: `'PyWeakref_GET_OBJECT’ was not declared`[/+] on Oct 20, 2025

  4. p-vdp commented on Oct 20, 2025

    @p-vdp
    p-vdp
    on Oct 20, 2025
    Author

    Still getting this with rc1:

    247.4       In member function ‘PyObject* arrow::py::PyExtensionType::GetInstance()
247.4       const’:
247.4       /root/builds/arrow/python/pyarrow/src/arrow/python/extension_type.cc:167:20:
247.4       error: ‘PyWeakref_GET_OBJECT’ was not declared in this scope
247.4         167 |   PyObject* inst = PyWeakref_GET_OBJECT(type_instance_.obj());
247.4             |                    ^~~~~~~~~~~~~~~~~~~~
  5. p-vdp
    changed the title [-][Python] Building `pyarrow` fails on apache-arrow-22.0.0-rc1 and Python 3.15.0a1: `'PyWeakref_GET_OBJECT’ was not declared`[/-] [+][Python][C++] Building `pyarrow` fails on apache-arrow-22.0.0-rc1 and Python 3.15.0a1: `'PyWeakref_GET_OBJECT’ was not declared`[/+] on Oct 21, 2025

  6. p-vdp commented on Oct 21, 2025

    @p-vdp
    p-vdp
    on Oct 21, 2025
    Author

    PyWeakref_GET_OBJECT has been deprecated since Python 3.13 and is being removed in 3.15: https://docs.python.org/3/c-api/weakref.html#c.PyWeakref_GetObject

    arrow/python/pyarrow/src/arrow/python/extension_type.cc

    Line 167 in 7df396e

    PyObject* inst = PyWeakref_GET_OBJECT(type_instance_.obj());

  7. AlenkaF commented on Oct 21, 2025

    @AlenkaF
    AlenkaF
    on Oct 21, 2025
    Member

    Thank you for opening up the issue @p-vdp !
    We do not yet support Python 3.15 but it's very helpful to get this kind of feedback early.

    The substitute for PyWeakref_GET_OBJECT will be PyWeakref_GetRef which was added in 3.13. We will need a solution that works for all supported Python versions (currently 3.10 to 3.14).

  8. p-vdp
    changed the title [-][Python][C++] Building `pyarrow` fails on apache-arrow-22.0.0-rc1 and Python 3.15.0a1: `'PyWeakref_GET_OBJECT’ was not declared`[/-] [+][Python][C++] Python 3.15 support[/+] on Oct 29, 2025

  9. paultiq commented on Nov 2, 2025

    @paultiq
    paultiq
    on Nov 2, 2025
    Contributor

    Would you consider an #if block specific for 3.15+ like the following, leaving <=3.14 alone.

    * This built/ran/passed tests on my local system.

    Other approaches:

    diff --git a/python/pyarrow/src/arrow/python/extension_type.cc b/python/pyarrow/src/arrow/python/extension_type.cc
index 8439ecf858..a2110f9799 100644
--- a/python/pyarrow/src/arrow/python/extension_type.cc
+++ b/python/pyarrow/src/arrow/python/extension_type.cc
@@ -164,6 +164,22 @@ PyObject* PyExtensionType::GetInstance() const {
     return nullptr;
   }
   ARROW_DCHECK(PyWeakref_CheckRef(type_instance_.obj()));
+#if PY_VERSION_HEX >= 0x030F0000  // Python 3.15+
+  PyObject* inst = NULL;
+  int result = PyWeakref_GetRef(type_instance_.obj(), &inst);
+  if (result < 0) {
+    // Error
+    return nullptr;
+  } else if (result == 0) {
+    // Reconstruct - XXX cache again? 
+    return DeserializeExtInstance(type_class_.obj(), storage_type_, serialized_);
+  } else {
+    // Cached instance still alive
+    return inst;
+  }
+#else
+  // PyWeakref_GET_OBJECT removed in 3.15
   PyObject* inst = PyWeakref_GET_OBJECT(type_instance_.obj());
   if (inst != Py_None) {
     // Cached instance still alive
     Py_INCREF(inst);
     return inst;
   } else {
     // Must reconstruct from serialized form
     // XXX cache again?
     return DeserializeExtInstance(type_class_.obj(), storage_type_, serialized_);
   }
+#endif
 }

  10. raulcd commented on Nov 3, 2025

    @raulcd
    raulcd
    on Nov 3, 2025
    Member

    In my opinion we should use pythoncapi-compat as it is already vendored in the project and used for other compatibility functions like PyDict_GetItemStringRef or PyList_GetItemRef.

    arrow/python/pyarrow/src/arrow/python/vendored/pythoncapi_compat.h

    Lines 561 to 584 in ff089cc

    // gh-105927 added PyWeakref_GetRef() to Python 3.13.0a1
    #if PY_VERSION_HEX < 0x030D0000
    static inline int PyWeakref_GetRef(PyObject *ref, PyObject **pobj)
    {
    PyObject *obj;
    if (ref != NULL && !PyWeakref_Check(ref)) {
    *pobj = NULL;
    PyErr_SetString(PyExc_TypeError, "expected a weakref");
    return -1;
    }
    obj = PyWeakref_GetObject(ref);
    if (obj == NULL) {
    // SystemError if ref is NULL
    *pobj = NULL;
    return -1;
    }
    if (obj == Py_None) {
    *pobj = NULL;
    return 0;
    }
    *pobj = Py_NewRef(obj);
    return (*pobj != NULL);
    }
    #endif

  11. paultiq commented on Nov 3, 2025

    @paultiq
    paultiq
    on Nov 3, 2025
    Contributor

    That works too!

    In that case, it'd be:

    diff --git a/python/pyarrow/src/arrow/python/extension_type.cc b/python/pyarrow/src/arrow/python/extension_type.cc
+ #include "arrow/python/vendored/pythoncapi_compat.h"
...
@@ -164,10 +165,13 @@ PyObject* PyExtensionType::GetInstance() const {
     return nullptr;
   }
   ARROW_DCHECK(PyWeakref_CheckRef(type_instance_.obj()));
-  PyObject* inst = PyWeakref_GET_OBJECT(type_instance_.obj());
-  if (inst != Py_None) {
+  PyObject* inst = NULL;
+  int result = PyWeakref_GetRef(type_instance_.obj(), &inst);
+  if (result < 0) {
+    // Error
+    return nullptr;
+  } else if (result > 0) {
     // Cached instance still alive
-    Py_INCREF(inst);
     return inst;
   } else {
    // Must reconstruct from serialized form
    // XXX cache again?
    return DeserializeExtInstance(type_class_.obj(), storage_type_, serialized_);
  }
 }

  12. raulcd commented on Nov 3, 2025

    @raulcd
    raulcd
    on Nov 3, 2025
    Member

    @paultiq do you want to push a PR? I am happy to follow up with testing/review

  13. github-actions
    assigned
    paultiq
    on Nov 3, 2025

  14. paultiq commented on Nov 3, 2025

    @paultiq
    paultiq
    on Nov 3, 2025
    Contributor

    PR opened.

    I'd suggest: Changing issue title to "Python 3.15 Builds Fail due to PyWeakref_GET_OBJECT Removal"? There's likely many other steps involved for 3.15 support.

    In terms of next steps, once PR lands, would it make sense to file a separate issue for adding Python 3.15 to nightlys?

  15. p-vdp
    changed the title [-][Python][C++] Python 3.15 support[/-] [+][Python][C++] Python 3.15 Builds Fail due to PyWeakref_GET_OBJECT Removal[/+] on Nov 3, 2025

  16. AlenkaF commented on Nov 4, 2025

    @AlenkaF
    AlenkaF
    on Nov 4, 2025
    Member

    In terms of next steps, once PR lands, would it make sense to file a separate issue for adding Python 3.15 to nightlys?

    Yes, that would make sense. We could try building wheels for Python 3.15 alpha.
    I have no experience here but musllinux might be an issue as we are stuck at June musllinux image, see #48028. Quite possible it wont be a problem though.

  17. AlenkaF
    closed this as completedin #48027on Nov 19, 2025
  18. AlenkaF
    added a commit that references this issue on Nov 19, 2025

    GH-47823: [Python] Use PyWeakref_GetRef instead of PyWeakref_GET_OBJE…

    a158036
  19. AlenkaF
    added this to the 23.0.0 milestone on Nov 19, 2025

  20. AlenkaF commented on Nov 19, 2025

    @AlenkaF
    AlenkaF
    on Nov 19, 2025
    Member

    Issue resolved by pull request 48027
    #48027

  21. mkillik-wayspring
    added a commit that references this issue on Nov 24, 2025

    apacheGH-47823: [Python] Use PyWeakref_GetRef instead of PyWeakref_GE…

    e0876ae
  22. Mottl
    added a commit that references this issue on May 26, 2026

    apacheGH-47823: [Python] Use PyWeakref_GetRef instead of PyWeakref_GE…

    6f48910
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Component: PythonType: bug

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions