Would it be possible to implement a project like PyPy cpyext but for CPython: emulate the Python C API... on top of the Python C API? :-)

If such project is feasible, it would be easier to remove functions which are causing us troubles (ex: borrowed references).

For me, there are multiple technical challenges which make such project either really complex to implement, inefficient, and may lead to inconsistent objects.

PyObject* is really a pointer in C: people rely on its address, Py_Is() function was only added recently. In PyPy, when a PyPy object is converted to a PyObject, the PyPy object becomes a link to the PyObject object. Would it make sense to have an internal PyObject and a compatibility PyObject, but both would have different memory addresses? Also, how are writes into these objects propagated between the two objects? The C API is based on the assumption that data can be freely read and write without Python being aware of it: see issue #57 "Function must not return a pointer to content without an explicit resource management". Should we duplicate all objects in memory? That sounds very inefficient.

C extensions expect structure members to not change and have the same behavior. For exemple, in Python 3.11, PyFrameObject members were removed from the C API: it sounds really hard to provide a backward compatibility PyFrameObject structure which is more or less consistent with the real data of the internal PyFrameObject strucutre.