Skip to content

C API exposes pointers to real Python objects #37

@steve-s

Description

@steve-s

The fact that PyObject* is a real pointer to some runtime data-structure can be (ab)used in many ways. Those may slowly become part of the contract of the C API even if not intended.

First problem is that by comparing the pointer one can check if two objects are the same. This forces the Python engine to keep mapping between its own internal objects and whatever is handed out to extensions, because "whatever is handed out to extensions" must be always the same for the same object.

It is trivial if the objects have stable address, but with a moving GC this is not the case. Both Java and .NET and many other systems have moving GC, so their objects do not have stable address, and explicit mapping must be maintained just for the sake of maintaining the expectations of the extensions that PyObject* is a real pointer even if opaque.

What I've seen in other languages is also this "nice" trick where people copy the definition of a C structure and then cast the opaque pointer they've gotten from the runtime to that struct so that they can access its internals. It can go as far as defining several versions of said struct for several versions of the interpreter and choosing one at runtime. Now, all the complications and breakages that this causes are the problem of people who decide to use such tricks (and maybe sometimes it is necessary, I am not judging). I am pointing this out as another example why not exposing pointers to the real data (or at least specify that what people get from API should be seen as opaque identifier of Python object, but not a pointer to it) would be better.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions