COMPAT: notify garbage collector when memory is allocated - #8907

Thanks @mattip .

Did you really mean PyPyPyGC? It's weird that the ifdef and call don't match...

I'm pretty sure this isn't the only place we allocate memory... (At the least there has to be both malloc and calloc calls.) Is it that the other places go through the PyMem APIs and are already covered, or...?

@njsmith The code as it stands is correct but admittedly misleading, the functions are aliases for one another. Explaination: PyPy prevents mixing CPython headers with the PyPy runtime by add an extra Py to the name exported in the shared object. For instance, _PyPyGC_AddMemoryPressure appears in the public headers and _PyPyPyGC_AddMemoryPressure is exported in the shared object. The header pypy_decl.h contains this magic.

I assumed all ndarray data allocations are done with _npy_alloc_cache (used by both npy_alloc_cache and npy_alloc_cache_zero. It seems that assumption is incorrect only in array_setstate which calls PyDataMem_NEW directly. AFAICT all the other calls are for short-lived buffers that are quickly released, or for small bits of memory needed to hold pointers.

Should array_setstate be using npy_alloc_cache rather than PyDataMem_NEW?

how does this work exactly? When the GC runs it just sets memory pressure back to zero?
If not shouldn't there be a remove pressure function in free?

The first answer is correct. Any GC tracked allocation (all PyPy internal allocations are tracked) is added to a counter, when the counter is large enough a GC collection cycle is triggered. The counter is reset after each cycle. Adding memory pressure simply increments the counter.