Skip to content

Commit 0f8976a

Browse files
Guilhem Saurelnim65s
authored andcommitted
[Python 3] fix python_parser test
Closing the interpreter is bugged. The idea to remove Py_Finalize comes from numpy/numpy#8097 Those discussions also provide a way to try to clean a bit. Here are some test results of the memory loss on the python_parser test: - python3 - with Py_Finalize unknown location(0): fatal error in "buildModel": memory access violation at address: 0x000000a0: no mapping at fault address - with the workaround > HEAP SUMMARY: > in use at exit: 6,623,041 bytes in 11,478 blocks > total heap usage: 40,988 allocs, 29,510 frees, 35,627,810 bytes allocated > > LEAK SUMMARY: > definitely lost: 162,918 bytes in 86 blocks > indirectly lost: 0 bytes in 0 blocks > possibly lost: 224,284 bytes in 192 blocks > still reachable: 6,235,839 bytes in 11,200 blocks > suppressed: 0 bytes in 0 blocks > > ERROR SUMMARY: 1470 errors from 115 contexts (suppressed: 0 from 0) - with nothing > HEAP SUMMARY: > in use at exit: 6,652,431 bytes in 11,580 blocks > total heap usage: 40,937 allocs, 29,357 frees, 35,604,459 bytes allocated > > LEAK SUMMARY: > definitely lost: 164,417 bytes in 87 blocks > indirectly lost: 0 bytes in 0 blocks > possibly lost: 223,187 bytes in 191 blocks > still reachable: 6,264,827 bytes in 11,302 blocks > suppressed: 0 bytes in 0 blocks > > ERROR SUMMARY: 1438 errors from 107 contexts (suppressed: 0 from 0) - python 2 - with Py_Finalize: > HEAP SUMMARY: > in use at exit: 3,209,728 bytes in 5,514 blocks > total heap usage: 23,363 allocs, 17,849 frees, 27,441,910 bytes allocated > > LEAK SUMMARY: > definitely lost: 0 bytes in 0 blocks > indirectly lost: 0 bytes in 0 blocks > possibly lost: 161,830 bytes in 89 blocks > still reachable: 3,047,898 bytes in 5,425 blocks > suppressed: 0 bytes in 0 blocks > > ERROR SUMMARY: 1902 errors from 95 contexts (suppressed: 0 from 0) - with the workaround: > HEAP SUMMARY: > in use at exit: 5,437,170 bytes in 7,454 blocks > total heap usage: 23,363 allocs, 15,909 frees, 27,441,795 bytes allocated > > LEAK SUMMARY: > definitely lost: 0 bytes in 0 blocks > indirectly lost: 0 bytes in 0 blocks > possibly lost: 177,798 bytes in 115 blocks > still reachable: 5,259,372 bytes in 7,339 blocks > suppressed: 0 bytes in 0 blocks > > ERROR SUMMARY: 1149 errors from 69 contexts (suppressed: 0 from 0) - with nothing: > > HEAP SUMMARY: > in use at exit: 5,472,144 bytes in 7,554 blocks > total heap usage: 23,361 allocs, 15,807 frees, 27,440,943 bytes allocated > > LEAK SUMMARY: > definitely lost: 0 bytes in 0 blocks > indirectly lost: 0 bytes in 0 blocks > possibly lost: 178,318 bytes in 116 blocks > still reachable: 5,293,826 bytes in 7,438 blocks > suppressed: 0 bytes in 0 blocks > > ERROR SUMMARY: 1150 errors from 67 contexts (suppressed: 0 from 0) So I think we should keep Py_Finalize as long as it works (ie. for Python 2), and use the workaround otherwise.
1 parent 81f68a8 commit 0f8976a

1 file changed

Lines changed: 30 additions & 2 deletions

File tree

‎bindings/python/parsers/python/model.cpp‎

Lines changed: 30 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88
#include <Python.h>
99
#include <boost/shared_ptr.hpp>
1010
#include <boost/version.hpp>
11+
#include <boost/algorithm/string/predicate.hpp>
1112

1213
// Boost 1.58
1314
#if BOOST_VERSION / 100 % 1000 == 58
@@ -68,11 +69,38 @@ namespace pinocchio
6869
std::cout << " degrees of freedom." << std::endl;
6970
}
7071

71-
// close interpreter
72+
// close the interpreter
73+
// cf. https://github.com/numpy/numpy/issues/8097
74+
#if PY_MAJOR_VERSION < 3
7275
Py_Finalize();
76+
#else
77+
78+
PyObject * poMainModule = PyImport_AddModule("__main__");
79+
PyObject * poAttrList = PyObject_Dir(poMainModule);
80+
PyObject * poAttrIter = PyObject_GetIter(poAttrList);
81+
PyObject * poAttrName;
82+
83+
while ((poAttrName = PyIter_Next(poAttrIter)) != NULL)
84+
{
85+
std::string oAttrName = PyUnicode_AS_DATA(poAttrName);
86+
87+
// Make sure we don't delete any private objects.
88+
if (!boost::starts_with(oAttrName, "__") || !boost::ends_with(oAttrName, "__"))
89+
{
90+
PyObject * poAttr = PyObject_GetAttr(poMainModule, poAttrName);
91+
92+
// Make sure we don't delete any module objects.
93+
if (poAttr && poAttr->ob_type != poMainModule->ob_type)
94+
PyObject_SetAttr(poMainModule, poAttrName, NULL);
95+
Py_DecRef(poAttr);
96+
}
97+
Py_DecRef(poAttrName);
98+
}
99+
Py_DecRef(poAttrIter);
100+
Py_DecRef(poAttrList);
101+
#endif
73102

74103
return model;
75104
}
76105
} // namespace python
77106
} // namespace pinocchio
78-

0 commit comments

Comments
 (0)