Extending -fanalyzer to support CPython embedding and extension modules

This page tracks extending -fanalyzer to add domain-specific checking for CPython, to help authors of CPython extension modules, and of projects embedding CPython.

See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107646

For reference, "cpychecker" was an experimental static analyzer DavidMalcolm wrote in 2010-2013 using the gcc-python-plugin, implementing many of these ideas. It was slow, buggy, had to handle both Python 2 and Python 3, and has bit-rotted to uselessness now. But some of the ideas may be applicable to -fanalyzer.

Domain-specific Checks

Reference-count checking

We should try to statically detect bugs in reference-count handling. This is probably the most useful thing to check, but the most difficult. Idea is to track changes to the ob_refcnt field of PyObject instances (and subclasses), and to track pointers to such objects, and to complain at the ends of execution paths when these are out of sync.

Status: implementation started by Eric for GSoC 2023 in the plugin in the test suite

Compare with https://gcc-python-plugin.readthedocs.io/en/latest/cpychecker.html#reference-count-checking

Error-handling checking

Many functions in the CPython API can fail; some return NULL and set the thread's exception state. Others return a numeric code. Ideally the analyzer ought to know about the behavior of these functions and bifurcate the analysis path, so that if we have an unconditional deref of a result, the analyzer can complain about the missing error checking.

Status: implementation started by Eric for GSoC 2023 in the plugin in the test suite

Compare with https://gcc-python-plugin.readthedocs.io/en/latest/cpychecker.html#reference-count-checking

See the list of the 20 most commonly used CPython API entrypoints from David Malcolm's 2013 survey

Rather than exhaustively handling all of the API as known_function subclasses, it may be easier to add new attributes for describing properties of the API entrypoints. For example, consider PyDict_SetItem. We could perhaps describe this via: 

   1 extern int PyDict_SetItem(PyObject *p, PyObject *key, PyObject *val)
   2   __attribute__((cpython_param_must_be_hashable (key))
   3   __attribute__((return_val_on_success (0))
   4   __attribute__((return_val_on_failure (-1))
   5   __attribute__((cpython_param_incremented_on_success (key))
   6   __attribute__((cpython_param_untouched_on_success (val))
   7 ;

(though this implies supporting parameter names in attributes).

Errors in exception-handling

The analyzer could keep track of the per-thread exception state, and issue a warning about any paths through functions returning a PyObject* that return NULL for which the per-thread exception state has not been set:

Status: not implemented

Compare with https://gcc-python-plugin.readthedocs.io/en/latest/cpychecker.html#errors-in-exception-handling

Format string checking

Various CPython APIs take format strings. We should detect mismatches between the number and types of arguments that are passed in, as compared with those described by the format string.

We could analyzer the following API entrypoints:

For example, type mismatches between int vs long can lead to flaws when the code is compiled on big-endian 64-bit architectures, where sizeof(int) != sizeof(long) and the in-memory layout of those types differs from what you might expect.

We should also issue a warning if the list of keyword arguments in a call to PyArg_ParseTupleAndKeywords is not NULL-terminated.

Status: not implemented

Compare with https://gcc-python-plugin.readthedocs.io/en/latest/cpychecker.html#format-string-checking

Verification of PyMethodDef tables

We should verify the types within tables of PyMethodDef initializers: the callbacks are typically cast to PyCFunction, but the exact type needs to correspond to the flags given. For example (METH_VARARGS | METH_KEYWORDS) implies a different function signature to the default, which the vanilla C compiler has no way of verifying.

We should also warn about tables of PyMethodDef initializers that are lacking a NULL sentinel value to terminate the iteration.

Status: not implemented

Compare with https://gcc-python-plugin.readthedocs.io/en/latest/cpychecker.html#verification-of-pymethoddef-tables

Checking arguments of "call" calls

We should verify the argument lists of invocations of PyObject_CallFunctionObjArgs and PyObject_CallMethodObjArgs, checking that all of the arguments are of the correct type (PyObject* or subclasses), and that the list is NULL-terminated.

Status: not implemented

Compare with https://gcc-python-plugin.readthedocs.io/en/latest/cpychecker.html#additional-tests

Checking tp_traverse callbacks

Status: not implemented (and wasn't implemented in cpychecker).

Errors in tp_traverse can mess up CPython's garbage collector. Examples of errors might be missing the callback altogether, or omitting fields.

Errors in GIL-handling

Other ideas for warnings?

TODO

Ideas for projects to test

None: StaticAnalyzer/CPython (last edited 2024-05-14 13:47:05 by DavidMalcolm)