Skip to content

Commit b82e17e

Browse files
zoobazooba
authored
bpo-36842: Implement PEP 578 (GH-12613)
Adds sys.audit, sys.addaudithook, io.open_code, and associated C APIs.
1 parent e788057 commit b82e17e

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

70 files changed

+3564
-1815
lines changed

‎Doc/c-api/code.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@ bound into a function.
4040
:c:func:`PyCode_New` directly can bind you to a precise Python
4141
version since the definition of the bytecode changes often.
4242
43+
.. audit-event:: code.__new__ "code filename name argcount kwonlyargcount nlocals stacksize flags"
4344
4445
.. c:function:: PyCodeObject* PyCode_NewEmpty(const char *filename, const char *funcname, int firstlineno)
4546

‎Doc/c-api/file.rst‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,32 @@ the :mod:`io` APIs instead.
6060
raised if the end of the file is reached immediately.
6161
6262
63+
.. c:function:: int PyFile_SetOpenCodeHook(Py_OpenCodeHookFunction handler)
64+
65+
Overrides the normal behavior of :func:`io.open_code` to pass its parameter
66+
through the provided handler.
67+
68+
The handler is a function of type :c:type:`PyObject *(\*)(PyObject *path,
69+
void *userData)`, where *path* is guaranteed to be :c:type:`PyUnicodeObject`.
70+
71+
The *userData* pointer is passed into the hook function. Since hook
72+
functions may be called from different runtimes, this pointer should not
73+
refer directly to Python state.
74+
75+
As this hook is intentionally used during import, avoid importing new modules
76+
during its execution unless they are known to be frozen or available in
77+
``sys.modules``.
78+
79+
Once a hook has been set, it cannot be removed or replaced, and later calls to
80+
:c:func:`PyFile_SetOpenCodeHook` will fail. On failure, the function returns
81+
-1 and sets an exception if the interpreter has been initialized.
82+
83+
This function is safe to call before :c:func:`Py_Initialize`.
84+
85+
.. versionadded:: 3.8
86+
87+
88+
6389
.. c:function:: int PyFile_WriteObject(PyObject *obj, PyObject *p, int flags)
6490
6591
.. index:: single: Py_PRINT_RAW

‎Doc/c-api/sys.rst‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -289,6 +289,56 @@ accessible to C code. They all work with the current interpreter thread's
289289
.. versionadded:: 3.2
290290
291291
292+
.. c:function:: int PySys_Audit(const char *event, const char *format, ...)
293+
294+
.. index:: single: audit events
295+
296+
Raises an auditing event with any active hooks. Returns zero for success
297+
and non-zero with an exception set on failure.
298+
299+
If any hooks have been added, *format* and other arguments will be used
300+
to construct a tuple to pass. Apart from ``N``, the same format characters
301+
as used in :c:func:`Py_BuildValue` are available. If the built value is not
302+
a tuple, it will be added into a single-element tuple. (The ``N`` format
303+
option consumes a reference, but since there is no way to know whether
304+
arguments to this function will be consumed, using it may cause reference
305+
leaks.)
306+
307+
:func:`sys.audit` performs the same function from Python code.
308+
309+
.. versionadded:: 3.8
310+
311+
312+
.. c:function:: int PySys_AddAuditHook(Py_AuditHookFunction hook, void *userData)
313+
314+
.. index:: single: audit events
315+
316+
Adds to the collection of active auditing hooks. Returns zero for success
317+
and non-zero on failure. If the runtime has been initialized, also sets an
318+
error on failure. Hooks added through this API are called for all
319+
interpreters created by the runtime.
320+
321+
This function is safe to call before :c:func:`Py_Initialize`. When called
322+
after runtime initialization, existing audit hooks are notified and may
323+
silently abort the operation by raising an error subclassed from
324+
:class:`Exception` (other errors will not be silenced).
325+
326+
The hook function is of type :c:type:`int (*)(const char *event, PyObject
327+
*args, void *userData)`, where *args* is guaranteed to be a
328+
:c:type:`PyTupleObject`. The hook function is always called with the GIL
329+
held by the Python interpreter that raised the event.
330+
331+
The *userData* pointer is passed into the hook function. Since hook
332+
functions may be called from different runtimes, this pointer should not
333+
refer directly to Python state.
334+
335+
See :pep:`578` for a detailed decription of auditing. Functions in the
336+
runtime and standard library that raise events include the details in each
337+
function's documentation.
338+
339+
.. versionadded:: 3.8
340+
341+
292342
.. _processcontrol:
293343
294344
Process Control

‎Doc/howto/instrumentation.rst‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -332,6 +332,15 @@ Available static markers
332332
.. versionadded:: 3.7
333333

334334

335+
.. c:function:: audit(str event, void *tuple)
336+
337+
Fires when :func:`sys.audit` or :c:func:`PySys_Audit` is called.
338+
``arg0`` is the event name as C string, ``arg1`` is a :c:type:`PyObject`
339+
pointer to a tuple object.
340+
341+
.. versionadded:: 3.8
342+
343+
335344
SystemTap Tapsets
336345
-----------------
337346

‎Doc/library/array.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,7 @@ The module defines the following type:
8383
to add initial items to the array. Otherwise, the iterable initializer is
8484
passed to the :meth:`extend` method.
8585

86+
.. audit-event:: array.__new__ "typecode initializer"
8687

8788
.. data:: typecodes
8889

‎Doc/library/ctypes.rst‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1509,6 +1509,17 @@ object is available:
15091509
:c:type:`int`, which is of course not always the truth, so you have to assign
15101510
the correct :attr:`restype` attribute to use these functions.
15111511

1512+
.. audit-event:: ctypes.dlopen name
1513+
1514+
Loading a library through any of these objects raises an
1515+
:ref:`auditing event <auditing>` ``ctypes.dlopen`` with string argument
1516+
``name``, the name used to load the library.
1517+
1518+
.. audit-event:: ctypes.dlsym "library name"
1519+
1520+
Accessing a function on a loaded library raises an auditing event
1521+
``ctypes.dlsym`` with arguments ``library`` (the library object) and ``name``
1522+
(the symbol's name as a string or integer).
15121523

15131524
.. _ctypes-foreign-functions:
15141525

@@ -2032,6 +2043,12 @@ Data types
20322043
This method returns a ctypes type instance using the memory specified by
20332044
*address* which must be an integer.
20342045

2046+
.. audit-event:: ctypes.cdata address
2047+
2048+
This method, and others that indirectly call this method, raises an
2049+
:func:`auditing event <sys.audit>` ``ctypes.cdata`` with argument
2050+
``address``.
2051+
20352052
.. method:: from_param(obj)
20362053

20372054
This method adapts *obj* to a ctypes type. It is called with the actual

‎Doc/library/functions.rst‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -275,6 +275,12 @@ are always available. They are listed here in alphabetical order.
275275
If you want to parse Python code into its AST representation, see
276276
:func:`ast.parse`.
277277

278+
.. audit-event:: compile "source filename"
279+
280+
Raises an :func:`auditing event <sys.audit>` ``compile`` with arguments
281+
``source`` and ``filename``. This event may also be raised by implicit
282+
compilation.
283+
278284
.. note::
279285

280286
When compiling a string with multi-line code in ``'single'`` or
@@ -473,6 +479,11 @@ are always available. They are listed here in alphabetical order.
473479
See :func:`ast.literal_eval` for a function that can safely evaluate strings
474480
with expressions containing only literals.
475481

482+
.. audit-event:: exec code_object
483+
484+
Raises an :func:`auditing event <sys.audit>` ``exec`` with the code object as
485+
the argument. Code compilation events may also be raised.
486+
476487
.. index:: builtin: exec
477488

478489
.. function:: exec(object[, globals[, locals]])
@@ -502,6 +513,11 @@ are always available. They are listed here in alphabetical order.
502513
builtins are available to the executed code by inserting your own
503514
``__builtins__`` dictionary into *globals* before passing it to :func:`exec`.
504515

516+
.. audit-event:: exec code_object
517+
518+
Raises an :func:`auditing event <sys.audit>` ``exec`` with the code object as
519+
the argument. Code compilation events may also be raised.
520+
505521
.. note::
506522

507523
The built-in functions :func:`globals` and :func:`locals` return the current
@@ -747,6 +763,16 @@ are always available. They are listed here in alphabetical order.
747763
If the :mod:`readline` module was loaded, then :func:`input` will use it
748764
to provide elaborate line editing and history features.
749765

766+
.. audit-event:: builtins.input prompt
767+
768+
Raises an :func:`auditing event <sys.audit>` ``builtins.input`` with
769+
argument ``prompt`` before reading input
770+
771+
.. audit-event:: builtins.input/result result
772+
773+
Raises an auditing event ``builtins.input/result`` with the result after
774+
successfully reading input.
775+
750776

751777
.. class:: int([x])
752778
int(x, base=10)
@@ -1176,6 +1202,11 @@ are always available. They are listed here in alphabetical order.
11761202
(where :func:`open` is declared), :mod:`os`, :mod:`os.path`, :mod:`tempfile`,
11771203
and :mod:`shutil`.
11781204

1205+
.. audit-event:: open "file mode flags"
1206+
1207+
The ``mode`` and ``flags`` arguments may have been modified or inferred from
1208+
the original call.
1209+
11791210
.. versionchanged::
11801211
3.3
11811212

‎Doc/library/io.rst‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,27 @@ High-level Module Interface
120120

121121
This is an alias for the builtin :func:`open` function.
122122

123+
.. audit-event:: open "path mode flags"
124+
125+
This function raises an :func:`auditing event <sys.audit>` ``open`` with
126+
arguments ``path``, ``mode`` and ``flags``. The ``mode`` and ``flags``
127+
arguments may have been modified or inferred from the original call.
128+
129+
130+
.. function:: open_code(path)
131+
132+
Opens the provided file with mode ``'rb'``. This function should be used
133+
when the intent is to treat the contents as executable code.
134+
135+
``path`` should be an absolute path.
136+
137+
The behavior of this function may be overridden by an earlier call to the
138+
:c:func:`PyFile_SetOpenCodeHook`, however, it should always be considered
139+
interchangeable with ``open(path, 'rb')``. Overriding the behavior is
140+
intended for additional validation or preprocessing of the file.
141+
142+
.. versionadded:: 3.8
143+
123144

124145
.. exception:: BlockingIOError
125146

‎Doc/library/mmap.rst‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,7 @@ To map anonymous memory, -1 should be passed as the fileno along with the length
6767
will be relative to the offset from the beginning of the file. *offset*
6868
defaults to 0. *offset* must be a multiple of the :const:`ALLOCATIONGRANULARITY`.
6969

70+
.. audit-event:: mmap.__new__ "fileno length access offset"
7071

7172
.. class:: mmap(fileno, length, flags=MAP_SHARED, prot=PROT_WRITE|PROT_READ, access=ACCESS_DEFAULT[, offset])
7273
:noindex:
@@ -155,6 +156,7 @@ To map anonymous memory, -1 should be passed as the fileno along with the length
155156

156157
mm.close()
157158

159+
.. audit-event:: mmap.__new__ "fileno length access offset"
158160

159161
Memory-mapped file objects support the following methods:
160162

‎Doc/library/os.rst‎

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -651,7 +651,7 @@ process and user.
651651
File Object Creation
652652
--------------------
653653

654-
This function creates new :term:`file objects <file object>`. (See also
654+
These functions create new :term:`file objects <file object>`. (See also
655655
:func:`~os.open` for opening file descriptors.)
656656

657657

@@ -829,11 +829,14 @@ as internal buffering of data.
829829
most *length* bytes in size. As of Python 3.3, this is equivalent to
830830
``os.truncate(fd, length)``.
831831

832+
.. audit-event:: os.truncate "fd length"
833+
832834
.. availability:: Unix, Windows.
833835

834836
.. versionchanged:: 3.5
835837
Added support for Windows
836838

839+
837840
.. function:: get_blocking(fd)
838841

839842
Get the blocking mode of the file descriptor: ``False`` if the
@@ -845,6 +848,7 @@ as internal buffering of data.
845848

846849
.. versionadded:: 3.5
847850

851+
848852
.. function:: isatty(fd)
849853

850854
Return ``True`` if the file descriptor *fd* is open and connected to a
@@ -912,6 +916,8 @@ as internal buffering of data.
912916
This function can support :ref:`paths relative to directory descriptors
913917
<dir_fd>` with the *dir_fd* parameter.
914918

919+
.. audit-event:: open "path mode flags"
920+
915921
.. versionchanged:: 3.4
916922
The new file descriptor is now non-inheritable.
917923

@@ -2756,6 +2762,8 @@ features:
27562762

27572763
This function can support :ref:`specifying a file descriptor <path_fd>`.
27582764

2765+
.. audit-event:: os.truncate "path length"
2766+
27592767
.. availability:: Unix, Windows.
27602768

27612769
.. versionadded:: 3.3
@@ -3715,6 +3723,8 @@ written in Python, such as a mail server's external command delivery program.
37153723
to using this function. See the :ref:`subprocess-replacements` section in
37163724
the :mod:`subprocess` documentation for some helpful recipes.
37173725

3726+
.. audit-event:: os.system command
3727+
37183728
.. availability:: Unix, Windows.
37193729

37203730

0 commit comments

Comments
 (0)