Skip to content

Commit 82a0cdf

Browse files
authored
Merge branch 'main' into warsaw/829-followup
2 parents 5fb40ee + 3efd2f4 commit 82a0cdf

28 files changed

Lines changed: 1621 additions & 186 deletions

.github/workflows/build.yml

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -278,13 +278,13 @@ jobs:
278278
# unsupported as it most resembles other 1.1.1-work-a-like ssl APIs
279279
# supported by important vendors such as AWS-LC.
280280
- { name: openssl, version: 1.1.1w }
281-
- { name: openssl, version: 3.0.19 }
282-
- { name: openssl, version: 3.3.6 }
283-
- { name: openssl, version: 3.4.4 }
284-
- { name: openssl, version: 3.5.5 }
285-
- { name: openssl, version: 3.6.1 }
281+
- { name: openssl, version: 3.0.20 }
282+
- { name: openssl, version: 3.3.7 }
283+
- { name: openssl, version: 3.4.5 }
284+
- { name: openssl, version: 3.5.6 }
285+
- { name: openssl, version: 3.6.2 }
286286
## AWS-LC
287-
- { name: aws-lc, version: 1.68.0 }
287+
- { name: aws-lc, version: 1.72.1 }
288288
env:
289289
SSLLIB_VER: ${{ matrix.ssllib.version }}
290290
MULTISSL_DIR: ${{ github.workspace }}/multissl
@@ -398,7 +398,7 @@ jobs:
398398
needs: build-context
399399
if: needs.build-context.outputs.run-ubuntu == 'true'
400400
env:
401-
OPENSSL_VER: 3.5.5
401+
OPENSSL_VER: 3.5.6
402402
PYTHONSTRICTEXTENSIONBUILD: 1
403403
steps:
404404
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
@@ -506,7 +506,7 @@ jobs:
506506
matrix:
507507
os: [ubuntu-24.04]
508508
env:
509-
OPENSSL_VER: 3.5.5
509+
OPENSSL_VER: 3.5.6
510510
PYTHONSTRICTEXTENSIONBUILD: 1
511511
ASAN_OPTIONS: detect_leaks=0:allocator_may_return_null=1:handle_segv=0
512512
steps:

.github/workflows/reusable-ubuntu.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ jobs:
3535
runs-on: ${{ inputs.os }}
3636
timeout-minutes: 60
3737
env:
38-
OPENSSL_VER: 3.5.5
38+
OPENSSL_VER: 3.5.6
3939
PYTHONSTRICTEXTENSIONBUILD: 1
4040
TERM: linux
4141
steps:

Doc/library/profiling.sampling.rst

Lines changed: 120 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -153,6 +153,10 @@ Attach to a running process by PID::
153153

154154
python -m profiling.sampling attach 12345
155155

156+
Print a single snapshot of a running process's stack::
157+
158+
python -m profiling.sampling dump 12345
159+
156160
Use live mode for real-time monitoring (press ``q`` to quit)::
157161

158162
python -m profiling.sampling run --live script.py
@@ -173,8 +177,9 @@ Enable opcode-level profiling to see which bytecode instructions are executing::
173177
Commands
174178
========
175179

176-
Tachyon operates through two subcommands that determine how to obtain the
177-
target process.
180+
Tachyon operates through several subcommands. ``run`` and ``attach`` collect
181+
samples over time; ``dump`` captures a single snapshot; ``replay`` converts
182+
binary profiles to other formats.
178183

179184

180185
The ``run`` command
@@ -217,6 +222,78 @@ On most systems, attaching to another process requires appropriate permissions.
217222
See :ref:`profiling-permissions` for platform-specific requirements.
218223

219224

225+
.. _dump-command:
226+
227+
The ``dump`` command
228+
--------------------
229+
230+
The ``dump`` command prints a single snapshot of a running process's Python
231+
stack and exits, similar to a traceback::
232+
233+
python -m profiling.sampling dump 12345
234+
235+
Unlike ``attach``, ``dump`` does not run a sampling loop: it reads the
236+
stack once. This is useful for investigating hung or unresponsive
237+
processes, or for answering "what is this process doing right now?".
238+
239+
The output mirrors a traceback (most recent call last) and annotates each
240+
thread with its current state (main thread, has GIL, on CPU, waiting for
241+
GIL, has exception, or idle):
242+
243+
.. code-block:: text
244+
245+
Stack dump for PID 12345, thread 140735 (main thread, has GIL, on CPU; most recent call last):
246+
File "server.py", line 28, in serve
247+
await handle_request(req)
248+
File "handler.py", line 91, in handle_request
249+
result = expensive_call(req)
250+
251+
When the target's source files are readable, ``dump`` prints the source
252+
line for each frame and highlights the executing expression.
253+
254+
Like ``attach``, ``dump`` requires permission to read the target process's
255+
memory. See :ref:`profiling-permissions`.
256+
257+
The ``dump`` command supports the following options:
258+
259+
``-a``, ``--all-threads``
260+
Dump every thread in the target process. Without this flag only the main
261+
thread is shown.
262+
263+
``--native``
264+
Include synthetic ``<native>`` frames marking transitions into C
265+
extensions or other non-Python code.
266+
267+
``--no-gc``
268+
Hide the synthetic ``<GC>`` frames that mark active garbage collection.
269+
270+
``--opcodes``
271+
Annotate each frame with the bytecode opcode the thread is currently
272+
executing (for example, ``opcode=CALL_KW``). Useful for
273+
instruction-level investigation, including identifying specializations
274+
chosen by the adaptive interpreter.
275+
276+
``--async-aware``
277+
Reconstruct stacks across ``await`` boundaries. ``dump`` walks the task
278+
graph and emits one section per task, with ``<task>`` markers separating
279+
coroutines awaiting each other.
280+
281+
``--async-mode {running,all}``
282+
Controls which tasks are included when ``--async-aware`` is enabled.
283+
``running`` shows only the task currently executing on each thread;
284+
``all`` (the default for ``dump``) also includes tasks suspended on a
285+
wait. ``attach``'s default for this flag is ``running``; ``dump``
286+
defaults to ``all`` because a single snapshot is most useful when it
287+
shows the full task graph.
288+
289+
``--blocking``
290+
Pause every thread in the target while reading its stack and resume
291+
them after. Guarantees a fully consistent snapshot at the cost of
292+
briefly stopping the target. Without it, ``dump`` reads memory while
293+
the target keeps running, which is faster but can occasionally produce
294+
a torn stack.
295+
296+
220297
.. _replay-command:
221298

222299
The ``replay`` command
@@ -1441,11 +1518,52 @@ Global options
14411518

14421519
Attach to and profile a running process by PID.
14431520

1521+
.. option:: dump
1522+
1523+
Print a single one-shot snapshot of a running process's Python stack.
1524+
14441525
.. option:: replay
14451526

14461527
Convert a binary profile file to another output format.
14471528

14481529

1530+
Dump options
1531+
------------
1532+
1533+
The following options apply to the ``dump`` subcommand:
1534+
1535+
.. option:: -a, --all-threads
1536+
1537+
Dump all threads in the target process instead of just the main thread.
1538+
1539+
.. option:: --native
1540+
1541+
Include ``<native>`` frames for non-Python code.
1542+
1543+
.. option:: --no-gc
1544+
1545+
Exclude ``<GC>`` frames for active garbage collection.
1546+
1547+
.. option:: --opcodes
1548+
1549+
Show bytecode opcode names when available.
1550+
1551+
.. option:: --async-aware
1552+
1553+
Reconstruct the stack across ``await`` boundaries for asyncio
1554+
applications.
1555+
1556+
.. option:: --async-mode <mode>
1557+
1558+
Async stack mode: ``running`` (only the running task) or ``all``
1559+
(all tasks including waiting). Defaults to ``all`` for ``dump``.
1560+
Requires :option:`--async-aware`.
1561+
1562+
.. option:: --blocking
1563+
1564+
Pause all threads in the target process while reading the stack.
1565+
1566+
14491567
Sampling options
14501568
----------------
14511569

Doc/library/typing.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2156,7 +2156,7 @@ without the dedicated syntax, as documented below.
21562156
Added support for the ``bound``, ``covariant``, ``contravariant``, and
21572157
``infer_variance`` parameters.
21582158

2159-
.. class:: ParamSpec(name, *, bound=None, covariant=False, contravariant=False, default=typing.NoDefault)
2159+
.. class:: ParamSpec(name, *, bound=None, covariant=False, contravariant=False, infer_variance=False, default=typing.NoDefault)
21602160

21612161
Parameter specification variable. A specialized version of
21622162
:ref:`type variables <typevar>`.

Doc/whatsnew/3.15.rst

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -322,6 +322,9 @@ Key features include:
322322
* Profile running processes by PID (``attach``) - attach to already-running applications
323323
* Run and profile scripts directly (``run``) - profile from the very start of execution
324324
* Execute and profile modules (``run -m``) - profile packages run as ``python -m module``
325+
* Capture a one-shot snapshot of a running process (``dump``) - print a
326+
traceback-style stack of every thread (or all asyncio tasks with
327+
``--async-aware``). Useful for investigating hung processes.
325328

326329
* **Multiple profiling modes**: Choose what to measure based on your performance investigation:
327330

Lib/_colorize.py

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -359,6 +359,23 @@ class LiveProfiler(ThemeSection):
359359
)
360360

361361

362+
@dataclass(frozen=True, kw_only=True)
363+
class ProfilerDump(ThemeSection):
364+
header: str = ANSIColors.BOLD_BLUE
365+
interpreter: str = ANSIColors.GREY
366+
thread: str = ANSIColors.BOLD_CYAN
367+
status: str = ANSIColors.YELLOW
368+
frame_index: str = ANSIColors.GREY
369+
frame: str = ANSIColors.BOLD_GREEN
370+
filename: str = ANSIColors.CYAN
371+
line_no: str = ANSIColors.YELLOW
372+
source: str = ANSIColors.WHITE
373+
source_highlight: str = ANSIColors.BOLD_YELLOW
374+
opcode: str = ANSIColors.GREY
375+
warning: str = ANSIColors.YELLOW
376+
reset: str = ANSIColors.RESET
377+
378+
362379
@dataclass(frozen=True, kw_only=True)
363380
class Pickletools(ThemeSection):
364381
annotation: str = ANSIColors.GREY
@@ -447,6 +464,7 @@ class Theme:
447464
http_server: HttpServer = field(default_factory=HttpServer)
448465
live_profiler: LiveProfiler = field(default_factory=LiveProfiler)
449466
pickletools: Pickletools = field(default_factory=Pickletools)
467+
profiler_dump: ProfilerDump = field(default_factory=ProfilerDump)
450468
syntax: Syntax = field(default_factory=Syntax)
451469
timeit: Timeit = field(default_factory=Timeit)
452470
tokenize: Tokenize = field(default_factory=Tokenize)
@@ -463,6 +481,7 @@ def copy_with(
463481
http_server: HttpServer | None = None,
464482
live_profiler: LiveProfiler | None = None,
465483
pickletools: Pickletools | None = None,
484+
profiler_dump: ProfilerDump | None = None,
466485
syntax: Syntax | None = None,
467486
timeit: Timeit | None = None,
468487
tokenize: Tokenize | None = None,
@@ -482,6 +501,7 @@ def copy_with(
482501
http_server=http_server or self.http_server,
483502
live_profiler=live_profiler or self.live_profiler,
484503
pickletools=pickletools or self.pickletools,
504+
profiler_dump=profiler_dump or self.profiler_dump,
485505
syntax=syntax or self.syntax,
486506
timeit=timeit or self.timeit,
487507
tokenize=tokenize or self.tokenize,
@@ -505,6 +525,7 @@ def no_colors(cls) -> Self:
505525
http_server=HttpServer.no_colors(),
506526
live_profiler=LiveProfiler.no_colors(),
507527
pickletools=Pickletools.no_colors(),
528+
profiler_dump=ProfilerDump.no_colors(),
508529
syntax=Syntax.no_colors(),
509530
timeit=Timeit.no_colors(),
510531
tokenize=Tokenize.no_colors(),

Lib/dataclasses.py

Lines changed: 30 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -757,22 +757,16 @@ def _is_kw_only(a_type, dataclasses):
757757
return a_type is dataclasses.KW_ONLY
758758

759759

760-
def _is_type(annotation, cls, a_module, a_type, is_type_predicate):
761-
# Given a type annotation string, does it refer to a_type in
762-
# a_module? For example, when checking that annotation denotes a
763-
# ClassVar, then a_module is typing, and a_type is
764-
# typing.ClassVar.
760+
def _get_type_from_annotation(annotation, cls):
761+
# Loosely parse a string annotation and return its type.
765762

766-
# It's possible to look up a_module given a_type, but it involves
767-
# looking in sys.modules (again!), and seems like a waste since
768-
# the caller already knows a_module.
763+
# We can't perform a full type hint evaluation at the point where @dataclass
764+
# was invoked because class's module is not fully initialized yet. So we resort
765+
# to parsing string annotation using regexp, and extracting a type before
766+
# the first square bracket.
769767

770768
# - annotation is a string type annotation
771769
# - cls is the class that this annotation was found in
772-
# - a_module is the module we want to match
773-
# - a_type is the type in that module we want to match
774-
# - is_type_predicate is a function called with (obj, a_module)
775-
# that determines if obj is of the desired type.
776770

777771
# Since this test does not do a local namespace lookup (and
778772
# instead only a module (global) lookup), there are some things it
@@ -803,24 +797,21 @@ def _is_type(annotation, cls, a_module, a_type, is_type_predicate):
803797
# https://github.com/python/cpython/issues/77634 for details.
804798
global _MODULE_IDENTIFIER_RE
805799
if _MODULE_IDENTIFIER_RE is None:
806-
_MODULE_IDENTIFIER_RE = re.compile(r'(?:\s*(\w+)\s*\.)?\s*(\w+)')
800+
_MODULE_IDENTIFIER_RE = re.compile(r'^\s*(\w+(?:\s*\.\s*\w+)*)')
807801

808802
match = _MODULE_IDENTIFIER_RE.prefixmatch(annotation)
809-
if match:
810-
ns = None
811-
module_name = match[1]
812-
if not module_name:
813-
# No module name, assume the class's module did
814-
# "from dataclasses import InitVar".
815-
ns = sys.modules.get(cls.__module__).__dict__
816-
else:
817-
# Look up module_name in the class's module.
818-
module = sys.modules.get(cls.__module__)
819-
if module and module.__dict__.get(module_name) is a_module:
820-
ns = sys.modules.get(a_type.__module__).__dict__
821-
if ns and is_type_predicate(ns.get(match[2]), a_module):
822-
return True
823-
return False
803+
if not match:
804+
return None
805+
806+
# Note: _MODULE_IDENTIFIER_RE guarantees that path is non-empty
807+
path = match[1].split(".")
808+
root = sys.modules.get(cls.__module__)
809+
for path_item in path:
810+
root = getattr(root, path_item.strip(), None)
811+
if root is None:
812+
return None
813+
814+
return root
824815

825816

826817
def _get_field(cls, a_name, a_type, default_kw_only):
@@ -858,17 +849,18 @@ def _get_field(cls, a_name, a_type, default_kw_only):
858849
# is actually of the correct type.
859850

860851
# For the complete discussion, see https://bugs.python.org/issue33453
852+
if isinstance(a_type, str):
853+
a_type_annotation = _get_type_from_annotation(a_type, cls)
854+
else:
855+
a_type_annotation = a_type
861856

862857
# If typing has not been imported, then it's impossible for any
863858
# annotation to be a ClassVar. So, only look for ClassVar if
864859
# typing has been imported by any module (not necessarily cls's
865860
# module).
866861
typing = sys.modules.get('typing')
867862
if typing:
868-
if (_is_classvar(a_type, typing)
869-
or (isinstance(f.type, str)
870-
and _is_type(f.type, cls, typing, typing.ClassVar,
871-
_is_classvar))):
863+
if _is_classvar(a_type_annotation, typing):
872864
f._field_type = _FIELD_CLASSVAR
873865

874866
# If the type is InitVar, or if it's a matching string annotation,
@@ -877,10 +869,7 @@ def _get_field(cls, a_name, a_type, default_kw_only):
877869
# The module we're checking against is the module we're
878870
# currently in (dataclasses.py).
879871
dataclasses = sys.modules[__name__]
880-
if (_is_initvar(a_type, dataclasses)
881-
or (isinstance(f.type, str)
882-
and _is_type(f.type, cls, dataclasses, dataclasses.InitVar,
883-
_is_initvar))):
872+
if _is_initvar(a_type_annotation, dataclasses):
884873
f._field_type = _FIELD_INITVAR
885874

886875
# Validations for individual fields. This is delayed until now,
@@ -1073,10 +1062,11 @@ def _process_class(cls, init, repr, eq, order, unsafe_hash, frozen,
10731062
dataclasses = sys.modules[__name__]
10741063
for name, type in cls_annotations.items():
10751064
# See if this is a marker to change the value of kw_only.
1076-
if (_is_kw_only(type, dataclasses)
1077-
or (isinstance(type, str)
1078-
and _is_type(type, cls, dataclasses, dataclasses.KW_ONLY,
1079-
_is_kw_only))):
1065+
if isinstance(type, str):
1066+
a_type_annotation = _get_type_from_annotation(type, cls)
1067+
else:
1068+
a_type_annotation = type
1069+
if _is_kw_only(a_type_annotation, dataclasses):
10801070
# Switch the default to kw_only=True, and ignore this
10811071
# annotation: it's not a real field.
10821072
if KW_ONLY_seen:

0 commit comments

Comments
 (0)