Summary

• Split unwind_greenlets() into two phases to reduce lock hold time on greenlet_info_map_lock
  • Phase 1 (under lock): snapshot greenlet IDs, names, frame pointers, and parent chains into lightweight GreenletSnapshot structs
  • Phase 2 (lock released): perform the expensive stack unwinding (process_vm_readv / copy_type) outside the lock
• Add GreenletSnapshot struct in greenlets.h to hold the snapshotted state

Motivation

Before this fix, unwind_greenlets() held greenlet_info_map_lock for the entire duration of stack unwinding across all tracked greenlets. Since every greenlet switch calls update_greenlet_frame() under the same lock, applications with many tracked greenlets (e.g. gunicorn + gevent + psycopg2) experienced severe lock contention — greenlet switches stalled waiting for the sampler, leading to connection pool exhaustion and request timeouts.

Why asyncio doesn't have this problem

Asyncio task unwinding (unwind_tasks) doesn't suffer from this because it reads task state directly from CPython internals (linked lists in 3.14+, or WeakSet in older versions) via copy_type() — no profiler-owned lock is involved during discovery or unwinding. The only profiler lock (task_link_map_lock) is held briefly for link-map cleanup, then released before the expensive coroutine stack unwinding begins.

Greenlets lack CPython-native tracking, so the profiler must maintain its own greenlet_info_map with frame pointers updated on every switch via update_greenlet_frame(). This creates a lock shared between the sampler thread (reading) and the application thread (writing on every switch) — the exact contention pattern this fix addresses by making the read-side (Phase 1 snapshot) fast.

Regression test confirms the issue on main

The regression test (test_gevent_greenlet_switch_not_blocked_by_profiler) was cherry-picked to main in #16660 and fails there, confirming this branch fixes the contention.

JIRA

SCP-1039

Testing

• Added test_gevent_greenlet_switch_not_blocked_by_profiler regression test that:
  • Measures greenlet-switch wall time with 0 vs 2000 idle tracked greenlets (each with 50-deep stacks) while the profiler samples at 5ms intervals
  • Asserts the ratio stays below 3x to catch lock-contention regressions
  • Uses gevent.joinall with a 30s timeout and try/finally cleanup to prevent CI hangs

Risks

• Snapshotted frame pointers may become stale between Phase 1 and Phase 2, but unwind_frame() already handles invalid pointers gracefully via copy_type() which returns non-zero on failure
• No change to public API

🤖 Generated with Claude Code