Scope: aiohttp/_http_parser.pyx, aiohttp/_http_writer.pyx,
aiohttp/_websocket/mask.pyx, aiohttp/_websocket/reader_py.py (compiled as
reader_c), supporting headers and build files.
| Agent | Group | Mode |
|---|---|---|
| gil-discipline-checker | A | run |
| resource-lifecycle-checker | A | run |
| type-slot-checker | B | run |
| module-state-checker | B | run |
| version-compat-scanner | C | run |
| parity-checker | D | run |
| stable-abi-checker | D | run |
| c-complexity-analyzer | E | run |
| git-history-analyzer | F | run (last) |
| refcount-auditor | — | skipped (Cython mode: 95–100% FP from generator patterns) |
| error-path-analyzer | — | skipped (Cython mode) |
| null-safety-scanner | — | skipped (Cython mode) |
| pyerr-clear-auditor | — | skipped (Cython mode) |
aiohttp's Cython layer has correctly declared freethreading_compatible=True
and removed the only previously identified global C buffer race
(_http_writer.pyx, fixed 2025-09-04). Three new correctness gaps remain:
the @cython.freelist allocator on both message types is non-atomic and
causes use-after-free under concurrent parsers; two parity gaps in
_http_parser.pyx are confirmed-incomplete fixes from recent commits
(a TCP upgrade bypass and a missing pipeline-queue bound on the response
parser). The complexity hotspots in the WebSocket reader and HTTP parser
carry the highest probability of harboring the next latent bug, consistent
with a fix-to-feature ratio of 27% observed in git history. No
_PyEval_StopTheWorld usage exists anywhere in the tree; all fixes are
straightforward substitutions or one-line additions.
| Field | Value |
|---|---|
| Modules | _http_parser, _http_writer, _websocket.mask, _websocket.reader_c |
| Code generation | Cython 3.2.5 |
| Python targets | ≥ 3.10 |
| Free-threading declared | Yes — freethreading_compatible=True in Makefile (all 3 Cython rules) |
| Limited API | No — py_limited_api absent from setup.py; PyByteArray_Resize is a blocker |
| Init style | Single-phase (m_size=-1 in generated C) |
| C library dependency | llhttp (vendored) via _cparser.pxd |
| Dimension | Status | FIX | CONSIDER | Top Finding |
|---|---|---|---|---|
| GIL Discipline | Y | 1 | 1 | @cython.freelist(250) non-atomic race on both message types |
| Resource Lifecycle | G | 0 | 1 | Py_buffer OOM tail-slice path (HPE_PAUSED) |
| Type Slots | G | 0 | 1 | HttpParser instantiable via __new__ without _init |
| Module State | G | 0 | 2 | 13× cdef object module globals break subinterpreter isolation |
| ABI Compliance | G | 0 | 1 | PyByteArray_Resize blocks stable ABI; abi3 infeasible |
| Version Compat | G | 0 | 3 | 3× DEF deprecated in Cython 4 |
| C/Python Parity | Y | 2 | 1 | "tcp" absent from ALLOWED_UPGRADES; max_msg_queue_size missing from HttpResponseParser |
| Complexity | G | 0 | 5 | _feed_data 183 LOC / CC 27 (WebSocket reader) |
| Refcount Safety | — | — | — | Skipped (Cython mode) |
| Error Handling | — | — | — | Skipped (Cython mode) |
| NULL Safety | — | — | — | Skipped (Cython mode) |
| PyErr_Clear Safety | — | — | — | Skipped (Cython mode) |
G = No FIX findings · Y = 1–3 FIX findings · R = 4+ FIX findings
| # | Finding | File:Line | Severity | Agents |
|---|---|---|---|---|
| 1 | @cython.freelist(250) on RawRequestMessage and RawResponseMessage — module-global C counter and object-pointer array written non-atomically on every alloc/dealloc; concurrent parsers (e.g. two feed_data calls on the same connection from different threads) cause use-after-free |
_http_parser.pyx:129,229 |
CRITICAL | gil-discipline-checker, git-history-analyzer |
| 2 | ALLOWED_UPGRADES = frozenset({"websocket"}) — "tcp" missing; Python parser's _is_supported_upgrade() accepts {"tcp", "websocket"} (fixed there in commit 8f23712616, March 2024); Cython parser silently rejects TCP upgrades 27 months later |
_http_parser.pyx:51,484 |
HIGH | parity-checker, git-history-analyzer |
| 3 | HttpResponseParser.__init__ does not accept max_msg_queue_size and does not forward it to self._init() — pipelined response queue is unbounded; HttpRequestParser received the parameter in commit 93a2b1c36 but HttpResponseParser was not updated (confirmed twin-class omission) |
_http_parser.pyx:752–762 |
HIGH | parity-checker, git-history-analyzer |
| # | Finding | File:Line | Severity |
|---|---|---|---|
| 4 | data[pos:] slice at line 650 can raise MemoryError after llhttp_execute returns HPE_PAUSED; if it does, control jumps out before PyBuffer_Release at line 652 — buffer stays exported and the backing object cannot be resized or freed |
_http_parser.pyx:648–652 |
MEDIUM |
| 5 | HttpParser and its Cython subclasses have __cinit__ (allocates C structs) but _init is a separate Python-level call; constructing via HttpParser.__new__(HttpResponseParser) without calling __init__ leaves _cparser allocated but unconfigured; _init is never a slot, so tp_init is a no-op on HttpParser itself |
_http_parser.pyx:343–418 |
MEDIUM |
| 6 | 12× cdef object module-level globals in _http_parser.pyx (lines 62–74: EMPTY_PAYLOAD, NO_EXTENSIONS, etc.) and _istr in _http_writer.pyx (line 12) are file-scope C statics (static PyObject*); the module was compiled with m_size=-1, so each subinterpreter shares the same static cells — subinterpreter isolation is broken |
_http_parser.pyx:62–74, _http_writer.pyx:12 |
MEDIUM |
| 7 | cdef list _http_method populated at module init by a for loop — a mutable list as a module-scope static; mutating it post-init would silently corrupt method-name lookup for all parsers |
_http_parser.pyx:105–109 |
LOW |
| 8 | DEF BUF_SIZE = 16 * 1024 in _http_writer.pyx — Cython DEF compile-time constants are deprecated since Cython 3.0 and removed in Cython 4; this DEF also feeds a C array dimension, which will need a const or enum replacement before migration |
_http_writer.pyx:10 |
LOW |
| 9 | DEF DEFAULT_FREELIST_SIZE = 250 — same deprecation class as Finding 8; feeds the @cython.freelist argument that is also the target of FIX Finding 1 |
_http_parser.pyx:52 |
LOW |
| 10 | DEF METHODS_COUNT = 46 — same deprecation class; feeds the range in _http_method construction and must stay in sync with the llhttp method enum |
_http_parser.pyx:103 |
LOW |
| 11 | _write_utf8 silently drops lone surrogates (return 0 at line 72 on \ud800–\udfff) while the pure-Python serializer raises UnicodeEncodeError; clients using the C writer get silent data truncation instead of an error |
_http_writer.pyx:70–72 |
MEDIUM |
| 12 | PyByteArray_Resize and PyByteArray_GET_SIZE (non-limited API) are called directly from the generated Cython C; py_limited_api is absent from setup.py, so no abi3 claim is made — but if adoption is ever attempted, these two calls are the primary blocker alongside a buffer-protocol Py_buffer floor conflict with Python 3.10 |
setup.py, _http_parser.pyx |
LOW |
| 13 | No cpXXXt-tagged (free-threaded) wheels are published to PyPI — cibuildwheel has no [tool.cibuildwheel.freethreading] section in pyproject.toml; users on CPython 3.13t/3.14t receive a GIL-compatible wheel that silently re-enables the GIL |
pyproject.toml (absent) |
LOW |
| 14 | WebSocketReader._feed_data: 183 LOC, cyclomatic complexity 27 — highest CC in the extension layer; recent commit 69344c6ef reorganized the max_msg_size check location, adding a new branch cluster that raised CC |
_websocket/reader_py.py:332–515 |
— |
| 15 | WebSocketReader._handle_frame: 136 LOC, CC 21 — interleaved opcode dispatch, masking, and continuation-frame logic; fragmented-mask bug was fixed here (commit e0cc020d5, April 2025) at the cost of additional branch depth |
_websocket/reader_py.py:194–330 |
— |
| 16 | HttpParser.feed_data: 78 LOC, CC 16 — llhttp dispatch + OOM guard + tail logic; commit b5a517076 added buffer-pointer caching here; the HPE_PAUSED slice-raise gap (Finding 4) is in this function |
_http_parser.pyx:602–680 |
— |
| 17 | HttpParser._on_headers_complete: 68 LOC, CC 14 — highest-churn function in git history alongside feed_data (18 substantive changes in 18 months); upgrade handling (FIX Finding 2) lives here |
_http_parser.pyx:466–534 |
— |
| 18 | HttpParser._on_status_complete: 47 LOC, nesting depth 5 — response-status dispatch with inlined payload-size accounting |
_http_parser.pyx:700–747 |
— |
-
Module state vs. migration cost (
_http_parser.pyx:62–74): module-state-checker flags the 12cdef objectglobals as subinterpreter-isolation failures; c-complexity-analyzer notes that migrating them to aPyModule_GetStatestruct requires threading the module pointer through every callback that currently reads them as file-statics. This is a multi-week refactor; the subinterpreter use case is still rare for aiohttp. Deferring is rational — but thePy_mod_multiple_interpretersslot should be set toPy_MOD_MULTIPLE_INTERPRETERS_NOT_SUPPORTEDso CPython rejects the mismatch explicitly. -
Stable ABI infeasibility (
stable-abi-checker):PyByteArray_Resizehas no limited-API equivalent; adopting abi3 would require a runtime-check workaround or replacement withbytesaccumulation. The complexity cost is not justified given aiohttp's standard wheel publishing workflow. Recommendation is to explicitly reject abi3 rather than leave the door open.
| # | Finding |
|---|---|
| 19 | Change cdef list _http_method to cdef tuple — matches immutability intent; http_method_str() only reads it; no post-init mutation needed |
| 20 | Change ALLOWED_CLOSE_CODES: Final[set[int]] to frozenset — Final does not prevent runtime mutation; frozenset is lock-free under free-threading and matches ALLOWED_UPGRADES style |
| 21 | Latent nogil trap: llhttp_execute C callbacks (cb_on_url, cb_on_headers_complete, etc.) call back into Python from C — if a nogil block were ever introduced around llhttp_execute, these callbacks would execute without the GIL. Document this constraint and add a # NOGIL-UNSAFE comment at the llhttp_execute call site |
| 22 | Add Py_mod_multiple_interpreters slot to module init (set to Py_MOD_MULTIPLE_INTERPRETERS_NOT_SUPPORTED) — signals the incompatibility explicitly rather than silently misbehaving in subinterpreter contexts |
| 23 | requirements/cython.in has no upper bound (Cython with no version pin) — Cython 4 will remove DEF (Findings 8–10) and may change freelist behaviour; add Cython>=3.2,<4 until the migration is done |
| 24 | abi3 and free-threading are mutually exclusive in the current CPython ABI: do not add py_limited_api=True to setup.py — it would conflict with freethreading_compatible=True and produce incorrect wheel tags |
| 25 | cython_limited_api=True is absent from the Makefile; if abi3 is ever revisited, this flag and py_limited_api in setup.py must be set simultaneously; a half-applied change would produce a wheel that lies about its ABI compliance |
ft-analysis.md Finding 5 — PyBuffer_Release not called on exception paths in
feed_data — DISMISSED by resource-lifecycle-checker during this analysis.
The llhttp callbacks (cb_on_url, cb_on_headers_complete, etc.) catch all
Python exceptions via try: ... except BaseException as ex: pyparser._last_error = ex; return -1. Because llhttp_execute has no Cython except annotation, control
unconditionally falls through to PyBuffer_Release at line 652 regardless of the
callback's return value. The stored exception is re-raised at line 668, which is
after the release. The ft-analysis was incorrect in classifying this as UNSAFE/HIGH.
Note: Finding 4 in this report identifies a residual, narrower gap on the HPE_PAUSED tail-slice path — a distinct code location that the original ft finding did not specify.
ALLOWED_UPGRADESandSINGLETON_HEADERS—cdef frozenset; immutable after module init, lock-free under free-threading;frozenset.__contains__is safe for concurrent reads._serialize_headersbuffer — switched from globalcdef char BUFFER[BUF_SIZE]to stack-local allocation with aheap_allocatedfallback flag (commit 945837cc2, 2025-09-04); the headline Cython FT race is gone.feed_databuffer-pointer caching —base = <char*>self.py_buf.bufis captured beforePyBuffer_Release; commit b5a517076 correctly fixed the use-after-release on the normal path.max_line_sizecumulative enforcement — bothcb_on_urlandcb_on_statususe accumulated-length checks; commit 36df6c138 was applied completely to both callbacks in one change; no twin-class gap exists here.mask.pyx— 48 lines, only 2 commits in its entire history; the XOR kernel is minimal and carries no thread-safety concerns.- No
nogil/Py_BEGIN_ALLOW_THREADSblocks — zero instances of GIL-released regions in any.pyxfile; the extension never invokes CPython APIs without the GIL. - No
_PyEval_StopTheWorld— confirmed by full-repo grep; none of the races require global quiescence to fix.
| File | Total commits | Recent (2025–2026) | Fix-to-feature ratio | Risk level |
|---|---|---|---|---|
_http_parser.pyx |
67 | 18 | ~27% | HIGH |
_websocket/reader_py.py |
18 | 12 | ~33% | HIGH |
_http_writer.pyx |
12 | 5 | ~17% | MEDIUM |
_websocket/mask.pyx |
2 | 0 | 0% | LOW |
Notable commit history:
- 2018, commit 28f15195c — Introduced
@cython.freelist(DEFAULT_FREELIST_SIZE)on bothRawRequestMessageandRawResponseMessagesimultaneously; they have never been touched since; neither was audited whenfreethreading_compatible=Truewas added in 2025. - 2024-03, commit 8f23712616 — Added
"tcp"to_is_supported_upgrade()in the Python parser, scoped explicitly to "the pure python http parser"; the CythonALLOWED_UPGRADESfrozenset was not updated and still lacks"tcp"27 months later. - 2025-09-04, commit 945837cc2 — Fixed global C buffer in
_http_writer.pyxand addedfreethreading_compatible=Trueto all Cython invocations;_http_parser.pyxwas only touched to remove a comment line — the freelist race was not audited. - 2026-06 (recent), commit 93a2b1c36 — Added
max_msg_queue_sizetoHttpRequestParser.__init__andHttpParser._init;HttpResponseParser.__init__was not updated (confirmed twin-class omission, trivially reproducible). - 2026-06, commit 36df6c138 — Enforced cumulative
max_line_sizeon bothcb_on_urlandcb_on_status; fix is complete — no gap found.
The fix-to-feature ratio of ~27% across all parser files is elevated relative to a healthy extension (typically ~10–15%). It reflects active correctness pressure on the HTTP parser and suggests a systematic audit sweep is more effective than continued point fixes.
-
Fix Finding 1 — disable
@cython.freelistfor free-threading buildsBoth
RawRequestMessage(line 129) andRawResponseMessage(line 229) must be addressed together — they were introduced in the same commit and carry identical risk.Option A — guard with a runtime check:
import sys as _sys _FREELIST_SIZE = 0 if not getattr(_sys, "_is_gil_enabled", lambda: True)() else DEFAULT_FREELIST_SIZE @cython.freelist(_FREELIST_SIZE) cdef class RawRequestMessage: ...
Option B — remove both decorators entirely and file a tracking issue to restore them only for GIL-enabled builds once Cython's freelist-disable-on-FT path is confirmed in the CI build environment.
-
Fix Finding 2 — add
"tcp"toALLOWED_UPGRADES# _http_parser.pyx line 51: cdef frozenset ALLOWED_UPGRADES = frozenset({"websocket", "tcp"})
One-character change; mirrors the Python parser fix from commit 8f23712616.
-
Fix Finding 3 — add
max_msg_queue_sizetoHttpResponseParser.__init__Mirror
HttpRequestParser.__init__exactly:# _http_parser.pyx HttpResponseParser.__init__ (line 752): def __init__( self, protocol, loop, int limit, timer=None, size_t max_line_size=8190, size_t max_headers=128, size_t max_field_size=8190, payload_exception=None, bint response_with_body=True, bint read_until_eof=False, bint auto_decompress=True, Py_ssize_t max_msg_queue_size=0 # ADD THIS ): self._init(cparser.HTTP_RESPONSE, protocol, loop, limit, timer, max_line_size, max_headers, max_field_size, payload_exception, response_with_body, read_until_eof, auto_decompress, max_msg_queue_size) # FORWARD IT
-
Fix Finding 4 — wrap HPE_PAUSED tail-slice in
try/finally# _http_parser.pyx feed_data (around line 634): PyObject_GetBuffer(data, &self.py_buf, PyBUF_SIMPLE) try: base = <char*>self.py_buf.buf data_len = <size_t>self.py_buf.len errno = cparser.llhttp_execute(self._cparser, base, data_len) # ... nb / pos / tail logic including data[pos:] slice ... finally: PyBuffer_Release(&self.py_buf) # raise ex follows the finally block unchanged
-
Fix Finding 5 — document or guard
HttpParser.__new__-without-__init__The simplest guard is to check
self._cparser is not NULLat the top of every public method that calls into llhttp, raisingRuntimeError("parser not initialized")if the check fails. Alternatively, initialize_cparserto NULL in__cinit__and call a safe-default_initfrom__cinit__with sentinel values that blockfeed_datafrom doing anything. -
Fix Finding 11 — align surrogate behaviour with the Python writer
In
_write_utf8, replace the silentreturn 0on surrogates with a Cythonraise:# _http_writer.pyx line 72: raise UnicodeEncodeError("utf-8", ...)
Or pass
errors="surrogatepass"in the serialiser and document the intentional deviation. Currently the silence is undocumented and breaks parity. -
Fix Findings 8–10 — migrate
DEFconstants before Cython 4 upgrade# Replace: DEF BUF_SIZE = 16 * 1024 DEF DEFAULT_FREELIST_SIZE = 250 DEF METHODS_COUNT = 46 # With: BUF_SIZE = 16 * 1024 # module-level Python int DEFAULT_FREELIST_SIZE = 250 METHODS_COUNT = 46
For
BUF_SIZEfeeding a C array dimension, usecdef enumor acdef intconstant depending on how the C array is declared in the generated code. -
Fix Finding 13 — publish free-threaded wheels to PyPI
Add to
pyproject.toml:[tool.cibuildwheel.freethreading] enable = true
Also add the
Programming Language :: Python :: Free Threading :: 3 - AlphaPyPI classifier once FIX findings 1–3 above are resolved.
-
Policy 19 — harden
_http_methodtotuplecdef tuple _http_method = tuple( cparser.llhttp_method_name(<cparser.llhttp_method_t>i).decode("ascii") for i in range(METHODS_COUNT) )
-
Policy 20 — convert
ALLOWED_CLOSE_CODEStofrozensetALLOWED_CLOSE_CODES: Final[frozenset[int]] = frozenset(int(i) for i in WSCloseCode)
-
Policy 22 — add
Py_mod_multiple_interpretersslotIn the generated module init (or via a Cython
cython: ... module_statedirective when available), setm_slotsto include:{Py_mod_multiple_interpreters, Py_MOD_MULTIPLE_INTERPRETERS_NOT_SUPPORTED} -
Policy 23 — cap Cython version in requirements
# requirements/cython.in: Cython>=3.2,<4 -
Finding 6 — migrate module-level globals to per-module state (long-term)
All 12
cdef objectglobals in_http_parser.pyx(lines 62–74) and_istrin_http_writer.pyx(line 12) should migrate to aPyModule_GetState-backed struct. This requiresm_size > 0and threading the module pointer through every init callback. Scope this as a standalone PR; it does not block the FIX items above.
- Free-threaded interpreter on this system:
python3t.exe(python3t -m pipfor package management) - The corresponding free-threading analysis (asyncio layer, cookie jar, connector pool,
WebSocket writer lock gap) is in
ft-analysis.md - For a phased migration plan:
/ft-review-toolkit:planor/cext-review-toolkit:plan - The ft-analysis.md Finding 5 (
PyBuffer_Releaseon exception) is DISMISSED per the resource-lifecycle-checker findings in this report — do not file it as an issue