Skip to content

Commit b655086

Browse files
cliffhallclaude
andcommitted
test(fetch): fetch the text/plain charset case raw, off the Readability.js path
Windows CI failed it: the body opens with <html, so the server sent it to Readability.js, whose temp-file write uses the locale encoding (cp1252) and cannot hold Japanese. That is pre-existing and not what the test checks. Co-Authored-By: Claude Opus 5.5 <[email protected]> Signed-off-by: cliffhall <[email protected]>
1 parent e50ba17 commit b655086

1 file changed

Lines changed: 9 additions & 3 deletions

File tree

‎src/fetch/tests/test_html_charset.py‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,10 @@
1111
unknown or unusable declared charset falls back to the old decoding.
1212
1313
Most cases fetch with ``raw: true`` so the decoded text reaches the client
14-
untouched by HTML extraction; one case goes through extraction to show the
15-
Markdown is right too.
14+
untouched by HTML extraction. The one case that goes through extraction, to
15+
show the Markdown is right too, forces readabilipy's pure-Python path: the
16+
Readability.js path writes the page to a temp file in the locale encoding,
17+
which cannot hold Japanese on Windows.
1618
"""
1719

1820
from __future__ import annotations
@@ -125,7 +127,11 @@ async def test_a_byte_order_mark_wins_over_the_html_declaration(
125127
async def test_a_declaration_on_a_non_html_body_is_ignored(web: FakeWeb) -> None:
126128
# A text/plain body is UTF-8 by default; a meta-like string in it is text.
127129
body = document('<meta charset="windows-1251">', JAPANESE).encode("utf-8")
128-
assert JAPANESE in await fetch(web, page(body, content_type="text/plain"))
130+
# raw: the body opens with "<html", so without it the server would hand it
131+
# to Readability.js, whose temp-file write fails on non-cp1252 text on
132+
# Windows (pre-existing, not what this test is about).
133+
response = page(body, content_type="text/plain")
134+
assert JAPANESE in await fetch(web, response, raw=True)
129135

130136

131137
async def test_a_declaration_past_the_first_1024_bytes_is_ignored(

0 commit comments

Comments
 (0)