tests: pin how c1 numeric character references extract - #367

Closes #76.

what this is

#76 reported that numeric character references in the reserved C1 range came out as the raw code point rather than what a browser shows, so … gave \x85 instead of …. That is fixed in libxml2 now, and I have posted the check on the issue.

Nothing in the suite holds it there, though, so this adds the coverage rather than leaving an eight year old behaviour resting on a dependency nobody is pinning for it.

what is covered

The whole range, not a sample. All 32 code points from 0x80 to 0x9f, through both the css and the xpath path, compared against html.unescape, which implements the same HTML5 replacement table:

for code_point in range(0x80, 0xA0):
    reference = f"&#{code_point};"
    expected = html.unescape(reference)
    sel = self.sscls(text=f"<p>{reference}</p>")
    assert sel.css("p::text").get() == expected, reference
    assert sel.xpath("//p/text()").get() == expected, reference

Plus the hex and named forms, since … and … reach the same place by a different route.

the xml one is deliberate

def test_xml_keeps_c1_character_references_literal(self) -> None:
    sel = self.sscls(text="<p>&#133;</p>", type="xml")
    assert sel.xpath("//p/text()").get() == "\x85"

XML has no CP1252 remapping, so … really is U+0085 there and the literal code point is correct. That is the behaviour the original report showed, using etree.fromstring, which is the XML parser. It is pinned so nobody later "fixes" the XML path to match HTML.

does it actually catch anything

A regression to the old behaviour differs from the assertion on 27 of the 32 code points. The other five, 0x81, 0x8d, 0x8f, 0x90 and 0x9d, have no entry in the replacement table, so the literal code point is the right answer for them either way.

checks

I stated those two rows loosely the first time, so to be exact. I compared against origin/master in a throwaway worktree using the same binaries rather than assuming. The finding sets are identical; the only difference is line numbers, PLR0917 moving from 1242 to 1270 and the two mypy lines from 11/23 to 12/24, which are exactly my added line counts. Nothing new is attributable to this change.

Both are also local environment artifacts rather than CI signal. tox.ini pins types-lxml, types-jmespath and types-psutil for the typing env and none are in my venv, so all 7 mypy errors disappear under CI. Local ruff is 0.16.5 while .pre-commit-config.yaml pins v0.15.22, so version skew explains findings CI would not raise.