Repository navigation
convert leaks SystemError on an out-of-range int → float, where json.decode cleanly raises ValidationError #1122
Description
Description
Summary
Converting a Python int that is too large to represent as a float via msgspec.convert leaks a SystemError ("<built-in function convert> returned a result with an exception set") instead of raising a msgspec.ValidationError. The equivalent json.decode handles the same out-of-range value correctly, raising ValidationError: Number out of range, so the two paths disagree on the same logical conversion.
Reproduction
import msgspec
big = 10**400 # far beyond the largest finite float (~1.8e308)
# convert: leaks SystemError (both strict=True and strict=False)
msgspec.convert(big, float)
# SystemError: <built-in function convert> returned a result with an exception set
# nested, same result:
msgspec.convert({"v": big}, dict[str, float])
# SystemError: ...
# json.decode: handled correctly
msgspec.json.decode(str(big).encode(), type=float)
# msgspec.ValidationError: Number out of rangeExpected behavior
convert should raise msgspec.ValidationError (e.g. "Number out of range"), matching json.decode. An out-of-range integer is a value error the caller can catch, not an internal interpreter fault.
Actual behavior
convert raises SystemError: <built-in function convert> returned a result with an exception set. This looks like a C-level conversion (int → C double) that overflows and sets a Python exception, but whose result is still returned to the interpreter, which then raises SystemError. A caller who wraps the call in except msgspec.ValidationError (the documented contract for a bad value) does not catch it.
- Affects both
strict=Trueandstrict=False. - Affects nested targets (
dict[str, float],tuple[float, ...], etc.), not just a barefloat. - A normal in-range
intconverts fine (convert(5, float) -> 5.0), so only the overflow path is affected.
Environment
- msgspec 0.21.1
- Python 3.11.10
- macOS 12.7.6 (x86_64)
Notes
The json.decode path already reports this as ValidationError: Number out of range, so the fix is presumably to make convert's int → float path apply the same range check / error rather than leaking the C exception as a SystemError.
Activity
Brohammad commentedon Aug 31, 2026
Hi has this issue been resolved? If not I would like to take it up.
provinzkraut commentedon Aug 31, 2026
@Brohammad we do not need any more PRs for this. There's already 2 closed and 2 open ones.
Brohammad commentedon Aug 31, 2026
oh ok closing mine
Siyet commentedon Sep 7, 2026
@provinzkraut two things, since you have been the one steering this issue.
On the choice between the two open PRs. You pointed contributors at #1128 back in July when you closed #1129, and I have gone the other way and merged #1162. The reasons, briefly:
isfinitebranch, and its code comment answers your July question incorrectly. You were right:PyLong_AsDoublecannot return+/-inf. It either takes the compact fast path, which is always finite, or calls_PyLong_Frexp, which normalises the significand into[0.5, 1.0)and increments the exponent when rounding would push it to1.0;exponent > DBL_MAX_EXPis then rejected withOverflowErrorand a return of exactly-1.0, so the largest value that can come back is(1 - 2**-53) * 2**1024, which isDBL_MAX. Instrumenting that branch on fix: raise ValidationError for out-of-range int→float in convert #1128 and running 22100 values around the boundary, it never fired once.## Unreleasedbecame## Version 0.22.0 (2026-08-11)it now lands the entry inside an already released section, silently.If you would rather I had fixed #1128 up in place and kept the queue order, say so and I will take that as the rule next time.
The same root cause is still live in the public API, and neither PR touches it.
PyLong_AsDoubleappears twice in_core.c. Theconvert_intsite is what got fixed. The other isensure_is_finite_numeric, reached straight fromMeta:Same for
ge,ltandle, positive or negative.multiple_ofis arguably worse, because it does not crash, it answers wrongly:The failed conversion leaves the
-1.0sentinel, which trips thepositive && x <= 0check, so a plainly positive value is reported as not positive.ensure_is_nonnegative_integerhas the same shape withPyLong_AsSsize_t, soMeta(min_length=10**400)saysmust be >= 0about a positive number.None of this is a regression. It reproduces on 0.21.1 and on 3.11, and
git log -Sputs it in the commit that first addedMetain 2022. It is a small fix in two functions and it needs its own change rather than being folded into the convert fix.Worth knowing for #1006, which rewrites
ensure_is_finite_numericforDecimalbounds but leaves the int branch unguarded: after it,Meta(ge=Decimal("1E+400"))is accepted while the numerically equalMeta(ge=10**400)still raisesSystemError.Fix SystemError when convert()'ing an out-of-range int to float (msgs…
Correct error messages for out-of-range Meta length bounds (msgspec#1172