Skip to content

Commit 3f67b34

Browse files
authored
Fix SystemError when convert()'ing an out-of-range int to float (#1162)
## Summary `msgspec.convert(obj, float)` converts a Python `int` to a C `double` via `PyLong_AsDouble` without checking for overflow. For an `int` too large to represent as a finite `float` (e.g. `10**400`), `PyLong_AsDouble` sets `OverflowError` internally but still returns `-1.0`, and that value was passed straight through to `ms_decode_float` and returned to the interpreter with the exception still set — which surfaces as: ``` SystemError: <built-in function convert> returned a result with an exception set ``` This bypasses the documented `ValidationError` contract entirely, so a caller wrapping the call in `except msgspec.ValidationError` doesn't catch it. `json.decode` already handles the equivalent case correctly, raising `ValidationError: Number out of range`. This PR applies the same "check `PyErr_Occurred()` after a lossy C conversion, then report via `ms_error_with_path`" pattern already used elsewhere in this file (e.g. `_constr_as_f64`), so `convert()` reports the overflow the same way — for both bare (`convert(big, float)`) and nested (`convert({"x": big}, dict[str, float])`) targets, under both `strict=True` and `strict=False`. Fixes #1122. ## Testing - Reverted the fix locally and reproduced the exact reported `SystemError` before reapplying it. - Added `test_float_from_int_out_of_range` to `TestFloat` in `tests/unit/test_convert.py`, covering bare and nested targets under both strict modes. - Ran the full `tests/unit/` suite: 6052 passed, 473 skipped, 0 failures.
1 parent 7c2f473 commit 3f67b34

2 files changed

Lines changed: 19 additions & 1 deletion

File tree

‎src/msgspec/_core.c‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20934,7 +20934,14 @@ convert_int(
2093420934
return ms_decode_int_enum_or_literal_pyint(obj, type, path);
2093520935
}
2093620936
else if (type->types & MS_TYPE_FLOAT) {
20937-
return ms_decode_float(PyLong_AsDouble(obj), type, path);
20937+
double val = PyLong_AsDouble(obj);
20938+
if (val == -1.0 && PyErr_Occurred()) {
20939+
/* `obj` is out of range for a C double (PyLong_AsDouble sets
20940+
* OverflowError but still returns -1.0); without this check
20941+
* that error leaks past this function as a SystemError. */
20942+
return ms_error_with_path("Number out of range%U", path);
20943+
}
20944+
return ms_decode_float(val, type, path);
2093820945
}
2093920946
else if (
2094020947
type->types & MS_TYPE_DECIMAL

‎tests/unit/test_convert.py‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -429,6 +429,17 @@ def test_float(self):
429429
with pytest.raises(ValidationError, match="Expected `float`, got `null`"):
430430
convert(None, float)
431431

432+
@pytest.mark.parametrize("strict", [True, False])
433+
def test_float_from_int_out_of_range(self, strict):
434+
# A python int too large to represent as a C double must raise a
435+
# ValidationError, matching json.decode, rather than leaking the
436+
# OverflowError PyLong_AsDouble sets as a SystemError.
437+
big = 10**400
438+
with pytest.raises(ValidationError, match="Number out of range"):
439+
convert(big, float, strict=strict)
440+
with pytest.raises(ValidationError, match="Number out of range"):
441+
convert({"x": big}, dict[str, float], strict=strict)
442+
432443
@pytest.mark.parametrize(
433444
"meta, good, bad",
434445
[

0 commit comments

Comments
 (0)