Repository navigation
Support large integer stringification thresholds in both encoders - #387
Merged
etrepum merged 5 commits intoSep 15, 2026
Merged
Conversation
etrepum
reviewed
Sep 11, 2026
etrepum
left a comment
etrepum
left a comment
Member
There was a problem hiding this comment.
Codex found an edge case in this one:
Reviewed e709f3b. Found one edge-case correctness issue.
[P2] Normalize integer subclasses once before formatting and checking their magnitude.
At _speedups.c:716–721, PyNumber_Long(obj) invokes int again after encoder_long_to_str() has already converted the object. A stateful conversion can therefore make the quoting decision use a different integer from the one being serialized:
class ChangingInt(int):
calls = 0
def __int__(self):
self.calls += 1
return (1 << 64) if self.calls == 1 else 1
simplejson.dumps(ChangingInt(1), int_as_string_bitcount=64)Confirmed results:
PR C encoder: 18446744073709551616 — unquoted, two conversions.
Base and PR Python encoders: "18446744073709551616" — quoted, one conversion.
Retain the normalized integer and use it for both decimal formatting and the magnitude check. Add a regression test covering repeated conversion.
No other actionable findings. On Python 3.12.14, the full C/Python runner passed 462 tests, 70 skipped; another 2,320 targeted checks passed for boundaries, huge thresholds, nesting, keys, and ordinary subclasses. GitHub CI is awaiting workflow approval, so the remaining platform matrix is unverified.
etrepum
reviewed
Sep 13, 2026
etrepum
left a comment
etrepum
left a comment
Member
There was a problem hiding this comment.
py27 tests fail
======================================================================
FAIL: test_large_bitcount_normalizes_subclass_once (simplejson.tests.test_bitsize_int_as_string.TestBitSizeIntAsString)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/tmp.03GZKyJMIB/venv/lib/python2.7/site-packages/simplejson/tests/test_bitsize_int_as_string.py", line 153, in test_large_bitcount_normalizes_subclass_once
value, int_as_string_bitcount=64, indent=indent))
AssertionError: '"18446744073709551616"' != '1'
----------------------------------------------------------------------
Ran 462 tests in 1.751s
FAILED (failures=1, skipped=77)
context canceled
Normalize legacy int and long subclasses through __int__ before widening for bit-length comparisons. Cover stateful conversions, positive and negative large values, native integers, and conversion failures. Declare the scanstring tests UTF-8 so the full Python 2 suite can import. Validation: full C and pure-Python suites pass on CPython 2.7.18 (464 tests), 3.9.6 (466), 3.12.13 (466), and 3.14.4 debug (466). An isolated installed CPython 2.7 wheel also passes 464 tests. Extension wiring verified in every environment; 3.14 debug builds with strict C89 warnings and -Werror.
Python 2 int() may preserve a returned long subclass. Read native int payloads directly and use the built-in long conversion slot to copy long payloads without invoking subclass hooks. Regressions cover changed values and raising hooks for int/long originals and small, positive-large, and negative-large results. Validation: actual CPython 2.7 source and isolated installed-wheel dual-encoder suites each pass 468 tests (77 skips). Builds reject implicit C declarations and integer/pointer conversions. Python 3.9 and 3.12 full suites pass 470 tests; Python 3.14 debug strict C89/-Werror build and 28 focused dual-encoder tests pass.
etrepum
reviewed
Sep 14, 2026
| } | ||
| if (!ge) | ||
| return encoded; | ||
| goto quote; |
Member
There was a problem hiding this comment.
I think an else would be easier to follow than a goto/label pair, normally goto is only used for error handling
| new feature, don't feel obligated to stay limited-API-compatible — | ||
| performance-sensitive code wins. | ||
|
|
||
| ## Integer stringification thresholds |
Member
There was a problem hiding this comment.
I don't think we need to add text on this topic to AGENTS.md
| @@ -1,3 +1,9 @@ | |||
| Unreleased | |||
Member
There was a problem hiding this comment.
The changelog is updated manually before each release, you don't need to update this file
etrepum
approved these changes
Sep 14, 2026
etrepum
pushed a commit
that referenced
this pull request
Oct 2, 2026
Add CHANGES entries for #387, #388 and #390. Test Python 3.15 and 3.15t via allow-prereleases so the jobs follow 3.15.0 final, add PyPy 3.12, move debug and TSan builds to CPython 3.14.8, and bump checkout, setup-python, cache, setup-uv and cibuildwheel. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01FpcGq7FDACHqLiW7juM4SK
netbsd-srcmastr
pushed a commit
to NetBSD/pkgsrc
that referenced
this pull request
Oct 3, 2026
4.2.0 released 2026-10-02 * Start building wheels for Python 3.15 and drop Python 3.8 support via upgrade to cibuildwheel v4.2.0 simplejson/simplejson#385 * Build and test riscv64 Linux wheels on native RISE runners simplejson/simplejson#389 * The C scanner now reports the same error message as the pure Python decoder for an invalid first key in an object simplejson/simplejson#388 * ``int_as_string_bitcount`` now supports thresholds of 64 bits and above (including 128-bit integers) in both the C and pure Python encoders; the C encoder previously rejected them simplejson/simplejson#387 * Circular references introduced through ``for_json()`` or ``_asdict()`` now raise ``ValueError("Circular reference detected")`` instead of recursing without bound simplejson/simplejson#390 * CI now tests Python 3.15 (including free-threaded 3.15t), PyPy 3.12, CPython 3.14.8 debug builds, and uses current GitHub Actions and cibuildwheel v4.2.1 simplejson/simplejson#386
netbsd-srcmastr
pushed a commit
to NetBSD/pkgsrc
that referenced
this pull request
Oct 13, 2026
4.2.0 released 2026-10-02 * Start building wheels for Python 3.15 and drop Python 3.8 support via upgrade to cibuildwheel v4.2.0 simplejson/simplejson#385 * Build and test riscv64 Linux wheels on native RISE runners simplejson/simplejson#389 * The C scanner now reports the same error message as the pure Python decoder for an invalid first key in an object simplejson/simplejson#388 * ``int_as_string_bitcount`` now supports thresholds of 64 bits and above (including 128-bit integers) in both the C and pure Python encoders; the C encoder previously rejected them simplejson/simplejson#387 * Circular references introduced through ``for_json()`` or ``_asdict()`` now raise ``ValueError("Circular reference detected")`` instead of recursing without bound simplejson/simplejson#390 * CI now tests Python 3.15 (including free-threaded 3.15t), PyPy 3.12, CPython 3.14.8 debug builds, and uses current GitHub Actions and cibuildwheel v4.2.1 simplejson/simplejson#386
netbsd-srcmastr
pushed a commit
to NetBSD/pkgsrc
that referenced
this pull request
Oct 21, 2026
4.2.0 released 2026-10-02 * Start building wheels for Python 3.15 and drop Python 3.8 support via upgrade to cibuildwheel v4.2.0 simplejson/simplejson#385 * Build and test riscv64 Linux wheels on native RISE runners simplejson/simplejson#389 * The C scanner now reports the same error message as the pure Python decoder for an invalid first key in an object simplejson/simplejson#388 * ``int_as_string_bitcount`` now supports thresholds of 64 bits and above (including 128-bit integers) in both the C and pure Python encoders; the C encoder previously rejected them simplejson/simplejson#387 * Circular references introduced through ``for_json()`` or ``_asdict()`` now raise ``ValueError("Circular reference detected")`` instead of recursing without bound simplejson/simplejson#390 * CI now tests Python 3.15 (including free-threaded 3.15t), PyPy 3.12, CPython 3.14.8 debug builds, and uses current GitHub Actions and cibuildwheel v4.2.1 simplejson/simplejson#386
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Allow
int_as_string_bitcountthresholds of 64 bits and above in the C encoder, including 128-bit integers. Keep the small-threshold comparison path, and use magnitude bit lengths for larger thresholds so even very large counts do not allocate a power-of-two boundary. The Python encoder uses the same magnitude rule.Closes #306. Tests cover signed boundaries through 1,024 bits, nested values, indented output, and counts larger than native integers. Documentation and the encoder ownership notes are updated.
Validation: the original C encoder rejects the new boundary cases. The full C/Python runner passes on Python 3.12, 3.14 release and 3.14 debug (462 tests each), with explicit C-extension wiring checks. Strict C89 and warnings-as-errors builds pass on 3.12, 3.14 release and 3.14 debug. Legacy Python 2.7, free-threaded builds and the remaining platform matrix are left to CI.