Skip to content

Fix encoding ref leak with non-English character#714

Merged
bwoodsend merged 3 commits intoultrajson:mainfrom
nhancdt2602:fix-encode/leak-ref-to-default
Apr 26, 2026
Merged

Fix encoding ref leak with non-English character#714
bwoodsend merged 3 commits intoultrajson:mainfrom
nhancdt2602:fix-encode/leak-ref-to-default

Conversation

@nhancdt2602
Copy link
Copy Markdown
Contributor

Fixed #631

Avoid overwriting the default()-returned object when creating temporary UTF-8 bytes during string encoding.

Signed-off-by: nhancdt2602 <[email protected]>
@hugovk hugovk added the changelog: Fixed For any bug fixes label Apr 24, 2026
@hugovk hugovk changed the title Fix Encoding Ref Leak with non-English character Fix encoding ref leak with non-English character Apr 24, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 24, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.43%. Comparing base (299c641) to head (0d27095).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #714      +/-   ##
==========================================
+ Coverage   91.35%   91.43%   +0.07%     
==========================================
  Files           7        7              
  Lines        1979     1997      +18     
==========================================
+ Hits         1808     1826      +18     
  Misses        171      171              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Collaborator

@bwoodsend bwoodsend left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice one tracking that down.

Can we get some tests for it? Something like your reproducer that covers unicode, non-unicode as well as the default() function returning a structure containing more than one string? If you pass --leak-max-loops=10000 to pytest, it applies a loop + forced gc + tracemalloc auditing to each test so you don't need to do any of that yourself – just run the offending code paths.

Comment thread src/ujson/python/objToJSON.c
@nhancdt2602
Copy link
Copy Markdown
Contributor Author

Nice one tracking that down.

Can we get some tests for it? Something like your reproducer that covers unicode, non-unicode as well as the default() function returning a structure containing more than one string? If you pass --leak-max-loops=10000 to pytest, it applies a loop + forced gc + tracemalloc auditing to each test so you don't need to do any of that yourself – just run the offending code paths.

Ofc, I'll be working on tests

@nhancdt2602
Copy link
Copy Markdown
Contributor Author

@bwoodsend I added 2 relevant test cases for this PR. Here is the test result

Test Result

Setup

Run command:

python -m pytest --leak-max-loops=5000 tests/test_ujson.py

Before Fix

======================================= short test summary info =======================================
FAILED tests/test_ujson.py::test_no_memory_leak_default_non_ascii - Failed: 242633B leaked (96.97561950439648 per iteration)
=================================== 1 failed, 300 passed in 15.89s ====================================

After Fix

======================================== 301 passed in 11.28s =========================================

@bwoodsend bwoodsend merged commit 9f90a8c into ultrajson:main Apr 26, 2026
28 checks passed
@bwoodsend
Copy link
Copy Markdown
Collaborator

Thanks for figuring that one out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog: Fixed For any bug fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Serious and weird memory leak when dumping dictionary containing python object and non-English character

3 participants