Skip to content

Add separators encoding parameter#557

Merged
bwoodsend merged 1 commit intoultrajson:mainfrom
JustAnotherArchivist:separators
Jul 10, 2022
Merged

Add separators encoding parameter#557
bwoodsend merged 1 commit intoultrajson:mainfrom
JustAnotherArchivist:separators

Conversation

@JustAnotherArchivist
Copy link
Copy Markdown
Collaborator

Closes #283

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Jun 27, 2022

Codecov Report

❌ Patch coverage is 82.75862% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.53%. Comparing base (67ec071) to head (7f95e11).
⚠️ Report is 218 commits behind head on main.

Files with missing lines Patch % Lines
python/objToJSON.c 78.26% 10 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #557      +/-   ##
==========================================
- Coverage   91.84%   91.53%   -0.32%     
==========================================
  Files           6        6              
  Lines        1852     1902      +50     
==========================================
+ Hits         1701     1741      +40     
- Misses        151      161      +10     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread python/objToJSON.c
oseparatorsItem = PyTuple_GetItem(oseparators, 0);
if (PyErr_Occurred())
{
return NULL;
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Surely this line isn't reachable if the check above has already confirmed that there are two separators?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right that it's currently impossible to hit this given CPython's implementation of PyTuple_GetItem. But there's no guarantee that it couldn't fail for reasons unrelated to the index not matching the tuple's size...
The check should probably be if (!oseparatorsItem) (or if (oseparatorsItem == NULL), existing code is inconsistent) though, which should perform better as well, although that's unlikely to matter in the setup function.

Comment thread tests/test_ujson.py
((True, 0), TypeError),
((",", True), TypeError),
((True, ":"), TypeError),
((b",", b":"), TypeError),
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have a malformed UTF8 string in here too? Coverage is moaning about those lines + the ERROR goto never getting tested.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, we can't, for the same reason that #537 can't be tested for: we pass through surrogates now, so the conversion from PyUnicode to char* can never fail, practically speaking. Testing those encoding failures will require the C API instrumentation mentioned in that issue, which does not exist yet apart from a basic PoC.

@bwoodsend bwoodsend merged commit 8a946e5 into ultrajson:main Jul 10, 2022
@hugovk hugovk added the changelog: Added For new features label Jul 11, 2022
@Serpens66
Copy link
Copy Markdown

Serpens66 commented Sep 15, 2022

Thank you for finally implementing (and publishing) the separators.

Could you please add this very important information to your documentation?
To use "ujson" as a "drop in replacement, you need to do:
ujson.dumps(exampledict, separators=(', ',': '))
to get exactly the same result as json.dumps(exampledict)

This is very important, since json.dumps is also often used to generate signatures for API requests and there the missing spaces causes trouble and it took me ages to find out why the signature is invalid now (when I used ujson instead of json).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog: Added For new features

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Accept a separators parameter

5 participants