-
-
Notifications
You must be signed in to change notification settings - Fork 8.6k
py: Fixes and test coverage for 64-bit big integer representations. #16953
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
py: Fixes and test coverage for 64-bit big integer representations. #16953
Conversation
ec883a8 to
c6f3856
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #16953 +/- ##
=======================================
Coverage 98.44% 98.44%
=======================================
Files 171 171
Lines 22208 22208
=======================================
Hits 21863 21863
Misses 345 345 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Code size report: |
0dd8885 to
5d8e0c1
Compare
This comment was marked as outdated.
This comment was marked as outdated.
f8acf08 to
c862fb1
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
Ah of course, thanks! |
c862fb1 to
aa0917b
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
8555e92 to
3b737ad
Compare
Thank you for asking. Feel free to do as you prefer. Let's say that if I don't see the new PR coming once this one gets integrated into the master, I will submit it :-) |
9ed365d to
ba9e3fc
Compare
Drat, when I tested the code size improvement from unsigned |
The issue is because I've pushed a commit to try and improve the situation there, by adjusting which local has it's address taken. That improves things for most ports, but others (unix, esp8266) increase a little with my change. Still, I think that's a reasonable improvement. That change to |
995e3dd to
ed0974a
Compare
|
Alright, a bit of finagling and we've got a very minor code size reduction on all the bare metal ports! 🎉 I am not sure which part is causing the minor increase in unix port sizes, but it's small enough to possibly be noise. I've rebased and simplified the commit history (removed the parsing fixes and improvements in |
ed0974a to
aa1477c
Compare
|
@projectgus I'm happy to squash my optimisation commit down into your commit (the one just prior to mine). I don't need the attribution for that. |
aa1477c to
c8f23ee
Compare
|
@dpgeorge OK, have squashed! (And rebased agian also.) |
These will run on all ports which support them, but importantly they'll also run on ports that don't support arbitrary precision but do support 64-bit long ints. Includes some test workarounds to account for things which will overflow once "long long" big integers overflow (added in follow-up commit): - uctypes_array_load_store test was failing already, now won't parse. - all the ffi_int tests contain 64-bit unsigned values, that won't parse as long long. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <[email protected]>
These tests cover the use of mp_obj_new_int_from_str_len when mp_parse_num_integer overflows the SMALLINT limit, and also the case where the value may not be null terminated. Placed in a separate test file so that extmod/json test doesn't rely on bigint support. Signed-off-by: Yoctopuce dev <[email protected]> Signed-off-by: Angus Gratton <[email protected]>
Signed-off-by: Angus Gratton <[email protected]>
Relies on arbitrary precision math, so won't run on a port which has threads & limited bigint support. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <[email protected]>
The other performance tests run and pass with only 64-bit big integer support. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <[email protected]>
Long long big integer support now raises an exception on overflow rather than returning an undefined result. Also adds an error when shifting by a negative value. The new arithmetic checks are added in the misc.h header. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <[email protected]>
Makes it compatible with the __builtin_mul_overflow() syntax, used in follow-up commit. Includes optimisation in runtime.c to minimise the code size impact from additional param. Signed-off-by: Damien George <[email protected]> Signed-off-by: Angus Gratton <[email protected]>
If big integer support is 'long long' then mp_parse_num_integer() can
parse to it directly instead of failing over from small int. This means
strtoll() is no longer pulled in, and fixes some bugs parsing long long
integers (i.e. can now parse negative values correctly, can now parse
values which aren't NULL terminated).
The (default) smallint parsing compiled code should stay the same here,
macros and a typedef are used to abstract some parts of it out.
When bigint is long long we parse to 'unsigned long long' first (to avoid
the code size hit of pulling in signed 64-bit math routines) and the
convert to signed at the end.
One tricky case this routine correctly overflows on is
int("9223372036854775808") which is one more than LLONG_MAX in decimal. No
unit test case added for this as it's too hard to detect 64-bit long
integer mode.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <[email protected]>
dpgeorge
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking good!
636b9bb to
17fbc5a
Compare
|
Thanks @projectgus and @yoctopuce for your efforts here. Lots of small details have been addressed which is great! |
|
Thanks @projectgus, you deserve all the credit. This is a really nice improvement for bare metal ports with limited ressources. |
There is currently no build using REPR_C in the unix CI tests. As discussed in PR micropython#16953, this is something that combines well with the longlong build. Signed-off-by: Yoctopuce dev <[email protected]>
There is currently no build using REPR_C in the unix CI tests. As discussed in PR micropython#16953, this is something that combines well with the longlong build. Signed-off-by: Yoctopuce dev <[email protected]>
There is currently no build using REPR_C in the unix CI tests. As discussed in PR micropython#16953, this is something that combines well with the longlong build. Signed-off-by: Yoctopuce dev <[email protected]>
Summary
As pointed out by @yoctopuce in this discussion and #16932, there is poor test coverage for the optional 64-bit bigint representation and this representation has some bugs.
This PR aims to improve this:
int_64equivalent.longlongunix build variant that enables 64-bit long int mode. Mostly useful for CI testing.mp_parse_num()to parse directly tolong longin the 64-bit big integer configuration, saving code size.Thanks to @yoctopuce for suggesting and demonstrating a bunch of these ideas, and improvements on the original version of this PR.
Testing
--via-mpy. All passing.Follow-up work (for new PRs)
longlongconfig to also testMICROPY_OBJ_REPR_C.Trade-offs and Alternatives