Skip to content

math: fix big.int div, gcd and bitAnd edge-cases#11108

Merged
andrewrk merged 1 commit intoziglang:masterfrom
tiehuis:big.int-fixes
Mar 10, 2022
Merged

math: fix big.int div, gcd and bitAnd edge-cases#11108
andrewrk merged 1 commit intoziglang:masterfrom
tiehuis:big.int-fixes

Conversation

@tiehuis
Copy link
Member

@tiehuis tiehuis commented Mar 10, 2022

Fixes #10932.

Disclaimer: The divmod change was fairly quick and I've yet to go through the algorithm again fully with the change in mind. The gcd/bitAnd change is fairly trivial.

@tiehuis
Copy link
Member Author

tiehuis commented Mar 10, 2022

To elaborate on the issues:

  • the gcd issue occurred when the gcd result exceeds a single limb and is reduced entirely in the main loop and not the single-limb processing at the end
  • the bitAnd was an incorrect assert which was being triggered when the limb_count(a) > limb_count(b) where a > b
  • the divmod issue occurs when a intermediate step results in a working value x which is normalized to 0, it's been a while since I reviewed this algorithm so no further comment yet

@andrewrk andrewrk merged commit 8bab1b4 into ziglang:master Mar 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

std.math.big.int panics (divFloor, gcd, bitAnd)

2 participants