-
Notifications
You must be signed in to change notification settings - Fork 52
Closed
Labels
type: bugSomething that should function correctly isn't.Something that should function correctly isn't.
Description
I was running some generative tests on my code and I ran into an issue with high precision floats. For example:
> x = 0.7531531167929774
> runParser (show x) testTokenParser.float
(Right 0.7531531167929775)
Note the final 5 instead of 4. The problem seems to be in the code that parses the decimal part of a number. Indeed, the following equivalent code gives the same erroneous result:
> digits = [7,5,3,1,5,3,1,1,6,7,9,2,9,7,7,4]
> foldr (\d f -> (f + toNumber d) / 10.0) 0.0 digits
0.7531531167929775
I'm not sure about the best way to modify fraction so as to avoid rounding errors. Maybe using the native parseFloat?
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
type: bugSomething that should function correctly isn't.Something that should function correctly isn't.