Skip to content

Temperature/Angle/Power conversion outputs change if there are more than three zeros after the input value, even though the value are mathematically equivalent #416

@alexyorke

Description

@alexyorke

Describe the bug
When appending zeros to a number when converting a temperature, the output suddenly becomes incorrect, even though the value is the same.

Steps To Reproduce

  1. Open Calculator
  2. Switch to temperature view
  3. Switch the first temperature type to Celculus
  4. Switch the second temperature type to Kelvin
  5. Type in 273.00000000000
  6. The output will change from 0.15 to 0.149999999998 when adding additional zeros beyond three.

Note: Also happens with angle and power conversions.

Expected behavior
The temperature should remain at 0.15, irregardless of how many zeros are in the input.

Device and Application Information (please complete the following information):

  • OS Build: 10.0.17134.0
  • Architecture: X64
  • Application Version: 10.1812.10048.0

Screenshots
image

image

Additional information

I had a look at IEEE 754 (https://en.wikipedia.org/wiki/Floating-point_arithmetic#IEEE_754:_floating_point_in_modern_computers), and it seems to cover situations where adding very small numbers together causes rounding errors, and not ones which end with just zeros.

A team member has edited this comment to update repro steps for clarity.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Area: User ExperienceIssue primarily affects end user experience and are not covered by other areasPri: 3fixed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions