Skip to content

fix(core): keep NaN semantics of negated float and double comparisons - #3048

Merged
skylot merged 1 commit into
skylot:masterfrom
b84330808:fix-float-compare-nan
Nov 2, 2026
Merged

skylot merged 1 commit into
skylot:masterfrom
b84330808:fix-float-compare-nan

Conversation

@b84330808

Copy link
Copy Markdown
Contributor

❗ Please review the guidelines for contributing

Description

Negated float and double comparisons are decompiled into the opposite comparison, which gives a different result for NaN:

public int notLess(double a, double b) {
    if (!(a < b)) {
        return 1;
    }
    return 2;
}

This is decompiled as if (a >= b) { return 1; }, so notLess(Double.NaN, 1.0) returns 2 instead of 1. The same happens with return !(a > b); (decompiled as return a <= b;) and with ternaries.

cmpg-*/cmpl-* return 1 or -1 when an argument is NaN, so cmp(a, b) op 0 equals a op b only for some ops. Also, once the condition is folded into a op b, the condition inversions that run later (IfOp.invert(), invertIfRegion) are no longer exact.

With this change, a float/double cmp stays wrapped in the condition until code generation, so every inversion still works on the integer cmp result. ConditionGen then writes a op b if that gives the same result for NaN, and !(a inverted_op b) otherwise. Long comparisons are folded as before. Comparisons that were already correct, like the ones in TestCmpOp/TestCmpOp2, decompile to the same code as before.

TestFloatCompareNaN is new. On master, it fails in both the d8-j11 and java-11 profiles (expected: 1 but was: 2 from the auto check). With the fix, ./gradlew test, spotlessCheck and checkstyle pass.

I used Claude Code to write this change.

`cmpg`/`cmpl` return 1/-1 for NaN arguments, so `cmp(a, b) op 0` was
folded into `a op b` even where that gives a different result for NaN,
and the later condition inversions then flipped `a < b` into `a >= b`.
For example `if (!(a < b))` was decompiled as `if (a >= b)`.

Keep float and double `cmp` in the condition until code generation, so
inversions stay exact, and generate `a op b` or `!(a inverted_op b)`
depending on which one keeps the result for NaN.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

@skylot skylot left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.
Although I prefer to apply changes only in visitors and keep codegen as simple as possible, but for this case I am not sure how to better implement that, so it is fine for now 🤣

@skylot
skylot merged commit c5a451e into skylot:master Nov 2, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants