Use FastDoubleParser where appropriate by headius · Pull Request #9150 · jruby/jruby

headius · 2025-12-28T05:59:14Z

This hooks up the FastDoubleParser project to our internal float parsing logic. Cases immediately rejected by FDP or by Ruby rules are passed through a slow-path cleanup and second attempt.

FastDoubleParser is the Java implementation by @wrandelshofer of Daniel Lemire's fast float parsing algorithm. See the project page here: https://github.com/wrandelshofer/FastDoubleParser

See ruby/ruby#15655 for a similar effort to add the Eisel-Lemire algorithm variant to CRuby, described in a blog post by @mensfeld here: https://mensfeld.pl/2025/12/ruby-string-to-float-optimization/.

Benchmarks are faster across the board and beat CRuby in every case (without @mensfeld's improvements).

BEFORE (10.1.0.0):

Simple decimals (1.5, 3.14)         0.091s
Prices (9.99, 19.95)                0.105s
Small integers (5, 42)              0.076s
Math constants (Pi, E)              0.351s
High precision decimals             0.229s
Scientific (1e5, 2e10)              0.122s

AFTER:

Simple decimals (1.5, 3.14)         0.077s
Prices (9.99, 19.95)                0.086s
Small integers (5, 42)              0.066s
Math constants (Pi, E)              0.129s
High precision decimals             0.141s
Scientific (1e5, 2e10)              0.085s

CRuby 4.0:

Simple decimals (1.5, 3.14)         0.129s
Prices (9.99, 19.95)                0.129s
Small integers (5, 42)              0.121s
Math constants (Pi, E)              0.681s
High precision decimals             0.519s
Scientific (1e5, 2e10)              0.123s

headius · 2025-12-28T06:00:31Z

CRuby 4.0 numbers for comparison:

Simple decimals (1.5, 3.14)         0.111s
Prices (9.99, 19.95)                0.111s
Small integers (5, 42)              0.104s
Math constants (Pi, E)              0.615s
High precision decimals             0.472s
Scientific (1e5, 2e10)              0.109s
Simple decimals (1.5, 3.14)         0.109s
Prices (9.99, 19.95)                0.111s
Small integers (5, 42)              0.104s
Math constants (Pi, E)              0.607s
High precision decimals             0.473s
Scientific (1e5, 2e10)              0.111s
Simple decimals (1.5, 3.14)         0.109s
Prices (9.99, 19.95)                0.111s
Small integers (5, 42)              0.105s
Math constants (Pi, E)              0.624s
High precision decimals             0.464s
Scientific (1e5, 2e10)              0.110s
Simple decimals (1.5, 3.14)         0.111s
Prices (9.99, 19.95)                0.113s
Small integers (5, 42)              0.112s
Math constants (Pi, E)              0.681s
High precision decimals             0.523s
Scientific (1e5, 2e10)              0.122s
Simple decimals (1.5, 3.14)         0.123s
Prices (9.99, 19.95)                0.124s
Small integers (5, 42)              0.118s
Math constants (Pi, E)              0.676s
High precision decimals             0.534s
Scientific (1e5, 2e10)              0.123s

lemire · 2026-01-01T19:16:09Z

@headius So faster than CRuby ?

headius · 2026-01-02T05:52:59Z

@lemire Yes, faster than CRuby but without your algorithm from @mensfeld's PR. I will try to compare that soon. JVM having no native 128-bit primitive type limits what it can do somewhat.

wrandelshofer · 2026-01-02T12:10:26Z

Have you tried, if the performance improves even more, when JavaDoubleParser accesses the byte array contained in the BytesList?

Like this:

    public static double fastByteListToDouble(ByteList bytes) {
        return JavaDoubleParser.parseDouble(bytes.getUnsafeBytes(), bytes.getBegin(), bytes.getRealSize());
    }

wrandelshofer · 2026-01-02T12:25:23Z

I think, you can get some speed improvement and pass all tests in JRuby, if you revert the changes in class ConvertDouble, and only change the last line of method completeCalculation() from this:

        private double completeCalculation() {
            ...

            return Double.valueOf(new String(chars, 0, charsIndex));
        }

to this:

        private double completeCalculation() {
            ...

            return JavaDoubleParser.parseDouble(chars, 0, charsIndex);
        }

This hooks up the Java implementation of Daniel Lemire's fast float parsing algorithm to our internal float parsing logic, excluding cases that are not 7-bit ASCII or which contain underscore characters (not currently allowed by FDP, see wrandelshofer/FastDoubleParser#85 for an attempt to add that feature).

headius · 2026-01-07T22:13:00Z

JavaDoubleParser accesses the byte array

It doesn't make much difference based on my measurements (ByteList implements CharSequence by just accessing the byte array).

revert the changes in class ConvertDouble, and only change the last line of method completeCalculation()

That does indeed avoid the failures, but isn't as fast as my patch.

My patch:

Simple decimals (1.5, 3.14)         0.062s
Prices (9.99, 19.95)                0.063s
Small integers (5, 42)              0.059s
Math constants (Pi, E)              0.076s
High precision decimals             0.081s
Scientific (1e5, 2e10)              0.063s

Your patch:

Simple decimals (1.5, 3.14)         0.072s
Prices (9.99, 19.95)                0.080s
Small integers (5, 42)              0.064s
Math constants (Pi, E)              0.124s
High precision decimals             0.127s
Scientific (1e5, 2e10)              0.082s

It's quite a bit better than the original, though!

No patch:

Simple decimals (1.5, 3.14)         0.148s
Prices (9.99, 19.95)                0.163s
Small integers (5, 42)              0.136s
Math constants (Pi, E)              0.414s
High precision decimals             0.337s
Scientific (1e5, 2e10)              0.153s

It's possible my patch is faster because it's not handling all those other forms that we need for Ruby support.

@wrandelshofer I really want to figure out how to use your library in JRuby but Ruby has so many oddities in float parsing. For just String#to_f, with the patch you provided in wrandelshofer/FastDoubleParser#85, the following failures remain:

parsing things like "45.6 degrees" by terminating parsing at the first unexpected character:

- treats leading characters of self as a floating point number (FAILED - 1)
- treats any non-numeric character other than '.', 'e' and '_' as terminals (FAILED - 3)
- takes an optional sign (FAILED - 4)
- treats a second 'e' as terminal (FAILED - 5)
- treats a second '.' as terminal (FAILED - 6)
- treats a '.' after an 'e' as terminal (FAILED - 7)
- treats non-printable ASCII characters as terminals (FAILED - 8)

parsing "NaN", "Infinity", and "-Infinity"

- treats special float value strings as characters (FAILED - 2)

It's pretty close at this point.

headius · 2026-01-07T22:23:43Z

treats special float value strings as characters (FAILED - 2)

The Ruby behavior is to treat strings like "NaN" as non parseable and return 0.0:

[] jruby $ cx 4.0.0 ruby -e 'p "NaN".to_f'
0.0
[] jruby $ cx 4.0.0 ruby -e 'p "Infinity".to_f'
0.0

I can configure this behavior:

.withInfinity(Collections.EMPTY_SET)
.withNaN(Collections.EMPTY_SET)

One down!

* Allow underscore as a group separator (essentially ignoring it) * Treat "NaN", "Infinity", and "+Infinity" as unparsable (0.0)

headius · 2026-01-07T22:42:45Z

The remaining failures are all due to FDP rejecting any trailing non-float characters and returning 0.0 for such cases. @wrandelshofer I don't see any way to configure this and I was not clear where in the code it decide to bail out for bad input.

[] jruby $ cx 4.0.0 ruby -e 'p "45.6 degrees".to_f'               
45.6
[] jruby $ jruby -e 'p "45.6 degrees".to_f'                               
0.0

…stic "fast path" approach, followed by a "slow path" for general cases.

wrandelshofer · 2026-01-08T17:02:15Z

It doesn't make much difference based on my measurements (ByteList implements CharSequence by just accessing the byte array).

The purpose of this change, is to get reliable performance of the parser regardless of the JIT.
The JIT is not always able to inline methods. For example, when the JVM is low on memory for byte code or when the code - for whatever reason - is not considered 'hot' enough for the JIT.

wrandelshofer · 2026-01-08T17:06:16Z

The remaining failures are all due to FDP rejecting any trailing non-float characters and returning 0.0 for such cases. @wrandelshofer I don't see any way to configure this and I was not clear where in the code it decide to bail out for bad input.

Yes. There is currently no API for this in the fast double parser library.
However, you should be able to achieve quite a good result without such an API, by using a "fast path/slow path" approach.
Please take a look at this pull request: headius#6

…e to ConvertDouble.

headius · 2026-01-08T18:43:11Z

@wrandelshofer Thank you!

mensfeld · 2026-04-03T10:07:52Z

Glad I could be of use :)

headius · 2026-05-08T16:15:58Z

Interestingly, the object size improvements that shipped with JRuby 10.1 have an enormous affect on performance here.

Here's numbers for this PR before the object size fixes:

[] jruby $ jruby bench_float_parse.rb              
Simple decimals (1.5, 3.14)         1.081s
Prices (9.99, 19.95)                0.755s
Small integers (5, 42)              0.729s
Math constants (Pi, E)              0.775s
High precision decimals             0.774s
Scientific (1e5, 2e10)              0.755s

Here's JRuby 10.1.0.0 with object size improvements and NO float parsing improvement:

[] jruby $ cx jruby-10.1 jruby bench_float_parse.rb
Simple decimals (1.5, 3.14)         0.202s
Prices (9.99, 19.95)                0.247s
Small integers (5, 42)              0.109s
Math constants (Pi, E)              0.371s
High precision decimals             0.335s
Scientific (1e5, 2e10)              0.205s

And here's the PR rebased onto current 10.1 master, combining all improvements:

[] jruby $ jruby bench_float_parse.rb 
Simple decimals (1.5, 3.14)         0.100s
Prices (9.99, 19.95)                0.076s
Small integers (5, 42)              0.073s
Math constants (Pi, E)              0.102s
High precision decimals             0.094s
Scientific (1e5, 2e10)              0.083s
Simple decimals (1.5, 3.14)         0.063s
Prices (9.99, 19.95)                0.074s
Small integers (5, 42)              0.061s
Math constants (Pi, E)              0.094s
High precision decimals             0.094s
Scientific (1e5, 2e10)              0.073s

Every case comes in under 0.1s now. Wow.

Compared to CRuby 4.0:

[] jruby $ cx ruby-4.0 ruby bench_float_parse.rb
Simple decimals (1.5, 3.14)         0.114s
Prices (9.99, 19.95)                0.114s
Small integers (5, 42)              0.107s
Math constants (Pi, E)              0.634s
High precision decimals             0.510s
Scientific (1e5, 2e10)              0.114s
Simple decimals (1.5, 3.14)         0.113s
Prices (9.99, 19.95)                0.114s
Small integers (5, 42)              0.107s
Math constants (Pi, E)              0.626s
High precision decimals             0.490s
Scientific (1e5, 2e10)              0.158s

I'll be incorporating @wrandelshofer's suggestions from headius#6 and trying to stabilize this to land for 10.1.1.0.

This hooks up the Java implementation of Daniel Lemire's fast float parsing algorithm to our internal float parsing logic, excluding cases that are not 7-bit ASCII or which contain underscore characters (not currently allowed by FDP, see wrandelshofer/FastDoubleParser#85 for an attempt to add that feature).

* Allow underscore as a group separator (essentially ignoring it) * Treat "NaN", "Infinity", and "+Infinity" as unparsable (0.0)

Fast float parsing using a "fast path then slow path" approach

Conflicting changes led to compile issues, so reintegrate the combined patches.

lemire · 2026-05-08T18:52:24Z

@headius This looks very impressive.

headius · 2026-05-08T19:08:12Z

@wrandelshofer A few unique cases also needed to be rejected, which I have done in the most recent commit. If you have any suggestion to eliminate this check, I'd love to hear it!

headius · 2026-05-08T19:12:07Z

@lemire All credit goes to @wrandelshofer!

Several unique cases are rejected by Ruby: * Leading '_' or trailing '_', '-', or '+' * '_' before or after 'e' or 'E' exponent separator This logic does a single scan of incoming bytes to reject these cases.

headius · 2026-05-08T19:55:35Z

There's still room for improvement here (it's not quite as fast as my original loose version that didn't reject inappropriate syntax) but we'll go with it. I've updated the description with the final numbers.

Probably just golfing at this point.

headius force-pushed the fast_float_parsing branch from 3628c5d to a402496 Compare January 7, 2026 21:41

headius changed the base branch from master to 10.1-dev January 7, 2026 21:41

headius mentioned this pull request Jan 7, 2026

Skip underscores wrandelshofer/FastDoubleParser#85

Closed

Use configurable parser

14a3511

* Allow underscore as a group separator (essentially ignoring it) * Treat "NaN", "Infinity", and "+Infinity" as unparsable (0.0)

headius force-pushed the fast_float_parsing branch from 1470290 to 14a3511 Compare January 7, 2026 22:40

headius mentioned this pull request Jan 7, 2026

Implement fast float parsing and to_s #9162

Open

Integrate ConfigurableDoubleParser into ConvertDouble using an optimi…

84eeb28

…stic "fast path" approach, followed by a "slow path" for general cases.

RubyFloat and RubyNumeric must pass the proper "strict" argument valu…

b255957

…e to ConvertDouble.

headius added 2 commits May 8, 2026 11:16

Use configurable parser

76e182d

* Allow underscore as a group separator (essentially ignoring it) * Treat "NaN", "Infinity", and "+Infinity" as unparsable (0.0)

headius force-pushed the fast_float_parsing branch from 14a3511 to 76e182d Compare May 8, 2026 16:17

headius added 2 commits May 8, 2026 11:18

Merge branch 'fast_float_parsing' into fast_float_parsing

9e3fe74

Merge pull request #6 from wrandelshofer/fast_float_parsing

c1f88a2

Fast float parsing using a "fast path then slow path" approach

headius changed the base branch from 10.1-dev to master May 8, 2026 16:23

Reintegrate fast parser

dea3b78

Conflicting changes led to compile issues, so reintegrate the combined patches.

headius force-pushed the fast_float_parsing branch from a42c28a to dea3b78 Compare May 8, 2026 16:44

headius force-pushed the fast_float_parsing branch from 62b5f68 to 1b7279e Compare May 8, 2026 18:18

headius force-pushed the fast_float_parsing branch from 1b7279e to 8703226 Compare May 8, 2026 19:07

headius force-pushed the fast_float_parsing branch from 8703226 to 9464bd4 Compare May 8, 2026 19:11

Restrict fast parsing cases to accommodate Ruby

5b2c6f7

Several unique cases are rejected by Ruby: * Leading '_' or trailing '_', '-', or '+' * '_' before or after 'e' or 'E' exponent separator This logic does a single scan of incoming bytes to reject these cases.

headius force-pushed the fast_float_parsing branch from 9464bd4 to 5b2c6f7 Compare May 8, 2026 19:35

headius added this to the JRuby 10.1.1.0 milestone May 8, 2026

Additional pair checking for internal /__|_.|+_|-_/

b18c419

headius force-pushed the fast_float_parsing branch from 3ea52af to b18c419 Compare May 8, 2026 20:53

Additional simplification

92a1d1e

Probably just golfing at this point.

headius merged commit 8b79e9e into jruby:master May 8, 2026
108 checks passed

headius deleted the fast_float_parsing branch May 8, 2026 21:21

Uh oh!

Uh oh!

Conversation

headius commented Dec 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

headius commented Dec 28, 2025

Uh oh!

lemire commented Jan 1, 2026

Uh oh!

headius commented Jan 2, 2026

Uh oh!

wrandelshofer commented Jan 2, 2026

Uh oh!

wrandelshofer commented Jan 2, 2026

Uh oh!

headius commented Jan 7, 2026

Uh oh!

headius commented Jan 7, 2026

Uh oh!

headius commented Jan 7, 2026

Uh oh!

wrandelshofer commented Jan 8, 2026

Uh oh!

wrandelshofer commented Jan 8, 2026

Uh oh!

headius commented Jan 8, 2026

Uh oh!

mensfeld commented Apr 3, 2026

Uh oh!

headius commented May 8, 2026

Uh oh!

lemire commented May 8, 2026

Uh oh!

headius commented May 8, 2026

Uh oh!

headius commented May 8, 2026

Uh oh!

headius commented May 8, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

headius commented Dec 28, 2025 •

edited

Loading