Improve worst case performance when using ARM Neon instructions by samyron · Pull Request #970 · ohler55/oj

samyron · 2025-06-01T03:32:40Z

This PR significantly improves the worst case performance in oj_dump_cstr when dumping strings with a significant number of characters that need to be escaped.

This PR also simplifies the ARM Neon implementation.

Using the same real-world and worst-case benchmarks as on #967 we can see that real-world performance is effectively the same (as compared to master) whereas the worst-case benchmarks are considerably less-bad.

Benchmarks

Compiler: clang

== Encoding activitypub.json (52595 bytes)
ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +PRISM [arm64-darwin24]
Warming up --------------------------------------
               after     2.224k i/100ms
Calculating -------------------------------------
               after     22.049k (± 1.9%) i/s   (45.35 μs/i) -    111.200k in   5.045147s

Comparison:
              before:    17756.8 i/s
               after:    22049.3 i/s - 1.24x  faster


== Encoding citm_catalog.json (500298 bytes)
ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +PRISM [arm64-darwin24]
Warming up --------------------------------------
               after    91.000 i/100ms
Calculating -------------------------------------
               after    908.132 (± 1.1%) i/s    (1.10 ms/i) -      4.550k in   5.010876s

Comparison:
              before:      921.4 i/s
               after:      908.1 i/s - same-ish: difference falls within error


== Encoding twitter.json (466906 bytes)
ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +PRISM [arm64-darwin24]
Warming up --------------------------------------
               after   219.000 i/100ms
Calculating -------------------------------------
               after      2.182k (± 1.2%) i/s  (458.22 μs/i) -     10.950k in   5.018345s

Comparison:
              before:     1992.3 i/s
               after:     2182.3 i/s - 1.10x  faster


== Encoding ohai.json (20145 bytes)
ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +PRISM [arm64-darwin24]
Warming up --------------------------------------
               after     2.854k i/100ms
Calculating -------------------------------------
               after     29.152k (± 0.8%) i/s   (34.30 μs/i) -    148.408k in   5.091188s

Comparison:
              before:    28830.3 i/s
               after:    29151.8 i/s - same-ish: difference falls within error


== Encoding bytes.15.bestcase (180001 bytes)
ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +PRISM [arm64-darwin24]
Warming up --------------------------------------
               after   549.000 i/100ms
Calculating -------------------------------------
               after      5.585k (± 1.8%) i/s  (179.04 μs/i) -     27.999k in   5.014602s

Comparison:
              before:     5646.1 i/s
               after:     5585.2 i/s - same-ish: difference falls within error


== Encoding bytes.15.worstcase (330001 bytes)
ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +PRISM [arm64-darwin24]
Warming up --------------------------------------
               after   274.000 i/100ms
Calculating -------------------------------------
               after      2.716k (± 1.0%) i/s  (368.19 μs/i) -     13.700k in   5.044700s

Comparison:
              before:     2823.2 i/s
               after:     2716.0 i/s - 1.04x  slower


== Encoding bytes.15.worstcase-2 (580001 bytes)
ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +PRISM [arm64-darwin24]
Warming up --------------------------------------
               after   277.000 i/100ms
Calculating -------------------------------------
               after      2.761k (± 2.2%) i/s  (362.22 μs/i) -     13.850k in   5.019212s

Comparison:
              before:     2684.6 i/s
               after:     2760.8 i/s - same-ish: difference falls within error


== Encoding bytes.32.worstcase (670001 bytes)
ruby 3.4.1 (2024-12-25 revision 48d4efcb85) +PRISM [arm64-darwin24]
Warming up --------------------------------------
               after   135.000 i/100ms
Calculating -------------------------------------
               after      1.322k (± 1.7%) i/s  (756.42 μs/i) -      6.615k in   5.005089s

Comparison:
              before:     1424.4 i/s
               after:     1322.0 i/s - 1.08x  slower

Compiler: gcc-14

== Encoding activitypub.json (52595 bytes)
ruby 3.3.8 (2025-04-09 revision b200bad6cd) +YJIT [arm64-darwin24]
Warming up --------------------------------------
               after     2.363k i/100ms
Calculating -------------------------------------
               after     23.530k (± 1.6%) i/s   (42.50 μs/i) -    118.150k in   5.022539s

Comparison:
              before:    16499.4 i/s
               after:    23530.1 i/s - 1.43x  faster


== Encoding citm_catalog.json (500298 bytes)
ruby 3.3.8 (2025-04-09 revision b200bad6cd) +YJIT [arm64-darwin24]
Warming up --------------------------------------
               after   110.000 i/100ms
Calculating -------------------------------------
               after      1.109k (± 1.0%) i/s  (902.08 μs/i) -      5.610k in   5.061164s

Comparison:
              before:     1122.9 i/s
               after:     1108.5 i/s - same-ish: difference falls within error


== Encoding twitter.json (466906 bytes)
ruby 3.3.8 (2025-04-09 revision b200bad6cd) +YJIT [arm64-darwin24]
Warming up --------------------------------------
               after   243.000 i/100ms
Calculating -------------------------------------
               after      2.441k (± 1.6%) i/s  (409.71 μs/i) -     12.393k in   5.078927s

Comparison:
              before:     2216.7 i/s
               after:     2440.7 i/s - 1.10x  faster


== Encoding ohai.json (20145 bytes)
ruby 3.3.8 (2025-04-09 revision b200bad6cd) +YJIT [arm64-darwin24]
Warming up --------------------------------------
               after     3.242k i/100ms
Calculating -------------------------------------
               after     32.851k (± 0.8%) i/s   (30.44 μs/i) -    165.342k in   5.033388s

Comparison:
              before:    33676.8 i/s
               after:    32850.9 i/s - 1.03x  slower


== Encoding bytes.15.bestcase (180001 bytes)
ruby 3.3.8 (2025-04-09 revision b200bad6cd) +YJIT [arm64-darwin24]
Warming up --------------------------------------
               after   636.000 i/100ms
Calculating -------------------------------------
               after      6.312k (± 2.9%) i/s  (158.42 μs/i) -     31.800k in   5.042145s

Comparison:
              before:     6487.5 i/s
               after:     6312.4 i/s - same-ish: difference falls within error


== Encoding bytes.15.worstcase (330001 bytes)
ruby 3.3.8 (2025-04-09 revision b200bad6cd) +YJIT [arm64-darwin24]
Warming up --------------------------------------
               after   215.000 i/100ms
Calculating -------------------------------------
               after      2.162k (± 2.0%) i/s  (462.44 μs/i) -     10.965k in   5.072660s

Comparison:
              before:     2183.6 i/s
               after:     2162.5 i/s - same-ish: difference falls within error


== Encoding bytes.15.worstcase-2 (580001 bytes)
ruby 3.3.8 (2025-04-09 revision b200bad6cd) +YJIT [arm64-darwin24]
Warming up --------------------------------------
               after   278.000 i/100ms
Calculating -------------------------------------
               after      2.744k (± 1.7%) i/s  (364.47 μs/i) -     13.900k in   5.067626s

Comparison:
              before:     2967.0 i/s
               after:     2743.7 i/s - 1.08x  slower


== Encoding bytes.32.worstcase (670001 bytes)
ruby 3.3.8 (2025-04-09 revision b200bad6cd) +YJIT [arm64-darwin24]
Warming up --------------------------------------
               after   109.000 i/100ms
Calculating -------------------------------------
               after      1.093k (± 2.6%) i/s  (914.78 μs/i) -      5.559k in   5.088850s

Comparison:
              before:     1406.4 i/s
               after:     1093.2 i/s - 1.29x  slower

… the worst case synthetic benchmarks.

ohler55 · 2025-06-01T12:45:25Z

There seems to be some confusion with the data files. I noticed some were used in the benchmarks and some were not. I also not that the benchmarks list other files not in the date such as "bytes.15.worstcase".

There is a duplicate data file activitypub.compact.txt and activitypub.compat.txt.

If the files are JSON and they appear to be then why give them a ".json" suffix instead of ".txt"?

Thee other changes tentatively look fine but I'd like to get a better feel for what type of files are slower and which are faster.

samyron · 2025-06-01T15:14:24Z

There seems to be some confusion with the data files. I noticed some were used in the benchmarks and some were not. I also not that the benchmarks list other files not in the date such as "bytes.15.worstcase".

The bytes.15.worstcase isn't a file, it's just a literal string. These are the synthetic benchmarks included in the results above:

benchmark_encoding "bytes.15.bestcase", ([("a" * 15)] * 10000)
benchmark_encoding "bytes.15.worstcase", ([('"' * 15)] * 10000)
benchmark_encoding "bytes.15.worstcase-2", (["\u0001a\u0002b\u0001a\u0002b\u0001a\u0002b\u0001a\u0002"] * 10000)
benchmark_encoding "bytes.32.worstcase", ([("\"\t\\\n" * 8)] * 10000)

There is a duplicate data file activitypub.compact.txt and activitypub.compat.txt.

Oops... apologies about that. I generated the first by hand then wrote a script... but I guess I had a typo in that first one. I will also add a snippet for how these files were generated to hopefully save someone a few minutes in the future should they ever need to be re-generated.

If the files are JSON and they appear to be then why give them a ".json" suffix instead of ".txt"?

They are, I'll fix the extension.

Thee other changes tentatively look fine but I'd like to get a better feel for what type of files are slower and which are faster.

Data that is very heavy in bytes that need to be escaped will benefit the most from these changes. Should someone decide to JSON encode binary data, I would expect these changes would help a lot as compared to the current ARM Neon implementation.

Text-based input likely wouldn't see much change from these changes.

ohler55 · 2025-06-01T18:56:32Z

All your responses make sense. Once you mark the PR as read for review I'll approve and merge. Assuming tests pass of course. :-)

samyron added 4 commits May 31, 2025 21:59

Refactored and simplified the ARM Neon implementation. This optimizes…

ebd400f

… the worst case synthetic benchmarks.

Merge branch 'develop' into arm-neon-optimize-oj-dump-cstr-worstcase

637585f

Move the definition of FORCE_INLINE out of the HAVE_SIMD_NEON block.

aa646b2

Formatting.

a6895c6

Renamed the expected real-world test data files.

c4566a2

samyron marked this pull request as ready for review June 2, 2025 01:47

ohler55 approved these changes Jun 2, 2025

View reviewed changes

ohler55 merged commit 2e95f15 into ohler55:develop Jun 2, 2025
53 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Improve worst case performance when using ARM Neon instructions#970

Improve worst case performance when using ARM Neon instructions#970
ohler55 merged 5 commits intoohler55:developfrom
samyron:arm-neon-optimize-oj-dump-cstr-worstcase

samyron commented Jun 1, 2025

Uh oh!

ohler55 commented Jun 1, 2025

Uh oh!

samyron commented Jun 1, 2025

Uh oh!

ohler55 commented Jun 1, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

samyron commented Jun 1, 2025

Benchmarks

Compiler: clang

Compiler: gcc-14

Uh oh!

ohler55 commented Jun 1, 2025

Uh oh!

samyron commented Jun 1, 2025

Uh oh!

ohler55 commented Jun 1, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants