Summary

This PR enables SIMD (Single Instruction, Multiple Data) optimizations automatically based on CPU capabilities, providing performance improvements for JSON string parsing without requiring manual configuration.

Previously, users had to pass --with-sse42 during gem installation to enable SIMD. Now it's enabled by default and automatically detects the best instruction set for the CPU.

Note: This PR was developed with Claude Code AI - an AI pair programming tool that helped with SIMD optimization, benchmarking, and implementation.

Performance Improvements

Benchmarked on:

• CPU: AMD EPYC 7282 16-Core Processor (SSE4.2 capable)
• RAM: 48GB
• Ruby: 3.4.7 (2025-10-08) +PRISM [x86_64-linux]
• Platform: x86_64-linux

Results (50,000 iterations):

Key Win: Best improvements on strings with escape sequences (most common real-world scenario).

Changes

1. Simplified extconf.rb (4 lines)

# Enable SIMD optimizations - try SSE4.2 on x86_64 for best performance
# Falls back to SSE2 or compiler defaults if not available
if try_cflags('-msse4.2')
  $CPPFLAGS += ' -msse4.2'
elsif try_cflags('-msse2')
  $CPPFLAGS += ' -msse2'
end

Before: Required gem install oj -- --with-sse42
After: Just gem install oj - SIMD enabled automatically ✨

2. Enhanced simd.h

• Unified CPU architecture detection
• Defines: HAVE_SIMD_SSE4_2, HAVE_SIMD_SSE2, HAVE_SIMD_NEON
• Clean #ifdef based conditional compilation
• Priority: SSE4.2 > NEON > SSE2 > scalar
• Ready for ARM NEON support

3. Optimized SIMD String Scanner (parse.c)

SSE4.2 implementation (modern x86_64):

• Processes 64 bytes per iteration (4×16-byte chunks)
• Prefetches next cache line with __builtin_prefetch()
• Parallel chunk loading for better instruction-level parallelism
• Branch prediction hints with __builtin_expect()

SSE2 fallback (older x86_64):

• Same 64-byte optimization strategy
• Uses SSE2 instructions (available on all x86_64 CPUs)
• Provides SIMD benefits even on older hardware

Testing

✅ All tests pass: 445 runs, 986 assertions, 0 failures, 0 errors
✅ Clean builds verified
✅ Proper baseline comparisons done

Breaking Changes

None. This is a pure improvement that maintains full backward compatibility.

Benefits

1. Better defaults - SIMD just works automatically
2. Improved performance - 3-8% faster string parsing
3. Broader compatibility - SSE2 fallback for older CPUs
4. Future-ready - Clean architecture for ARM NEON
5. Better UX - No manual flags needed during installation

Development Process

This PR was developed with Claude Code AI, which assisted with:

• SIMD optimization strategies and implementation
• Performance benchmarking and analysis
• Architecture detection across x86_64 and ARM
• Testing and validation

Related Issues

Addresses user requests for automatic SIMD enablement and improved default performance.

🤖 Built with Claude Code

Co-Authored-By: Claude [email protected]