Batch ByteBuffer slice in RLE PACKED decode to reduce allocation overhead

XMLWordPrintableJSON

      Replaces per-group in.slice(bitWidth) (one ByteBuffer allocation per 8 values) with a single bulk slice for the entire RLE PACKED run in the Parquet vectorized reader.

      Changes

      • Batch ByteBuffer slice: Instead of allocating a new ByteBuffer for each 8-value group in a PACKED run, allocate a single slice covering all groups and advance a position cursor.
      • Eliminates ~128K short-lived ByteBuffer allocations per 1M-value page.

      Benchmark Results (GHA, AMD EPYC 7763)

      bitWidth Speedup (readIntegers) Speedup (skipIntegers)
      4 2.0x 2.1x
      8 2.0x 2.4x
      12 1.6x 1.6x
      20 1.4x 1.4x

      PR: https://github.com/apache/spark/pull/55922
      Parent issue: https://github.com/apache/spark/issues/56011

            Assignee:
            Ismaël Mejía
            Reporter:
            Ismaël Mejía
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: