You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 007894e
Browse filesBrowse the repository at this point in the historyBrowse files
Reuse buffers when interleaving byte views (apache#11281)
# Which issue does this PR close?
Fixesapache#11280
And also helps with: apache/datafusion#23565
# Rationale for this change
See the original issue, but, basically the interleave kernel can make
sub-optimal string views especially when buffers are shared.
# What changes are included in this PR?
We use a btreemap on the seen buffers (not individual values) and use
that to prevent us from cloning the same buffer multiple times.
# Are these changes tested?
Yes tested, and locally benchmarks have shown a moderate speed increase
in some of the interleave benchmarks, on my dev machine which has an
**AMD Ryzen 9 9950X3D** processor:
| benchmark | main | this PR | change |
|-----------------------------------------------------------------------|----------|----------|--------|
| interleave str_view(0.0) 100 [0..100, 100..230, 450..1000] | 284.5 ns
| 268.6 ns | -5.6% |
| interleave str_view(0.0) 400 [0..100, 100..230, 450..1000] | 706.9 ns
| 563.9 ns | -20.2% |
| interleave str_view(0.0) 1024 [0..100, 100..230, 450..1000] | 1.717 µs
| 1.250 µs | -27.2% |
| interleave str_view(0.0) 1024 [0..100, 100..230, 450..1000, 0..1000] |
1.783 µs | 1.274 µs | -28.6% |
# Are there any user-facing changes?
No user facing changes, just an optimization.
0 commit comments