Skip to content

perf: optimize upper (6% faster) - #23588

Merged
comphead merged 1 commit into
apache:mainfrom
andygrove:auto-opt/upper-datafusion-20260714-174535
Jul 15, 2026
Merged

comphead merged 1 commit into
apache:mainfrom
andygrove:auto-opt/upper-datafusion-20260714-174535

Conversation

@andygrove

@andygrove andygrove commented Jul 15, 2026 •

Copy link
Copy Markdown
Member

Which issue does this PR close?

N/A

Rationale for this change

Optimize existing expression.

What changes are included in this PR?

Stream uppercase char mapping directly into the string builder in the non-ASCII case-conversion path, eliminating the per-row String heap allocation and copy that str::to_uppercase incurred (lowercase left unchanged for final-sigma correctness; Utf8View left unchanged to preserve byte-identical layout).

Are these changes tested?

Existing tests.

Benchmark (criterion):

  • upper_unicode_utf8: 7.073% faster (base 325499ns -> cand 302477ns)
  • upper_unicode_large_utf8: 6.065% faster (base 325116ns -> cand 305397ns)

Full criterion output:

upper_unicode_utf8      time:   [302.60 µs 303.42 µs 304.22 µs]
                        change: [−7.2166% −7.0726% −6.9250%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 14 outliers among 100 measurements (14.00%)
  14 (14.00%) high severe

upper_unicode_large_utf8
                        time:   [305.90 µs 306.55 µs 307.12 µs]
                        change: [−6.2853% −6.0652% −5.8544%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

Are there any user-facing changes?

No

@github-actions github-actions Bot added the functions Changes to functions implementation label Jul 15, 2026
@andygrove andygrove changed the title perf: optimize upper in datafusion-functions perf: optimize upper (6% faster) Jul 15, 2026
@andygrove
andygrove marked this pull request as ready for review July 15, 2026 13:32

@comphead comphead left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @andygrove

@comphead
comphead added this pull request to the merge queue Jul 15, 2026
Merged via the queue into apache:main with commit ccbf487 Jul 15, 2026
40 checks passed
@andygrove andygrove added the performance Make DataFusion faster label Jul 16, 2026
kosiew pushed a commit to kosiew/datafusion that referenced this pull request Aug 12, 2026
## Which issue does this PR close?

N/A

## Rationale for this change

Optimize existing expression.

## What changes are included in this PR?

Stream uppercase char mapping directly into the string builder in the
non-ASCII case-conversion path, eliminating the per-row String heap
allocation and copy that str::to_uppercase incurred (lowercase left
unchanged for final-sigma correctness; Utf8View left unchanged to
preserve byte-identical layout).

## Are these changes tested?

Existing tests.

Benchmark (criterion):

- upper_unicode_utf8: 7.073% faster (base 325499ns -> cand 302477ns)
- upper_unicode_large_utf8: 6.065% faster (base 325116ns -> cand
305397ns)

Full criterion output:

```text
upper_unicode_utf8      time:   [302.60 µs 303.42 µs 304.22 µs]
                        change: [−7.2166% −7.0726% −6.9250%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 14 outliers among 100 measurements (14.00%)
  14 (14.00%) high severe

upper_unicode_large_utf8
                        time:   [305.90 µs 306.55 µs 307.12 µs]
                        change: [−6.2853% −6.0652% −5.8544%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
```


## Are there any user-facing changes?

No

<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->

<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->
bvolpato pushed a commit to bvolpato/datafusion that referenced this pull request Oct 16, 2026
## Which issue does this PR close?

- N/A

## Rationale for this change

For input arrays with at least one non-ASCII string, apache#23588 changed
`upper` on `Utf8` and `LargeUtf8` to convert each value one character at
a time, writing straight into the output instead of calling
`str::to_uppercase`, to avoid allocating a `String` per row. apache#26192 did
the same for `Utf8View`. However, `str::to_uppercase` has a fast-path
for inputs with leading ASCII bytes, so inputs with a mix of ASCII and
non-ASCII characters got slower: for example, 128-byte ASCII `Utf8View`
values with one non-ASCII value in the array became 19x slower.

This PR reverts that change, but in the course of doing so, we also add
several related optimizations to `upper`, `lower`, and the optimized
string builders; details below.

Benchmarks: (x86, AMD EPYC Milan)

upper
- Some non-ASCII rows:
  - Utf8View, 4096 rows: 952.7 µs → 254.5 µs (−73.3%)
  - Utf8View, 4096 rows, 10% nulls: 881.7 µs → 270.6 µs (−69.3%)
  - Utf8View, 8192 rows: 1.94 ms → 539.7 µs (−72.2%)
  - Utf8View, 8192 rows, 10% nulls: 1.79 ms → 559.4 µs (−68.7%)
- Non-ASCII words:
  - Utf8: 825.5 µs → 726.6 µs (−12.0%)
  - LargeUtf8: 824.7 µs → 726.3 µs (−11.9%)
  - Utf8View: 759.5 µs → 727.6 µs (−4.2%)
- Mostly ASCII:
  - Utf8, len 8: 204.0 µs → 19.6 µs (−90.4%)
  - Utf8View, len 8: 214.6 µs → 36.5 µs (−83.0%)
  - Utf8, len 32: 789.6 µs → 23.2 µs (−97.1%)
  - Utf8View, len 32: 757.4 µs → 36.2 µs (−95.2%)
  - Utf8, len 128: 3.13 ms → 51.7 µs (−98.3%)
  - Utf8View, len 128: 2.89 ms → 47.5 µs (−98.4%)
- Late non-ASCII:
  - Utf8, len 8: 268.2 µs → 154.1 µs (−42.5%)
  - Utf8View, len 8: 259.4 µs → 156.5 µs (−39.7%)
  - Utf8, len 32: 881.8 µs → 136.4 µs (−84.5%)
  - Utf8View, len 32: 798.5 µs → 138.1 µs (−82.7%)
  - Utf8, len 128: 3.20 ms → 164.5 µs (−94.9%)
  - Utf8View, len 128: 2.95 ms → 163.9 µs (−94.4%)

lower
- Some non-ASCII rows:
  - Utf8View, 4096 rows: 270.3 µs → 232.8 µs (−13.9%)
  - Utf8View, 4096 rows, 10% nulls: 271.0 µs → 235.8 µs (−13.0%)
  - Utf8View, 8192 rows: 568.8 µs → 495.8 µs (−12.8%)
  - Utf8View, 8192 rows, 10% nulls: 572.5 µs → 502.4 µs (−12.3%)
- Non-ASCII words:
  - Utf8: 514.2 µs → 511.0 µs (−0.6%)
  - LargeUtf8: 507.8 µs → 516.4 µs (+1.7%)
  - Utf8View: 510.2 µs → 509.4 µs (−0.2%)
- Mostly ASCII:
  - Utf8, len 8: 67.8 µs → 18.6 µs (−72.5%)
  - Utf8View, len 8: 73.9 µs → 36.5 µs (−50.7%)
  - Utf8, len 32: 60.5 µs → 22.9 µs (−62.2%)
  - Utf8View, len 32: 68.4 µs → 36.5 µs (−46.6%)
  - Utf8, len 128: 76.7 µs → 52.1 µs (−32.1%)
  - Utf8View, len 128: 82.6 µs → 46.8 µs (−43.3%)
- Late non-ASCII:
  - Utf8, len 8: 136.9 µs → 138.8 µs (+1.4%)
  - Utf8View, len 8: 140.0 µs → 139.8 µs (−0.1%)
  - Utf8, len 32: 115.3 µs → 124.8 µs (+8.2%)
  - Utf8View, len 32: 117.4 µs → 125.5 µs (+6.9%)
  - Utf8, len 128: 138.9 µs → 150.0 µs (+8.0%)
  - Utf8View, len 128: 142.4 µs → 152.3 µs (+7.0%)

translate (ASCII scalar from/to, no deletions)
- Utf8:
  - 1024 rows, len 8: 10.1 µs → 9.1 µs (−9.8%)
  - 1024 rows, len 32: 17.9 µs → 15.9 µs (−11.2%)
  - 1024 rows, len 128: 48.8 µs → 39.2 µs (−19.6%)
  - 1024 rows, len 1024: 363.8 µs → 270.7 µs (−25.6%)
  - 4096 rows, len 8: 37.5 µs → 33.7 µs (−10.1%)
  - 4096 rows, len 32: 68.6 µs → 62.8 µs (−8.5%)
  - 4096 rows, len 128: 195.8 µs → 154.6 µs (−21.1%)
  - 4096 rows, len 1024: 1.42 ms → 1.07 ms (−24.7%)
- Utf8View:
  - 1024 rows, len 8: 9.1 µs → 8.4 µs (−8.2%)
  - 1024 rows, len 32: 14.5 µs → 13.8 µs (−4.6%)
  - 1024 rows, len 128: 36.2 µs → 35.0 µs (−3.3%)
  - 1024 rows, len 1024: 236.2 µs → 235.4 µs (−0.3%)
  - 4096 rows, len 8: 34.4 µs → 31.1 µs (−9.5%)
  - 4096 rows, len 32: 55.4 µs → 52.5 µs (−5.2%)
  - 4096 rows, len 128: 142.2 µs → 137.0 µs (−3.7%)
  - 4096 rows, len 1024: 937.7 µs → 940.8 µs (+0.3%)

replace (single ASCII byte, Utf8)
- 1024 rows, len 32: 26.2 µs → 9.0 µs (−65.6%)
- 1024 rows, len 128: 92.6 µs → 16.4 µs (−82.3%)
- 1024 rows, len 32, 20% nulls: 22.8 µs → 9.1 µs (−60.3%)
- 1024 rows, len 128, 20% nulls: 76.5 µs → 14.6 µs (−80.9%)
- 4096 rows, len 32: 103.2 µs → 31.2 µs (−69.7%)
- 4096 rows, len 128: 372.6 µs → 61.9 µs (−83.4%)
- 4096 rows, len 32, 20% nulls: 89.9 µs → 33.5 µs (−62.7%)
- 4096 rows, len 128, 20% nulls: 316.5 µs → 56.8 µs (−82.1%)

## What changes are included in this PR?

* Add `StringViewArrayBuilder::try_append_byte_map`, a fallible variant
of the existing `append_byte_map`. (We need this to avoid regressing
error handling behavior on overflow when we use this method below.)
* `lower`, `upper`, for non-ASCII input arrays: we now check if each row
is ASCII. If it is, we construct the result value directly via the
string builder's `try_append_byte_map` method, which vectorizes.
Non-ASCII rows are converted via `std::to_uppercase /
std::to_lowercase`.
* Make `GenericStringArrayBuilder::try_append_byte_map` copy the bytes
into the value buffer and then map them in place, instead of calling
Arrow's `MutableBuffer::extend` with a mapping iterator. The compiler
turns the in-place loop into vector code, while the `extend` loop stayed
scalar. This also speeds up `translate` and `replace`, which call this
method.
* Comment cleanups / improvements
* Test consolidation: remove redundant tests, move more tests to SLT,
improve test coverage
* Benchmark consolidation: move `upper` and `lower` benchmarks to the
same place, add benchmarks for `upper` and `lower` on mixed ASCII and
non-ASCII inputs, add benchmark for `translate` on `Utf8View`

## What is the testing strategy for this PR?

Existing tests pass; new tests added.

## Are there any user-facing changes?

No.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation performance Make DataFusion faster v55.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants