feat(arrow-string): support run-end encoded operands in like kernels by asubiotto · Pull Request #12154 · apache/arrow-rs

asubiotto · 2026-11-10T13:34:12Z

Which issue does this PR close?

Closes Support RunEndEncoded arrays in string matching kernels (like, ilike, starts_with, etc.) #12153.

Rationale for this change

The string matching kernels return an error for RunEndEncoded inputs, so callers have to decode before filtering.

What changes are included in this PR?

like, ilike, nlike, nilike, starts_with, ends_with, contains and eq_ignore_ascii_case accept run-end encoded operands, including run-end encoded dictionaries. Value types are the same as for unencoded inputs, so binary values also work with starts_with, ends_with and contains.

Run-end encoded operands are rewritten into encodings the kernels already support:

Against a scalar: match each run once and repeat the result across the run.
Two arrays: look up each row through a dictionary view of the runs, without copying values.

Adds an ree_like benchmark group.

Are these changes tested?

Yes. The existing string test macros also run each case on RunArray, and new tests cover slicing mid-run, nulls, dictionary values, run-end encoded patterns, empty inputs, type errors and binary values.

Benchmarks for 65,536 rows:

Benchmark	phys=64	phys=1024	phys=32768
`like_flat_scalar`	335 µs	326 µs	330 µs
`like_ree_scalar`	0.86 µs	9.4 µs	257 µs
`like_ree_dict_scalar`	1.03 µs	10.0 µs	271 µs
`like_ree_decoded_scalar`	787 µs	847 µs	976 µs
`like_ree_flat`	871 µs	887 µs	1.21 ms

Are there any significant user-facing changes?

No. Run-end encoded inputs that previously returned an error are now supported.

AI assistance was used for this PR but I thoroughly reviewed this change before opening a PR.

cc @alamb @Rich-T-kid

Accept run-end encoded operands, including dictionary-encoded run values, in like, ilike, nlike, nilike, starts_with, ends_with, contains and eq_ignore_ascii_case, for both string and binary values. Run-end encoded operands are rewritten into encodings the kernels already support, so type checks, null handling and errors are shared with the unencoded path. A scalar pattern is matched once per run covered by the slice and the result is repeated across the run. Array patterns look up each row through a dictionary view of the runs, which does not copy values. The existing string test macros now also run every case on run-end encoded arrays. Add an ree_like benchmark group alongside ree_comparison. Prepared with AI assistance. Co-authored-by: Amp <[email protected]>

LIKE and ILIKE type coercion used to cast run-end encoded operands to their string value type, decoding every row before matching. Keep a run-end encoded operand uncast when its values (optionally dictionary encoded) already have the common LIKE type, so the kernels match each run once. Other operands are still cast to the common type, and SIMILAR TO still decodes because the regex kernels need plain strings. The rule lives next to `like_coercion` in `datafusion-expr-common` and is shared by the analyzer and the physical `like` type check. Depends on apache/arrow-rs#12154, which adds run-end encoded support to the Arrow LIKE kernels. Without it these queries fail with "Invalid string/binary operation".

LIKE type coercion decoded run-end encoded operands to their string values, which expands every run before matching. Look through run-end encoding in `like_value_type`, so a run-end encoded operand whose values (optionally dictionary encoded) already have the common type is kept and the kernels match each run once. Depends on apache/arrow-rs#12154, which adds run-end encoded support to the Arrow LIKE kernels. Without it these queries fail with "Invalid string/binary operation".

Rich-T-kid

can we split the benchmarks into another pr or make it the first commit so we can use the bot to compare

asubiotto · 2026-11-10T19:34:15Z

The benchmarks would just error without this change so there is nothing to compare against.

asubiotto · 2026-11-12T12:28:54Z

Friendly ping. Does the PR look good otherwise?

Rich-T-kid · 2026-11-12T14:07:19Z

sorry I haven't taken too close of a look. i've been busy with other work, ill try and take a look soon

Keep an outer RunEndEncoded over Utf8 or dictionary-encoded Utf8 when the coerced pattern is Utf8. Preserve ordinary casts for other layouts and SIMILAR TO. Requires Arrow LIKE kernel support from apache/arrow-rs#12154. Amp-Thread-ID: https://ampcode.com/threads/T-01a1c09d-06c3-76af-8d8a-d3c4ebe25726 Co-authored-by: Amp <[email protected]>

Use arrow-string from apache/arrow-rs#12154 at 5d192d080196a116075a18712ea2d572e00547b1. Keep all other Arrow packages on registry 60.1.0 so the branch has one Arrow type graph. Replace this wiring with a released Arrow dependency before upstream integration. Amp-Thread-ID: https://ampcode.com/threads/T-01a1c09d-06c3-76af-8d8a-d3c4ebe25726 Co-authored-by: Amp <[email protected]>

Keep an outer RunEndEncoded over Utf8 or dictionary-encoded Utf8 when the coerced pattern is Utf8. Preserve ordinary casts for other layouts and SIMILAR TO. Requires Arrow LIKE kernel support from apache/arrow-rs#12154. Co-authored-by: Amp <[email protected]> Backport note: the SIMILAR TO cases in run_end_encoded.slt use the pattern 'Ab_c' instead of 'Ab%'. This branch lacks apache#23188, which translates SQL wildcards in SIMILAR TO, so 'Ab%' matches a literal %. The cases still check that SIMILAR TO keeps the cast. (cherry picked from commit 019dde2) Amp-Thread-ID: https://ampcode.com/threads/T-01a1cfce-da43-7260-bbd6-097791371a27

Keep an outer RunEndEncoded over Utf8 or Dictionary(_, Utf8) uncast when the LIKE pattern coerces to Utf8, as LIKE already does for Dictionary(_, Utf8). Other layouts and SIMILAR TO keep their casts. Requires Arrow LIKE kernel support from apache/arrow-rs#12154. Co-authored-by: Amp <[email protected]> Amp-Thread-ID: https://ampcode.com/threads/T-01a1d03b-18cf-75e2-9af2-58698a0382fd

Use arrow-string from apache/arrow-rs#12154 at 5d192d080196a116075a18712ea2d572e00547b1. Keep all other Arrow packages on registry 60.1.0 so the branch has one Arrow type graph. Drop this commit once an Arrow release includes apache/arrow-rs#12154. Co-authored-by: Amp <[email protected]> Amp-Thread-ID: https://ampcode.com/threads/T-01a1d03b-18cf-75e2-9af2-58698a0382fd

Rich-T-kid · 2026-11-12T22:18:52Z

+        Ok(expand_runs(left, &like_op(op, &run_values(left), rhs)?))
+    } else if left_is_scalar {
+        Ok(expand_runs(right, &like_op(op, lhs, &run_values(right))?))
+    } else if left.is_empty() {


This returns Ok([]) before validating the underlying types. Could we preserve type validation here and add a mismatched empty-input test?

let run: RunArray<Int32Type> = std::iter::empty::<&str>().collect(); let right = Int32Array::from(Vec::<i32>::new()); assert!(like(&run, &right).is_err());

This currently returns Ok(BooleanArray::from([])).

Good catch, done!

Empty run-end encoded inputs returned an empty result without checking operand types. Match their run values through like_op instead, which checks types and avoids building a dictionary without values.

Rich-T-kid

thanks @asubiotto

alamb

I took a good look at this too -- thank you @Rich-T-kid and @asubiotto

alamb · 2026-11-13T20:33:05Z

+    } else if left_is_scalar {
+        Ok(expand_runs(right, &like_op(op, lhs, &run_values(right))?))
+    } else if left.is_empty() {
+        // The dictionary kernels panic on a dictionary without values, so match the


is there a reason that the dictionary kernels panic on empty values? Maybe we should fix that (and we could remove this special case) -- to consider for a follow on PR

Created a follow-up issue for this: #12243

boring-cyborg Bot added arrow-core Core crates related to representing arrow data arrow-kernels Crates that transform arrow data (e.g. compute, cast) labels Nov 10, 2026

Rich-T-kid reviewed Nov 10, 2026

View reviewed changes

Rich-T-kid self-requested a review November 10, 2026 20:24

asubiotto mentioned this pull request Nov 12, 2026

Avoid casting run-end encoded Utf8 columns in LIKE / ILIKE apache/datafusion#27376

Open

Rich-T-kid requested changes Nov 12, 2026

View reviewed changes

asubiotto mentioned this pull request Nov 13, 2026

perf: preserve run-end encoded UTF-8 LIKE operands apache/datafusion#27402

Draft

fix(arrow-string): check operand types for empty run-end encoded inputs

dd79d96

Empty run-end encoded inputs returned an empty result without checking operand types. Match their run values through like_op instead, which checks types and avoids building a dictionary without values.

Jefffrey mentioned this pull request Nov 13, 2026

Release arrow-rs / parquet Minor version 60.2.0 (November 2026) #10526

Open

6 of 16 tasks

Rich-T-kid self-requested a review November 13, 2026 15:27

Rich-T-kid approved these changes Nov 13, 2026

View reviewed changes

alamb approved these changes Nov 13, 2026

View reviewed changes

alamb added this pull request to the merge queue Nov 13, 2026

Rich-T-kid mentioned this pull request Nov 13, 2026

Investigate dictionary string kernel panic with empty values #12243

Open

Merged via the queue into apache:main with commit 0ada6ff Nov 13, 2026
22 checks passed

Navigation Menu

Repository navigation

feat(arrow-string): support run-end encoded operands in like kernels - #12154

feat(arrow-string): support run-end encoded operands in like kernels#12154
alamb merged 2 commits into
apache:mainfrom
dash0hq:direct-ree-like

asubiotto commented Nov 10, 2026 •

edited

Loading

Uh oh!

Rich-T-kid left a comment

Uh oh!

asubiotto commented Nov 10, 2026

Uh oh!

asubiotto commented Nov 12, 2026

Uh oh!

Rich-T-kid commented Nov 12, 2026

Uh oh!

Rich-T-kid Nov 12, 2026 •

edited

Loading

Uh oh!

asubiotto Nov 13, 2026

Uh oh!

Rich-T-kid left a comment

Uh oh!

alamb left a comment

Uh oh!

alamb Nov 13, 2026

Uh oh!

Rich-T-kid Nov 13, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

asubiotto commented Nov 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any significant user-facing changes?

Uh oh!

Rich-T-kid left a comment

Choose a reason for hiding this comment

Uh oh!

asubiotto commented Nov 10, 2026

Uh oh!

asubiotto commented Nov 12, 2026

Uh oh!

Rich-T-kid commented Nov 12, 2026

Uh oh!

Rich-T-kid Nov 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

asubiotto Nov 13, 2026

Choose a reason for hiding this comment

Uh oh!

Rich-T-kid left a comment

Choose a reason for hiding this comment

Uh oh!

alamb left a comment

Choose a reason for hiding this comment

Uh oh!

alamb Nov 13, 2026

Choose a reason for hiding this comment

Uh oh!

Rich-T-kid Nov 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

asubiotto commented Nov 10, 2026 •

edited

Loading

Rich-T-kid Nov 12, 2026 •

edited

Loading