Repository navigation
Execute lower and strpos inputs through execute_step - #11407
Conversation
While a scalar function waits on the executor stack for an input, execution can rewrite that input into an encoding whose parent-reduce rules now match. The optimizer only saw the input as it was before execution, so those rules never ran. A slice over a run-end array executes into the run-end array, but `RunEndScalarFnRule` never pushed the suspended between into the run values, so every row was decoded and compared. After a child under a suspended parent is rewritten, put it back into the parent and try its parent-reduce rules, session kernels first. A rewrite replaces the parent; otherwise the child is detached again. Both are in place when the parent is uniquely owned. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_018SWbEuVduQbmfuXAPx4p46 Signed-off-by: Robert Kruszewski <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_018SWbEuVduQbmfuXAPx4p46 Signed-off-by: Robert Kruszewski <[email protected]>
Lower and strpos decoded their input in a nested executor, so a slice or mask over a run-end or chunked array reached them as a canonical array and they ran once per row. Request the input with execute_step, as Like and Between do, so the encoding can first push the function into its values. Signed-off-by: Alfonso Subiotto <[email protected]> Co-authored-by: Amp <[email protected]>
Merging this PR will improve performance by 11.97%2 warnings
Performance Changes
Tip Curious why performance improved? Comment Comparing 
Footnotes
|
Summary
Stacked on #11310. Until that merges, this PR shows only the last commit.
LowerandStrposdecode their inputs insideexecute, in a nested executor. When the input is a slice or a mask over a run-end or chunked array, the function gets a canonical array and runs once per row, even when the encoding could have pushed it into its values. #11310 lets the executor retry parent-reduce rules once a child is rewritten, but only for functions that hand their inputs back to the executor throughexecute_step. This continues theexecute_stepmigration from #11100 and #11154 forLowerandStrpos.Changes
Lower::execute_stepandStrpos::execute_steprequest their inputs withrequire_input!(args, i => AnyColumnar), asLikeandBetweendo, then callexecute.lower'spreserves_runs_and_dictionarygets a sliced case, andstrposgetspreserves_runs_of_a_slice. Both check that a function over a sliced run-end input stays run-end encoded. Both fail without this change, and both fail without Try reduce_parent rules after execute_parent rewrites the current node #11310.Checks:
cargo test -p vortex-arraypasses.cargo clippy -p vortex-array --all-targets --all-features -- -D warningspasses.cargo +nightly-2026-09-10 fmt -p vortex-array --checkpasses.Benchmark numbers will follow. No existing benchmark covers scalar functions over sliced or masked encoded inputs.
This change was written with help from an AI coding agent and I haven't reviewed it yet.