You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Preserve dictionary-value nulls in scalar regex operators #23963
regex_match_dyn_scalar evaluates a dictionary's values once and gathers the resulting booleans with BooleanArray::take_iter. The gather preserves null dictionary keys, but it does not preserve nulls in the evaluated dictionary values.
As a result, a valid key that references a null dictionary value becomes a valid false. Negated regex operators then turn that value into true, which violates SQL three-valued logic and differs from evaluating the same input after casting it to Utf8.
To Reproduce
Construct a dictionary array whose values contain both a string and a null, and whose keys contain both valid keys and a null key. Evaluate it against a scalar pattern with:
RegexMatch
RegexIMatch
RegexNotMatch
RegexNotIMatch
Compare the result with evaluation after casting the dictionary array to Utf8. The current dictionary-specialized path loses validity for keys that reference the null dictionary value; the cast path retains it.
Expected behavior
Scalar regex evaluation on dictionary input should preserve both dictionary-key validity and dictionary-value validity. Its values and null bitmap should match the equivalent cast-to-Utf8 evaluation for all four regex operators.
Additional context
The dictionary/scalar optimization was introduced in #12768. Current main and the 54.1.0 tag still use the validity-dropping take_iter gather.
#23722 is related but distinct: it addresses dictionary input in the array-pattern regex_match_dyn path, while this report concerns null propagation in regex_match_dyn_scalar.
A downstream reproduction and focused fix are available in GreptimeTeam#22.
Activity
fix: preserve dictionary-value nulls in scalar regex operators (apach…
fix: preserve dictionary-value nulls in scalar regex operators (apach…