Skip to content

Preserve dictionary-value nulls in scalar regex operators #23963

Closed

Description

@discord9

Describe the bug

regex_match_dyn_scalar evaluates a dictionary's values once and gathers the resulting booleans with BooleanArray::take_iter. The gather preserves null dictionary keys, but it does not preserve nulls in the evaluated dictionary values.

As a result, a valid key that references a null dictionary value becomes a valid false. Negated regex operators then turn that value into true, which violates SQL three-valued logic and differs from evaluating the same input after casting it to Utf8.

To Reproduce

Construct a dictionary array whose values contain both a string and a null, and whose keys contain both valid keys and a null key. Evaluate it against a scalar pattern with:

  • RegexMatch
  • RegexIMatch
  • RegexNotMatch
  • RegexNotIMatch

Compare the result with evaluation after casting the dictionary array to Utf8. The current dictionary-specialized path loses validity for keys that reference the null dictionary value; the cast path retains it.

Expected behavior

Scalar regex evaluation on dictionary input should preserve both dictionary-key validity and dictionary-value validity. Its values and null bitmap should match the equivalent cast-to-Utf8 evaluation for all four regex operators.

Additional context

The dictionary/scalar optimization was introduced in #12768. Current main and the 54.1.0 tag still use the validity-dropping take_iter gather.

#23722 is related but distinct: it addresses dictionary input in the array-pattern regex_match_dyn path, while this report concerns null propagation in regex_match_dyn_scalar.

A downstream reproduction and focused fix are available in GreptimeTeam#22.

Activity

  1. added a commit that references this issue on Jul 30, 2026
    cc1326a
  2. added a commit that references this issue on Aug 12, 2026
    6c95227
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions