Expand LIKE simplification: cover `NULL` pattern/expression and constant by findepi · Pull Request #13260 · apache/datafusion

findepi · 2024-11-05T12:51:11Z

cover expression known not to be null
cover NULL pattern
cover repeated '%%' in pattern
currently also includes changes from Simplify EXPR LIKE 'constant' to expr = 'constant' #13061 to resolve conflicts

findepi · 2024-11-05T13:39:35Z

currently depends on #13259

findepi · 2024-11-05T15:04:19Z

CI is green, so ready to review
but this will conflict with @adriangb's #13061, because this changes code structure a little bit
conflict resolution will be trivial though.
@adriangb if you want, i can do this, adding your changes to this PR

findepi · 2024-11-05T15:12:34Z

seemed easy enough, done.

adriangb · 2024-11-05T15:15:40Z

I’m happy with my changes being included in this PR :)

findepi · 2024-11-05T15:22:34Z

draft - to be rebased after #13259 lands

still ready to review
cc @crepererum @goldmedal

goldmedal

Just roughly review now. Overall looks to me. I will check the test cases tomorrow.

datafusion/optimizer/Cargo.toml

goldmedal · 2024-11-06T02:51:07Z

draft - to be rebased after #13259 lands

still ready to review cc @crepererum @goldmedal

#13259 has been merged. 👍

findepi · 2024-11-06T07:31:35Z

@goldmedal rebased, thanks!

alamb

Thank you @findepi and @goldmedal

This looks like a great change to me except for the handling of %% which I am not sure about. Otherwise 👍

alamb · 2024-11-06T11:38:51Z

datafusion/optimizer/src/simplify_expressions/expr_simplifier.rs

    }

-    fn like(expr: Expr, pattern: &str) -> Expr {
+    fn like(expr: Expr, pattern: impl Into<Expr>) -> Expr {


I think we could use Expr::like https://docs.rs/datafusion/latest/datafusion/logical_expr/enum.Expr.html#method.like and similar here instead of these functions

This is likely left over from when the Expr API was less expressive

datafusion/optimizer/src/simplify_expressions/expr_simplifier.rs

datafusion/sqllogictest/test_files/string/string_view.slt

datafusion/optimizer/src/simplify_expressions/expr_simplifier.rs

datafusion/sqllogictest/test_files/string/string_view.slt

crepererum · 2024-11-06T15:43:55Z

since there are already two reviewers on it, I'll gonna skip this PR. However if you need my input, ping me.

alamb

Thank you @findepi and @goldmedal

I had some small testing suggestions but I think we can add that coverage as a follow on PR as well

datafusion/sqllogictest/test_files/string/string_query.slt.part

findepi · 2024-11-07T09:56:29Z

FYI, when testing LIKE patterns with potential escapes, beware of difference between sqllogictests and CLI when it comes to backslash (like implicit escape) -- #13286

datafusion/sqllogictest/test_files/string/string_query.slt.part

alamb · 2024-11-07T18:06:35Z

Looks like this branch has some conflicts now to resolve but then I think it will be ready to go

- cover expression known not to be null - cover NULL pattern - cover repeated '%%' in pattern

findepi · 2024-11-08T14:58:00Z

The conflicts are because #13288 is now merged. Let me rebase.

findepi · 2024-11-08T14:59:05Z

(just rebased)

findepi · 2024-11-08T15:22:47Z

Applied comments and fixed handling of implicit escape.

alamb

🚀

alamb · 2024-11-08T18:45:22Z

Thanks @findepi @adriangb and @goldmedal for this PR and the reviews

…ant (apache#13260) * Expand LIKE simplification - cover expression known not to be null - cover NULL pattern - cover repeated '%%' in pattern * Simplify `EXPR LIKE 'constant'` to `expr = 'constant'` * Correctness and style fixes * fix typo --------- Co-authored-by: Adrian Garcia Badaracco <[email protected]>

github-actions bot added optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt) labels Nov 5, 2024

findepi force-pushed the findepi/expand-like-simplification-e96eca branch from c6285bf to be26107 Compare November 5, 2024 13:39

findepi mentioned this pull request Nov 5, 2024

Incorrect LIKE and ILIKE result for NULL input, `%', and '%%' pattern #12637

Closed

findepi force-pushed the findepi/expand-like-simplification-e96eca branch from 88d6df1 to 6c57af7 Compare November 5, 2024 15:14

findepi marked this pull request as draft November 5, 2024 15:21

goldmedal reviewed Nov 5, 2024

View reviewed changes

datafusion/optimizer/Cargo.toml Outdated Show resolved Hide resolved

adriangb mentioned this pull request Nov 5, 2024

Simplify EXPR LIKE 'constant' to expr = 'constant' #13061

Closed

goldmedal mentioned this pull request Nov 6, 2024

Fix incorrect ... LIKE '%' simplification with NULLs #13259

Merged

findepi force-pushed the findepi/expand-like-simplification-e96eca branch from e4eae46 to 520ad2b Compare November 6, 2024 07:31

findepi marked this pull request as ready for review November 6, 2024 07:31

alamb reviewed Nov 6, 2024

View reviewed changes

alamb changed the title ~~Expand LIKE simplification~~ Expand LIKE simplification: cover NULL pattern/expression and constant Nov 6, 2024

goldmedal reviewed Nov 6, 2024

View reviewed changes

datafusion/optimizer/src/simplify_expressions/expr_simplifier.rs Show resolved Hide resolved

datafusion/sqllogictest/test_files/string/string_view.slt Outdated Show resolved Hide resolved

alamb approved these changes Nov 6, 2024

View reviewed changes

findepi commented Nov 7, 2024

View reviewed changes

datafusion/sqllogictest/test_files/string/string_query.slt.part Outdated Show resolved Hide resolved

findepi mentioned this pull request Nov 7, 2024

Test LIKE with implicit \ escape #13288

Merged

goldmedal reviewed Nov 7, 2024

View reviewed changes

datafusion/sqllogictest/test_files/string/string_query.slt.part Outdated Show resolved Hide resolved

findepi and others added 2 commits November 8, 2024 15:57

Expand LIKE simplification

96a6cd2

- cover expression known not to be null - cover NULL pattern - cover repeated '%%' in pattern

Simplify EXPR LIKE 'constant' to expr = 'constant'

5a03823

findepi force-pushed the findepi/expand-like-simplification-e96eca branch from e05417a to 5a03823 Compare November 8, 2024 14:58

Correctness and style fixes

0517b5d

fix typo

2585591

alamb approved these changes Nov 8, 2024

View reviewed changes

alamb merged commit 667b302 into apache:main Nov 8, 2024

findepi deleted the findepi/expand-like-simplification-e96eca branch November 8, 2024 18:54

findepi mentioned this pull request Nov 8, 2024

chore: remove unnecessary test helpers #13317

Merged

alamb mentioned this pull request Nov 20, 2024

Nov 20. 2024: This week in DataFusion #13503

Closed

3 tasks

ngli-me mentioned this pull request Feb 19, 2025

Simplify EXPR LIKE 'constant' to expr = 'constant' #13192

Closed

Comments

Conversation

findepi commented Nov 5, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

findepi commented Nov 5, 2024

Uh oh!

findepi commented Nov 5, 2024

Uh oh!

findepi commented Nov 5, 2024

Uh oh!

adriangb commented Nov 5, 2024

Uh oh!

findepi commented Nov 5, 2024

Uh oh!

goldmedal left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

goldmedal commented Nov 6, 2024

Uh oh!

findepi commented Nov 6, 2024

Uh oh!

alamb left a comment

Choose a reason for hiding this comment

Uh oh!

alamb Nov 6, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

crepererum commented Nov 6, 2024

Uh oh!

alamb left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

findepi commented Nov 7, 2024

Uh oh!

Uh oh!

alamb commented Nov 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

findepi commented Nov 8, 2024

Uh oh!

findepi commented Nov 8, 2024

Uh oh!

findepi commented Nov 8, 2024

Uh oh!

alamb left a comment

Choose a reason for hiding this comment

Uh oh!

alamb commented Nov 8, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

findepi commented Nov 5, 2024 •

edited

Loading

alamb commented Nov 7, 2024 •

edited

Loading