Skip to content

Support conditions in JOIN ON section#24420

Merged
abyss7 merged 27 commits intoClickHouse:masterfrom
vdimir:issue-21868
Jul 21, 2021
Merged

Support conditions in JOIN ON section#24420
abyss7 merged 27 commits intoClickHouse:masterfrom
vdimir:issue-21868

Conversation

@vdimir
Copy link
Copy Markdown
Member

@vdimir vdimir commented May 22, 2021

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Changelog category (leave one):

  • New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Detailed description / Documentation draft:

Conditions for left or right table columns can be specified in JOIN ON section as sequence of expression connected with AND, in addition to main equality. Only rows where conditions hold will be joined.

Current status:

  • Add more tests
  • Support MergeJoin
  • Ensure that performance not affected, waiting for perftests in CI [current tests passed, maybe need to write more specific?]

@robot-clickhouse robot-clickhouse added doc-alert pr-feature Pull request with new product feature labels May 22, 2021
@vdimir vdimir changed the title [WIP] Try to enforce table identification in CollectJoinOnKeysMatcher [WIP] Support filtering conditions in JOIN ON May 23, 2021
@vdimir vdimir marked this pull request as ready for review May 23, 2021 14:18
@vdimir vdimir force-pushed the issue-21868 branch 2 times, most recently from 815785a to 26a9778 Compare May 24, 2021 09:01
@vdimir vdimir linked an issue May 24, 2021 that may be closed by this pull request
@vdimir vdimir marked this pull request as draft May 31, 2021 09:50
@vdimir vdimir force-pushed the issue-21868 branch 3 times, most recently from 98bbd41 to a352a2d Compare June 4, 2021 09:18
@vdimir vdimir changed the title [WIP] Support filtering conditions in JOIN ON Support filtering conditions in JOIN ON Jun 11, 2021
@vdimir vdimir marked this pull request as ready for review June 11, 2021 10:49
@vdimir vdimir changed the title Support filtering conditions in JOIN ON Support conditions in JOIN ON section Jun 13, 2021
@abyss7 abyss7 self-assigned this Jun 17, 2021
@PHaroZ
Copy link
Copy Markdown

PHaroZ commented Jun 22, 2021

many thanks for your work !

Copy link
Copy Markdown
Contributor

@abyss7 abyss7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should add some tests

if (table_numbers.first == JoinIdentifierPos::Unknown)
throw Exception("Ambiguous column in expression '" + queryToString(ast) + "' in JOIN ON section",
ErrorCodes::AMBIGUOUS_COLUMN_NAME);
data.analyzed_join.addJoinCondition(ast, isLeftIdentifier(table_numbers.first));
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[self-note] Putting ASTFunction into on_filter_condition_asts_*


String deriveTempName(const String & name)
{
return "--" + name;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why such name?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some internal table name needed, I haven't found good solution for it. This column added at the beginning of function joinBlock and removed at the end, so it's not visible outside. But we haven't some mechanism for generating temporary names that also cannot be already presented in block.
Do you have any ideas how to solve it?

@vdimir vdimir requested a review from abyss7 June 30, 2021 14:48
@vdimir vdimir mentioned this pull request Jul 2, 2021
@vdimir
Copy link
Copy Markdown
Member Author

vdimir commented Jul 16, 2021

Merged with master.

@abyss7 take a look, please, review comments are resolved

@abyss7 abyss7 merged commit 576b407 into ClickHouse:master Jul 21, 2021
@vdimir vdimir deleted the issue-21868 branch July 22, 2021 07:33
@sevirov
Copy link
Copy Markdown
Contributor

sevirov commented Jul 22, 2021

Internal documentation ticket: DOCSUP-12064

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-feature Pull request with new product feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

JOIN on constant fails with 21.5.5.12 - worked with 21.4.7.3 Support filters in JOIN ON section

5 participants