Try fix pk in tuple performance#12062
Merged
CurtizJ merged 6 commits intoClickHouse:masterfrom Jul 2, 2020
nvartolomei:nv/set-index-tuple-types
Merged
Try fix pk in tuple performance#12062CurtizJ merged 6 commits intoClickHouse:masterfrom nvartolomei:nv/set-index-tuple-types
CurtizJ merged 6 commits intoClickHouse:masterfrom
nvartolomei:nv/set-index-tuple-types
Conversation
Possible approach for fixing #10574 The problem is that prepared sets are built correctly, it is a hash map of key -> set where key is a hash of AST and list of data types (when we a list of tuples of literals). However, when the key is built from the index to try and find if there exists a prepared set that would match it looks for data types of the primary key (see how data_types is populated) because the primary key has only one field (v in my example) it can not find the prepared set. The patch looks for any prepared indexes where data types match for the subset of fields found in primary key, we are not interested in other fields anyway for the purpose of primary key pruning.
Member
|
Also you can use setting |
Contributor
|
Let's try to backport it since it's a regression when updating from pre 19.1 |
filimonov
added
the
pr-must-backport
This was referenced Jul 27, 2020
robot-clickhouse
pushed a commit
that referenced
this pull request
Jul 27, 2020
robot-clickhouse
pushed a commit
that referenced
this pull request
Jul 27, 2020
This was referenced Jul 27, 2020
robot-clickhouse
pushed a commit
that referenced
this pull request
Jul 27, 2020
CurtizJ
added a commit
that referenced
this pull request
Jul 27, 2020
Backport #12062 to 20.3: Try fix pk in tuple performance
CurtizJ
added a commit
that referenced
this pull request
Jul 27, 2020
Backport #12062 to 20.5: Try fix pk in tuple performance
CurtizJ
added a commit
that referenced
this pull request
Jul 27, 2020
Backport #12062 to 20.4: Try fix pk in tuple performance
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Fix "#10574 Index not used for IN operator with literals", performance regression introduced around v19.3.
Detailed description / Documentation draft:
Fixes #10574.
The problem is that prepared sets are built correctly, it is a hash map of key -> set
where key is a hash of AST and list of data types (when we a list of
tuples of literals).
However, when the key is built from the index to try and find if there
exists a prepared set that would match it looks for data types of the
primary key (see how data_types is populated) because the primary key
has only one field (v in my example) it can not find the prepared set.
The patch looks for any prepared indexes where data types match for the
subset of fields found in primary key, we are not interested in other
fields anyway for the purpose of primary key pruning.