[RFC] Fix data skipping indexes for columns with additional actions during merge#11162
Conversation
azat
changed the title
Affected versions: 19.17.9.60+
…merge For example for AggregatingMergeTree if the column in the data skipping index is under some aggregate function, then data skipping index will be incorrect. Fix this by calculating data skipping indexes after aggregation has been done.
|
Rebased, previous HEAD - 7d196d08e4af015664675beac6d2f6714db673a5 (contains tests) |
azat
force-pushed
the
data-skip-index-merging-params-fix
branch
from
7d196d0 to
c36708c
Compare
|
@abyss7 No work done on review, unassigned. |
|
Ok. I've edited the changelog entry to make it more comprehensible (so the user can understand the scope of this issue): From:
To:
|
I always forget making changelog entry "user friendly", thanks! |
[RFC] Fix data skipping indexes for columns with additional actions during merge (cherry picked from commit 8accde7)
[RFC] Fix data skipping indexes for columns with additional actions during merge (cherry picked from commit 8accde7)
[RFC] Fix data skipping indexes for columns with additional actions during merge (cherry picked from commit 8accde7)
…arams-fix" This reverts commit 459559b.
…arams-fix" This reverts commit d3b2c50.
|
Reverted fix from 20.1 and 20.3 because it cause exception |
|
@KochetovNicolai Should we also revert from 20.4 and master? |
|
As I see from tests, it works for 20.4 and master. |
|
This PR should not be backported without #11259 |
…arams-fix" This reverts commit d7ba41d.
|
I've decided to not backport this pr at all. |
5 participants
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
If data skipping index is dependent on columns that are going to be modified during background merge (for SummingMergeTree, AggregatingMergeTree as well as for TTL GROUP BY), it was calculated incorrectly. This issue is fixed by moving index calculation after merge so the index is calculated on merged data.
Detailed description / Documentation draft:
For example for AggregatingMergeTree if the column in the data skipping
index is under some aggregate function, then data skipping index will be
incorrect.
Fix this by calculating data skipping indexes after aggregation has been
done.
P.S. looks like all versions affected