Non replicated merge tree deduplication#22514
Conversation
robot-clickhouse
added
doc-alert
pr-feature
|
Need to try enabled setting by default |
| /// Preserves order using linked list and remove elements | ||
| /// on overflow in FIFO order. | ||
| template <typename V> | ||
| class LimitedOrderedHashMap |
There was a problem hiding this comment.
But it's not the last recently used map, it's the last recently added map.
|
At least sever didn't crash. I'll check failures. |
|
Support of deduplication for MergeTree in old syntax is quite complicated because we need to serialize part name, which depends on storage partition key, which depends on metadata snapshot. So I'll just ban it. |
|
Ok, everything looks expected. Integration s3 merge tree test depends on the order. Func tests just insert equal data. It's especially not obvious for |
|
I'll disable the setting by default and ban for old-syntax merge tree. |
|
need to support alter setting |
|
Let's wait for performance, but I'm almost sure it's not affected. |
|
Yes, no related changes in performance. |
|
Internal documentation ticket: DOCSUP-8347. |
4 participants
I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Add new setting
non_replicated_deduplication_windowfor non-replicated MergeTree inserts deduplication.