Dedicated Mark/Uncompressed cache for skip indices#27961
Dedicated Mark/Uncompressed cache for skip indices#27961kitaisreal merged 2 commits intoClickHouse:masterfrom
Conversation
robot-clickhouse
added
the
pr-improvement
|
|
| if (auto index_uncompressed_cache = getContext()->getIndexUncompressedCache()) | ||
| { | ||
| new_values["IndexUncompressedCacheBytes"] = index_uncompressed_cache->weight(); | ||
| new_values["IndexUncompressedCacheCells"] = index_uncompressed_cache->count(); |
There was a problem hiding this comment.
So there are different metrics for bytes/cells/files but same metric for cache misses/hits.
I think that it worth adding new metrics for hits/misses for these separate caches.
@amosbird what do you think?
There was a problem hiding this comment.
@azat I plan to test this pull request and add them. Also enable them by default.
There was a problem hiding this comment.
@azat Yes, it'll be useful. @kitaisreal Great!
|
@amosbird BTW have you tested this on a real workload? Maybe you can share some numbers then? |
We saw some improvement but it's not substantial because the related workload always requests realtime data and the cache hit rate is poor. It might be useful to fill the cache during insert/merge. |
|
|
||
| /// Size of cache for index marks (index of MergeTree skip indices). It is necessary. | ||
| /// Specify default value for index_mark_cache_size explicitly! | ||
| size_t index_mark_cache_size = config().getUInt64("index_mark_cache_size", 0); |
There was a problem hiding this comment.
This major performance degradation went unnoticed for two years.
There was a problem hiding this comment.
IIRC, before this PR, there is no mark/uncompressed cache for secondary indices at all. I don't think it's a degradation.
6 participants
I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Add dedicated Mark/Uncompressed cache for skip indices. Not sure how to test it properly.
Detailed description / Documentation draft:
.