Skip to content

New caching strategies #34651

Merged
kssenii merged 30 commits intoClickHouse:masterfrom
alexX512:master
Aug 12, 2022
Merged

New caching strategies #34651
kssenii merged 30 commits intoClickHouse:masterfrom
alexX512:master

Conversation

@alexX512
Copy link
Contributor

@alexX512 alexX512 commented Feb 16, 2022

Changelog category:

  • New Feature

Changelog entry:
Add SLRU cache policy for uncompressed cache and marks cache.

@robot-clickhouse robot-clickhouse added doc-alert pr-feature Pull request with new product feature labels Feb 16, 2022
@CLAassistant
Copy link

CLAassistant commented Feb 16, 2022

CLA assistant check
All committers have signed the CLA.

@kssenii kssenii self-assigned this Feb 16, 2022
@kssenii kssenii added the can be tested Allows running workflows for external contributors label Feb 16, 2022
@robot-ch-test-poll2 robot-ch-test-poll2 added the submodule changed At least one submodule changed in this PR. label Jun 14, 2022
@robot-clickhouse robot-clickhouse removed the submodule changed At least one submodule changed in this PR. label Jun 14, 2022
@alexX512 alexX512 changed the title WIP New caching strategies New caching strategies Jul 16, 2022
@alexX512 alexX512 force-pushed the master branch 2 times, most recently from d6ee34a to 3b93067 Compare August 7, 2022 17:36
@alexX512
Copy link
Contributor Author

The following figures are a comparison of Segmented LRU and basic LRU caching strategies. The main difference between these strategies is that SLRU save frequent elements in a special place and they don't remove them from a cache after full scanning of a table.

For tests were used queries from this benchmark https://clickhouse.com/docs/en/getting-started/example-datasets/star-schema/. The first 5 queries on the graphic are Q1.1 with different years, the next 5 queries are exactly the same, so there is a 1.0 hitrate (proportion of reading data from the cache to all reading data). After there is a Q2.1 query that scans all table and we can see that hitrate decrease there. After we run 10 Q1.1 queries in the same way as in the previous one. We can see that for SLRU hitrate increase faster than for LRU. Also if reading from disk was a bottleneck part then the graphic of time for these queries looks like on the second picture, thereafter a big scan we can see that LRU works slower than SLRU for some time.

In other cases, SLRU and LRU show exactly similar results as in the first half of graphics.

image

@kssenii kssenii merged commit a6cfc7b into ClickHouse:master Aug 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

can be tested Allows running workflows for external contributors pr-feature Pull request with new product feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants