-
Notifications
You must be signed in to change notification settings - Fork 8.3k
Cache for primary index #71992
Copy link
Copy link
Closed
Labels
Description
Use case
In some extremely large workloads (trillions of rows) primary index may take significant amount of memory: tens or hundreds of GBs. Instead we can use cache for primary similar to mark cache instead of always storing index for parts.
Describe the solution you'd like
Use a cache similar to mark cache. If lazy load and primary index cache are enabled store index in that cache instead of storing it in data part.
Describe alternatives you've considered
Lazy load of index, but the problem is that index is never evicted and once primary index was analyzed for some query the index will be stored in memory until restart or SYSTEM UNLOAD PRIMARY KEY query.
Reactions are currently unavailable