Repository navigation
Commit 73c481f
authored
[feature](cloud) Support file cache write index only (apache#64995)
### Purpose
This PR adds a new BE config, `enable_file_cache_write_index_file_only`, to make cloud rowset writes and compaction outputs write only index-related content into file cache. This avoids actively putting segment data into file cache during write, reducing cache pollution from large data pages.
The main target scenario is a limited local cache where query performance should prioritize indexes and segment metadata. With this mode enabled, the write path protects inverted index files and segment index/footer ranges, while normal data pages are still loaded and cached by the query read path when needed.
### High-Level Changes
- Add BE config `enable_file_cache_write_index_file_only`, default `false`.
- Make rowset/segment file-cache write decisions file-type aware:
- `SEGMENT_FILE` does not create a full-file cache builder and does not allow adaptive write in index-only mode.
- `INVERTED_INDEX_FILE` is still written into file cache through the direct write path.
- segment index/footer ranges are preloaded into file cache by `SegmentIndexFileCacheLoader` after segment close.
- Update packed file handling to honor the original file type when deciding whether to write file cache.
- Add profile statistics for segment footer/index reads and separate them from data-page and inverted-index reads.
- Replace correlated boolean parameters in `CachedRemoteFileReader::_update_stats` with a read-type enum:
- data page
- inverted index
- segment footer/index
- Add BE UTs and cloud regressions for both normal file and packed file paths.
### Behavior
When `enable_file_cache_write_index_file_only=true`:
- Segment data is not actively written into file cache during write, flush, or compaction output.
- Segment index/footer ranges are synchronously preloaded into file cache after segment close.
- Independent inverted index files are written into file cache through the direct write path.
- Query-side data-page reads keep the existing behavior. The query read path may still fill cache according to current rules.
- Segment open, footer/index access, and inverted index access can hit the index-related content preloaded by the write path.
- Query profile exposes separate cache/remote read statistics for data pages, inverted indexes, and segment footer/index reads.
### Interaction With Existing Configs
The write-side file-cache behavior is decided in this order:
1. `enable_file_cache=false` is the highest-priority global switch.
- All file-cache writes are disabled.
- This includes independent inverted index direct cache and segment index/footer preload.
2. `enable_file_cache=true` and `enable_file_cache_write_index_file_only=true` enables global forced index-only mode.
- Segment data does not create a full-file cache builder.
- Segment data cannot be written by adaptive write.
- Segment index/footer ranges are written into file cache by `SegmentIndexFileCacheLoader` after segment close.
- Independent inverted index files are written through the `INVERTED_INDEX_FILE` direct write path.
- This config has higher priority than `enable_file_cache_write_base_compaction_index_only` and `enable_file_cache_write_cumu_compaction_index_only`.
- These index writes do not depend on `enable_file_cache_adaptive_write`, compaction keep configs, hit-ratio thresholds, or request/session `write_file_cache`.
3. `enable_file_cache=true`, `enable_file_cache_write_index_file_only=false`, and a compaction-level index-only config is enabled applies index-only behavior only to the matching compaction output.
- `enable_file_cache_write_base_compaction_index_only=true` only forces base compaction output to use index-only file-cache writes.
- `enable_file_cache_write_cumu_compaction_index_only=true` only forces cumulative compaction output to use index-only file-cache writes.
- These two configs do not affect load, normal flush, schema change output, or query read-path cache fill.
- Compaction types not matched by these configs continue to use existing `write_file_cache`, adaptive write, compaction keep, and hit-ratio decisions.
4. `enable_file_cache=true` and all three index-only configs are `false` preserves existing behavior.
- `write_file_cache`, `enable_file_cache_adaptive_write`, compaction keep configs, schema-change hit ratios, and existing rowset/file-writer decisions continue to control whether cache is written.
#### With `enable_file_cache_adaptive_write`
- When index-only is disabled, `enable_file_cache_adaptive_write=true` keeps the existing adaptive write behavior. Segment data may be written into file cache if the cache-space check passes.
- When index-only is enabled, adaptive write no longer participates in segment data decisions.
- Even if `enable_file_cache_adaptive_write=true`, segment data is still not written into file cache, while segment index/footer and independent inverted index files are still written.
#### With Compaction Output Keep Configs
Related configs include:
- `enable_file_cache_keep_base_compaction_output`
- `file_cache_keep_base_compaction_output_min_hit_ratio`
- compaction output hit-ratio checks
When index-only is disabled, existing compaction output cache behavior is preserved.
When index-only is enabled, compaction output is forced into index-only behavior:
- Compaction output segment data is not actively written into file cache.
- Segment index/footer ranges are synchronously written into file cache.
- Independent inverted index files are direct-cached.
- Compaction keep configs and hit-ratio thresholds no longer decide whether index content is written into file cache.
#### With Base/Cumulative Compaction Index-Only Configs
Related configs include:
- `enable_file_cache_write_base_compaction_index_only`
- `enable_file_cache_write_cumu_compaction_index_only`
These two configs are narrower compaction-output policies and only take effect when `enable_file_cache_write_index_file_only=false`.
- When `enable_file_cache_write_base_compaction_index_only=true`, base compaction output uses index-only behavior: output segment data is not actively cached, while output segment index/footer ranges and independent inverted index files are written into file cache.
- When `enable_file_cache_write_cumu_compaction_index_only=true`, cumulative compaction output uses index-only behavior: output segment data is not actively cached, while output segment index/footer ranges and independent inverted index files are written into file cache.
- If only one of the two configs is enabled, the other compaction type keeps the existing cache-write decision flow.
- If `enable_file_cache_write_index_file_only=true`, the global index-only config takes precedence and the base/cumulative compaction-specific configs no longer change the final behavior.
- `enable_file_cache=false` is still the highest-priority switch and disables all of these file-cache writes.
#### With Schema Change Output Thresholds
- When index-only is disabled, `file_cache_keep_schema_change_output_min_hit_ratio` keeps its existing semantics.
- When index-only is enabled, schema change output also follows index-only behavior:
- segment data is not actively written into file cache;
- index-related content is written into file cache;
- schema-change hit-ratio thresholds no longer decide whether index content is cached.
#### With `disable_file_cache` / Request `write_file_cache`
- `enable_file_cache=false` disables everything and has the highest priority.
- When index-only is enabled, the BE config has higher priority than request/session `write_file_cache=false`.
- Even if session `disable_file_cache=true` makes request-side `write_file_cache=false`, as long as BE `enable_file_cache=true` and index-only is enabled:
- index-related content is still written into file cache;
- segment data is still not written into file cache.
- When index-only is disabled, `disable_file_cache=true` preserves existing behavior and load output does not actively write file cache.
#### With Packed File
- When index-only is disabled, packed small files keep the existing whole-file cache behavior.
- When index-only is enabled:
- packed small segment files must not be whole-file cached, preventing segment data from entering file cache;
- segment index/footer ranges are written into file cache by `SegmentIndexFileCacheLoader`;
- packed independent index small files are independent index files and can be whole-file cached.
#### With `enable_flush_file_cache_async`
- Segment index/footer ranges are written through synchronous dry-run reads and do not depend on write-path `enable_flush_file_cache_async`.
- Independent inverted index files still use the direct write path, so they continue to follow existing async flush behavior.
#### With `index_percent` in `file_cache_path`
`SegmentIndexFileCacheLoader` marks index-related reads as index data so that they go to the `FileCacheType::INDEX` queue when possible.
After enabling index-only mode, `index_percent` should be reviewed based on index size. The default `index_percent=5` may be insufficient for large indexes.
### Behavior Matrix
| Scenario | index-only=false | index-only=true |
| --- | --- | --- |
| load segment data | follows existing cache policy | not actively cached |
| load segment index/footer | depends on existing whole-file cache or read path | synchronously range-read into file cache after segment close |
| load independent inverted index file | follows existing cache policy | direct-cached by write path |
| cumulative compaction segment data | may be cached by existing behavior | not actively cached |
| cumulative compaction segment index/footer | may be cached by existing behavior | synchronously range-read into file cache after segment close |
| base compaction force-keep output | whole output rowset may be cached | only output segment index/footer is preloaded |
| adaptive write has enough space | segment data may be cached | segment data is still not cached |
| packed small index file | may be cached | can be whole-file cached |
| packed small segment file | may be cached | not whole-file cached; internal index/footer is preloaded |
| query read fill cache | unchanged | unchanged |
| event-driven warm-up | unchanged | unchanged |
### Release note
Add BE config `enable_file_cache_write_index_file_only` to support index-only file-cache writes for cloud rowset writes and compaction outputs.1 parent 4f8c144 commit 73c481f
31 files changed
Lines changed: 2333 additions & 70 deletions
File tree
- be
- src
- cloud
- io
- cache
- fs
- storage
- compaction
- rowset
- segment
- test
- cloud
- storage
- compaction
- regression-test/suites/cloud_p0/cache/write_index_only
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
131 | 131 | | |
132 | 132 | | |
133 | 133 | | |
| 134 | + | |
134 | 135 | | |
135 | 136 | | |
136 | 137 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
175 | 175 | | |
176 | 176 | | |
177 | 177 | | |
| 178 | + | |
178 | 179 | | |
179 | 180 | | |
180 | 181 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
98 | 98 | | |
99 | 99 | | |
100 | 100 | | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
101 | 111 | | |
102 | 112 | | |
103 | 113 | | |
| |||
156 | 166 | | |
157 | 167 | | |
158 | 168 | | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
159 | 188 | | |
160 | 189 | | |
161 | 190 | | |
| |||
193 | 222 | | |
194 | 223 | | |
195 | 224 | | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
196 | 244 | | |
197 | 245 | | |
198 | 246 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
98 | 98 | | |
99 | 99 | | |
100 | 100 | | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
101 | 111 | | |
102 | 112 | | |
103 | 113 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
321 | 321 | | |
322 | 322 | | |
323 | 323 | | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
324 | 329 | | |
325 | | - | |
| 330 | + | |
326 | 331 | | |
327 | 332 | | |
328 | 333 | | |
329 | 334 | | |
330 | | - | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
331 | 341 | | |
332 | 342 | | |
333 | 343 | | |
| |||
609 | 619 | | |
610 | 620 | | |
611 | 621 | | |
612 | | - | |
| 622 | + | |
613 | 623 | | |
614 | 624 | | |
615 | 625 | | |
| |||
639 | 649 | | |
640 | 650 | | |
641 | 651 | | |
642 | | - | |
| 652 | + | |
| 653 | + | |
| 654 | + | |
| 655 | + | |
| 656 | + | |
643 | 657 | | |
644 | | - | |
645 | | - | |
| 658 | + | |
| 659 | + | |
646 | 660 | | |
647 | 661 | | |
648 | | - | |
649 | | - | |
650 | | - | |
| 662 | + | |
| 663 | + | |
| 664 | + | |
651 | 665 | | |
652 | 666 | | |
653 | | - | |
654 | | - | |
655 | | - | |
| 667 | + | |
| 668 | + | |
| 669 | + | |
656 | 670 | | |
657 | | - | |
| 671 | + | |
| 672 | + | |
| 673 | + | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
| 679 | + | |
| 680 | + | |
| 681 | + | |
| 682 | + | |
| 683 | + | |
| 684 | + | |
| 685 | + | |
| 686 | + | |
| 687 | + | |
| 688 | + | |
| 689 | + | |
| 690 | + | |
| 691 | + | |
| 692 | + | |
| 693 | + | |
| 694 | + | |
| 695 | + | |
| 696 | + | |
| 697 | + | |
658 | 698 | | |
659 | 699 | | |
660 | 700 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
75 | 75 | | |
76 | 76 | | |
77 | 77 | | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
78 | 84 | | |
79 | 85 | | |
80 | 86 | | |
| |||
83 | 89 | | |
84 | 90 | | |
85 | 91 | | |
86 | | - | |
| 92 | + | |
87 | 93 | | |
88 | 94 | | |
89 | 95 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
| 24 | + | |
24 | 25 | | |
25 | 26 | | |
26 | 27 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
| 47 | + | |
47 | 48 | | |
48 | 49 | | |
49 | 50 | | |
| |||
109 | 110 | | |
110 | 111 | | |
111 | 112 | | |
112 | | - | |
| 113 | + | |
| 114 | + | |
113 | 115 | | |
114 | 116 | | |
115 | 117 | | |
116 | 118 | | |
117 | 119 | | |
118 | 120 | | |
| 121 | + | |
119 | 122 | | |
120 | 123 | | |
121 | 124 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
| 23 | + | |
23 | 24 | | |
24 | 25 | | |
25 | 26 | | |
| |||
96 | 97 | | |
97 | 98 | | |
98 | 99 | | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
99 | 105 | | |
100 | | - | |
| 106 | + | |
101 | 107 | | |
102 | 108 | | |
103 | 109 | | |
| |||
106 | 112 | | |
107 | 113 | | |
108 | 114 | | |
109 | | - | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
110 | 124 | | |
111 | | - | |
| 125 | + | |
112 | 126 | | |
113 | | - | |
| 127 | + | |
114 | 128 | | |
115 | 129 | | |
116 | 130 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
| 38 | + | |
38 | 39 | | |
39 | 40 | | |
40 | 41 | | |
| |||
195 | 196 | | |
196 | 197 | | |
197 | 198 | | |
| 199 | + | |
198 | 200 | | |
199 | 201 | | |
200 | 202 | | |
| |||
0 commit comments