Repository navigation
Commit df9ec8f
committed
[opt](build) Prepare cutting the RLE/FMT_COMPILE instantiation amplifiers
Pure-additive / behavior-preserving preparation so that two kinds of
template-instantiation edges can be cut next: both parquet decoder.h
headers' util/rle_encoding.h include (their inline BaseDictDecoder bodies
force ~530 TUs to instantiate the RleBatchDecoder<uint32_t> ->
GetLiteralValues -> UnpackBatch -> UnpackValues chain at ~0.43 CPU s per
TU, 231 CPU s total) and the FMT_COMPILE formatter instantiations that
uint24.h / large_int_value.h push into ~1150 TUs (53.5 CPU s). No include
edge is removed here:
- format/parquet/decoder.h: move set_data, skip_values and the defaulted
~BaseDictDecoder out of line into decoder.cpp; everything moved is a
per-page/per-batch virtual call that is dispatched through the vtable at
every call site already, so outlining changes no generated call. With the
dtor out of line the unique_ptr<RleBatchDecoder<uint32_t>> member no
longer needs the complete type in the header; forward-declare it
- format_v2/parquet/reader/native/decoder.h: same treatment for set_data,
decode_dictionary_indices, decode_selected_dictionary_indices,
_decode_fragmented_selection, _decode_and_validate_skipped,
_decode_dictionary_values and the dtor; the pure orchestrator bodies
(skip_values, decode_dictionary_values, decode_selected_dictionary_values)
stay inline as they only call the now-declared members
- decoder.cpp (both trees): include util/rle_encoding.h and
common/cast_set.h directly (cast_set previously rode in through
rle_encoding.h)
- core/uint24.h: move to_string (the FMT_COMPILE("{:04d}-{:02d}-{:02d}")
date formatter, the single biggest fmt instantiation in the codebase)
into a new core/uint24.cpp; a std::string-returning formatter is
allocation-dominated, so the outlined call is noise
- core/value/large_int_value.h: move to_buffer/to_string x2 (the
FMT_COMPILE int128 formatters) into large_int_value.cpp, which now
includes fmt/compile.h + fmt/format.h directly instead of riding its own
header
The RleBatchDecoder methods stay fully inline in rle_encoding.h for the
real decode TUs; the per-value-hot LevelDecoder::get_next path is
deliberately untouched.1 parent 13fca3d commit df9ec8f
8 files changed
Lines changed: 320 additions & 235 deletions
File tree
- be/src
- core
- value
- format_v2/parquet/reader/native
- format/parquet
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
128 | 128 | | |
129 | 129 | | |
130 | 130 | | |
131 | | - | |
132 | | - | |
133 | | - | |
134 | | - | |
135 | | - | |
136 | | - | |
137 | | - | |
138 | | - | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
139 | 134 | | |
140 | 135 | | |
141 | 136 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
20 | 23 | | |
21 | 24 | | |
22 | 25 | | |
23 | 26 | | |
24 | 27 | | |
25 | 28 | | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
26 | 41 | | |
27 | 42 | | |
28 | 43 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
34 | | - | |
35 | | - | |
36 | | - | |
37 | | - | |
38 | | - | |
39 | | - | |
40 | | - | |
41 | | - | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
42 | 40 | | |
43 | 41 | | |
44 | 42 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
| 23 | + | |
23 | 24 | | |
24 | 25 | | |
25 | 26 | | |
| |||
28 | 29 | | |
29 | 30 | | |
30 | 31 | | |
| 32 | + | |
31 | 33 | | |
32 | 34 | | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
33 | 54 | | |
34 | 55 | | |
35 | 56 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
45 | 45 | | |
46 | 46 | | |
47 | 47 | | |
| 48 | + | |
| 49 | + | |
48 | 50 | | |
49 | 51 | | |
50 | 52 | | |
| |||
94 | 96 | | |
95 | 97 | | |
96 | 98 | | |
97 | | - | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
98 | 103 | | |
99 | 104 | | |
100 | | - | |
101 | | - | |
102 | | - | |
103 | | - | |
104 | | - | |
105 | | - | |
106 | | - | |
107 | | - | |
108 | | - | |
| 105 | + | |
109 | 106 | | |
110 | 107 | | |
111 | 108 | | |
| |||
146 | 143 | | |
147 | 144 | | |
148 | 145 | | |
149 | | - | |
150 | | - | |
151 | | - | |
152 | | - | |
153 | | - | |
| 146 | + | |
154 | 147 | | |
155 | 148 | | |
156 | 149 | | |
| |||
0 commit comments