Repository navigation
Commit 83cd46f
committed
fix: count each memory pool once in analyze_trace
analyze_trace derived the total pool reservation by summing the
per-thread thread_NNN_comet_memory_reserved counters. Those come from
total_reserved_for_thread, which deduplicates shared pools only within a
thread, so a task-shared pool reports its full reservation on every
thread that references it and summing multiplies it by the thread count.
Measured on TPC-H SF100 with tracing enabled, the reported total reached
500x the real allocation, 27% of samples claimed more memory reserved
than the allocator had handed out, and the median total was 0 because
threads with no registered pools contribute nothing. The tool's peak gap
then landed on samples where the total happened to be zero, reporting
peak allocation rather than a gap.
Add total_reserved_across_tasks(), which sums reserved() over the
distinct entries of TASK_SHARED_MEMORY_POOLS, and emit it as a single
comet_memory_reserved_total counter beside the existing allocation
counter so both halves of a sample are true at the same instant.
analyze_trace prefers it, and still reads older traces from the
per-thread sum while saying plainly that those totals over-count.
The new counter has to be matched before the existing
name.contains("comet_memory_reserved") branch, which would otherwise
fold the process-wide total into the per-thread map.
The per-thread counters are unchanged: they remain useful for per-thread
attribution in a trace viewer.1 parent 20faa23 commit 83cd46f
4 files changed
Lines changed: 136 additions & 17 deletions
File tree
- docs/source/contributor-guide
- native
- common/src/bin
- core/src/execution
- memory_pools
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
78 | 78 | | |
79 | 79 | | |
80 | 80 | | |
81 | | - | |
82 | | - | |
83 | | - | |
84 | | - | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
85 | 88 | | |
86 | 89 | | |
87 | 90 | | |
| |||
121 | 124 | | |
122 | 125 | | |
123 | 126 | | |
124 | | - | |
125 | | - | |
126 | | - | |
127 | | - | |
128 | | - | |
129 | | - | |
130 | | - | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
37 | 37 | | |
38 | 38 | | |
39 | 39 | | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
40 | 48 | | |
41 | 49 | | |
42 | 50 | | |
| |||
75 | 83 | | |
76 | 84 | | |
77 | 85 | | |
78 | | - | |
| 86 | + | |
| 87 | + | |
79 | 88 | | |
| 89 | + | |
| 90 | + | |
80 | 91 | | |
81 | 92 | | |
82 | 93 | | |
| |||
146 | 157 | | |
147 | 158 | | |
148 | 159 | | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
149 | 166 | | |
150 | 167 | | |
151 | 168 | | |
| |||
161 | 178 | | |
162 | 179 | | |
163 | 180 | | |
164 | | - | |
| 181 | + | |
165 | 182 | | |
166 | 183 | | |
167 | 184 | | |
168 | 185 | | |
169 | | - | |
| 186 | + | |
| 187 | + | |
170 | 188 | | |
171 | 189 | | |
172 | 190 | | |
| |||
198 | 216 | | |
199 | 217 | | |
200 | 218 | | |
201 | | - | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
202 | 232 | | |
203 | 233 | | |
204 | 234 | | |
| |||
210 | 240 | | |
211 | 241 | | |
212 | 242 | | |
213 | | - | |
| 243 | + | |
214 | 244 | | |
215 | 245 | | |
216 | 246 | | |
| |||
245 | 275 | | |
246 | 276 | | |
247 | 277 | | |
248 | | - | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
249 | 282 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1107 | 1107 | | |
1108 | 1108 | | |
1109 | 1109 | | |
| 1110 | + | |
| 1111 | + | |
| 1112 | + | |
| 1113 | + | |
| 1114 | + | |
| 1115 | + | |
| 1116 | + | |
1110 | 1117 | | |
1111 | 1118 | | |
1112 | 1119 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
94 | 94 | | |
95 | 95 | | |
96 | 96 | | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
97 | 113 | | |
98 | 114 | | |
99 | 115 | | |
| |||
121 | 137 | | |
122 | 138 | | |
123 | 139 | | |
| 140 | + | |
124 | 141 | | |
125 | 142 | | |
126 | 143 | | |
| |||
133 | 150 | | |
134 | 151 | | |
135 | 152 | | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
136 | 211 | | |
137 | 212 | | |
138 | 213 | | |
| |||
0 commit comments