Repository navigation
Commit b172a36
committed
[CoreML] Cache the quantizer's pattern graphs instead of rebuilding them per model
The CoreML quantizer's annotation builds its subgraph-match patterns by exporting
small pattern modules, and it rebuilds every one of them for each model it
annotates. A single prepare_pt2e performs 405 pattern builds, and those builds are
99.1% of its runtime:
prepare_pt2e total : 28.51s
pattern graph builds : 405
time in them : 28.24s (99.1% of prepare_pt2e)
mean per build : 0.070s
The patterns do not depend on the model being annotated. Instrumenting the getters
with their actual arguments over three different models shows 1215 builds with only
405 distinct argument signatures, and the key set for a linear model is identical to
the key set for a conv model. So every model after the first rebuilds, from scratch,
patterns that are byte-for-byte the ones already built.
Nothing amortized it. Five consecutive prepare_pt2e calls in one process, each on a
fresh module, cost 28.94s, 32.24s, 34.23s, 35.40s, 32.16s.
The five getters take only hashable arguments and build their example_inputs
internally at fixed shapes, so each is a pure function of its arguments and can be
memoized.
The cached graph cannot be handed out directly. SubgraphMatcherWithNameNodeMap
splits name_node_map out of the pattern's return tuple in place, so a second
consumer of the same object fails with "Expecting the pattern graph to return a
tuple". Callers therefore get a copy, which is far cheaper than rebuilding: about
0.7 ms against 62 ms per pattern.
Measured with this change, prepare_pt2e plus convert_pt2e on an M4, alternating
conv and linear modules so the cache is exercised across model types:
case 1 (conv2d): 29.92s <- cold, builds the 405 patterns once
case 2 (conv2d): 0.53s
case 3 (linear): 0.54s
case 4 (conv2d): 0.55s
case 5 (linear): 0.54s
55x on warm calls. The quantized output is unchanged: the fully converted graphs
for a conv module and a linear module are byte identical before and after, both in
node targets and in printed code.
Reported in #21623, where the coreml_static_int8 operators suite runs to the 180
minute CI cap and is cancelled. Per-case quantize there drops from roughly 29s to
0.5s with one warmup per worker process.1 parent 900b9fd commit b172a36
1 file changed
Lines changed: 34 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
| 6 | + | |
| 7 | + | |
6 | 8 | | |
7 | 9 | | |
8 | 10 | | |
| |||
283 | 285 | | |
284 | 286 | | |
285 | 287 | | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
286 | 316 | | |
287 | 317 | | |
288 | 318 | | |
| |||
322 | 352 | | |
323 | 353 | | |
324 | 354 | | |
| 355 | + | |
325 | 356 | | |
326 | 357 | | |
327 | 358 | | |
| |||
347 | 378 | | |
348 | 379 | | |
349 | 380 | | |
| 381 | + | |
350 | 382 | | |
351 | 383 | | |
352 | 384 | | |
| |||
376 | 408 | | |
377 | 409 | | |
378 | 410 | | |
| 411 | + | |
379 | 412 | | |
380 | 413 | | |
381 | 414 | | |
| |||
395 | 428 | | |
396 | 429 | | |
397 | 430 | | |
| 431 | + | |
398 | 432 | | |
399 | 433 | | |
400 | 434 | | |
| |||
0 commit comments