Repository navigation
Commit 8c099dc
perf(session): cut
## Which issue does this PR close?
Adresses: #13814
## Rationale for this change
`datafusion-session` is 1,619 lines of source and spends **18.8s** in
the
frontend during a cold `cargo build -p datafusion` (`cargo build
--timings`) —
11.6ms per line, the worst ratio in the workspace, against
0.09–0.55ms/line for
crates like `datafusion-datasource` or `datafusion-physical-plan`.
`-Zself-profile` puts ~72% of the crate's compile time in
`evaluate_obligation`,
and 98% of that in `Send`/`Sync`.
## What changes are included in this PR?
`TableProvider::{delete_from, update, merge_into}` are one-line
`not_impl_err!`
stubs that nevertheless build a coroutine capturing `Vec<Expr>` / `Expr`
—
proving that coroutine `Send` walks the whole `Expr`/`LogicalPlan` type
graph.
They are now written as the desugaring of `async fn` returning
`ready(..)`, so
no coroutine is created and there is nothing expensive to prove. The
signatures
are exactly what `#[async_trait]` generates — verified against
`-Zunpretty=expanded` output of this crate — so implementors are
unaffected.
I originally converted seven bodies, but measuring each one's marginal
contribution (reverting one at a time; `evaluate_obligation` self time,
all-converted baseline 659ms) showed only the ones taking `Expr` matter:
| method left as `async fn` | trait solving | marginal cost |
|---|---|---|
| `delete_from` | 1.29s | +631ms |
| `update` | 1.30s | +641ms |
| `merge_into` | 1.29s | +631ms |
| `truncate` | 697ms | +38ms |
| `insert_into` | 665ms | ~0 |
## Are these changes tested?
Interleaved A/B of `cargo rustc -p datafusion-session --lib`
```
base: 1.582s 1.576s 1.590s
fix: 0.933s 0.938s 0.932s
```
`evaluate_obligation` drops from 1.28s to 0.646s.
Those are standalone-build numbers. In the feature-unified build this
crate's
frontend is 18.8s rather than ~1.6s (each obligation is several times
dearer
there), so the absolute saving on a real build should be larger — I have
not
measured that directly, since isolating one crate's unit in a full build
is hard
to do without confounding it with machine drift.
## Are there any user-facing changes?
No public signature changes
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 5 <[email protected]>datafusion-session compile time ~1.7x (#24326)1 parent 519687c commit 8c099dc
3 files changed
Lines changed: 43 additions & 15 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
37 | 37 | | |
38 | 38 | | |
39 | 39 | | |
| 40 | + | |
40 | 41 | | |
41 | 42 | | |
42 | 43 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
21 | 22 | | |
22 | 23 | | |
23 | 24 | | |
| |||
27 | 28 | | |
28 | 29 | | |
29 | 30 | | |
| 31 | + | |
30 | 32 | | |
31 | 33 | | |
32 | 34 | | |
| |||
351 | 353 | | |
352 | 354 | | |
353 | 355 | | |
354 | | - | |
355 | | - | |
356 | | - | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
357 | 359 | | |
358 | | - | |
359 | | - | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
360 | 370 | | |
361 | 371 | | |
362 | 372 | | |
363 | 373 | | |
364 | 374 | | |
365 | 375 | | |
366 | | - | |
367 | | - | |
368 | | - | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
369 | 379 | | |
370 | 380 | | |
371 | | - | |
372 | | - | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
373 | 391 | | |
374 | 392 | | |
375 | 393 | | |
| |||
391 | 409 | | |
392 | 410 | | |
393 | 411 | | |
394 | | - | |
395 | | - | |
396 | | - | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
397 | 415 | | |
398 | 416 | | |
399 | 417 | | |
400 | 418 | | |
401 | | - | |
402 | | - | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
| 423 | + | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
403 | 429 | | |
404 | 430 | | |
405 | 431 | | |
| |||
0 commit comments