Repository navigation
Commit 234c2ce
feat: release the output of a materialized CTE when the last scan finishes
The buffered output of a materialized CTE (the in-memory batches, their memory reservations and the spill files) was held until the `TaskContext` of the run and all replay streams were dropped.
The output of a run now counts its readers.
It is released when each partition of each bound scan was executed in the run, and each of these executions has finished or was dropped.
The count is of the partitions that can still be executed, so the output is not released between two scans that are executed one after the other.
A scan partition that is never executed keeps the output until the run ends, as before.
A scan partition that is executed after the release computes the body again, so it never reads an empty output.
A second execution of a plan with the same `TaskContext` is such a case.
`RecursiveQueryExec` executes its recursive term one time for each iteration.
It holds the outputs of the CTEs that the recursive term reads and does not declare, so that they are computed one time for the whole query.
Closes apache#26233.
Co-Authored-By: Claude Opus 5.5 <[email protected]>1 parent 71f28c3 commit 234c2ce
3 files changed
Lines changed: 542 additions & 54 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
593 | 593 | | |
594 | 594 | | |
595 | 595 | | |
| 596 | + | |
| 597 | + | |
| 598 | + | |
| 599 | + | |
| 600 | + | |
| 601 | + | |
| 602 | + | |
| 603 | + | |
| 604 | + | |
| 605 | + | |
| 606 | + | |
| 607 | + | |
| 608 | + | |
| 609 | + | |
| 610 | + | |
| 611 | + | |
| 612 | + | |
| 613 | + | |
| 614 | + | |
| 615 | + | |
| 616 | + | |
| 617 | + | |
| 618 | + | |
| 619 | + | |
| 620 | + | |
| 621 | + | |
| 622 | + | |
| 623 | + | |
| 624 | + | |
| 625 | + | |
| 626 | + | |
| 627 | + | |
| 628 | + | |
| 629 | + | |
| 630 | + | |
| 631 | + | |
| 632 | + | |
| 633 | + | |
0 commit comments