Skip to content

Commit ba7c892

Browse files
authored
fix: let a spilled final aggregate read its spill files back past its memory share (apache#6544)
* fix: let a spilled final aggregate read its spill files back past its memory share DataFusion 55's FinalHashAggregateStream replays its merged spill files through a stream that can't spill, so a refused memory request there fails the task. The merge's read buffers are a sibling reservation of the same consumer and take as many spill files as fit, so the replay often finds the consumer's share already taken (apache#6254). Both Comet pools now record that request as overcommit instead of refusing it. They recognize it as a request from a FinalHashAggregateStream consumer while another of its reservations holds memory, which in DataFusion 55.1 happens only during the replay. The replay emits its finished groups after every batch, so the overcommit stays around one batch of groups, and releases repay it first. Refusals while the aggregate reads its input, and while the merge picks its files, are unchanged. Remove this once Comet's DataFusion includes apache/datafusion#25383. * fix: let an ordered final aggregate read its spill files back too DataFusion runs a final aggregate whose input is sorted on some of its grouping keys as an OrderedFinalAggregateStream. It merges and replays its spill files the same way FinalHashAggregateStream does, so its replay failed the task the same way. Treat its consumer as a final aggregate too. Explain in spill_replay.rs why recording the replay's request is safe: the replay asks for memory only after it has aggregated a batch, so the memory already exists, as it does for a grow. Describe the exception in the memory management guide, which said the fair pool always refuses a request that fails its local checks. * docs: tighten the spill replay comments and guide The fair pool is the only one with a share and a pool total to skip, so say so. The greedy pool takes its tracking lock for other consumers when Spark refuses them, not never. Point whoever removes the workaround at the tests that show whether the replay still needs it. * test: share the spill replay tests between the pools fair_pool.rs and unified_pool.rs each had a copy of the same two spill replay scenarios. Run them once, in spill_replay.rs, against both pool types as createPlan builds them, which also checks that the wrappers pass register through to the greedy pool's tracking. Keep a greedy pool test for its per-consumer map, and drop a test import that the pool's own imports now cover. Share the final aggregate spill checks between the two apache#6254 tests in CometAggregateSuite, and use checkCometAnswer, which collects once and labels the Comet answer as Comet's. * refactor: move the spill replay workaround into a pool wrapper The workaround for apache#6254 lived in both Comet pools. The fair pool sent its three refusals through a check, and the greedy pool tracked each final aggregate across its reservations. apache#5613 reworks the fair pool's try_grow, and apache#6583 has to take the workaround out again, so both would have had to rework the pools. SpillReplayPool, in spill_replay.rs, now wraps the tracked Comet pool instead. It keeps a total for each final aggregate's consumer. When the pool refuses a request from one while another of its reservations holds memory, it calls the pool's grow, which skips the fair pool's limits and carries what Spark doesn't grant as overcommit. fair_pool.rs and unified_pool.rs are back to main's versions, and overcommit() looks through the wrapper. The Rust tests run against both pool types as createPlan builds them. They now reach the fair pool's pool-limit refusal too, and check that a failed Spark call during the replay is returned rather than recorded. The guide adds the wrapper to the pool stack and keeps a short section. The DataFusion 55.1 facts that an upgrade has to re-check stay in spill_replay.rs, which names apache#6583 next to the removal condition.
1 parent bd7cfcb commit ba7c892

4 files changed

Lines changed: 490 additions & 9 deletions

File tree

‎docs/source/contributor-guide/memory_management.md‎

Lines changed: 20 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -253,14 +253,17 @@ ignored and the pool is always `UnboundedMemoryPool`.
253253
the inside out, a Comet plan in the default configuration sees:
254254

255255
```text
256-
[LoggingMemoryPool] <- only when spark.comet.debug.memory=true
257-
[TaskSharedMemoryPool] <- RAII handle for the per-task registry
258-
[TrackConsumersPool] <- DataFusion; names the top 10 consumers in error messages
259-
[CometFairMemoryPool] <- delegates acquire/release to Spark over JNI
256+
[LoggingMemoryPool] <- only when spark.comet.debug.memory=true
257+
[TaskSharedMemoryPool] <- RAII handle for the per-task registry
258+
[SpillReplayPool] <- lets a spilled final aggregate read its spill files back
259+
[TrackConsumersPool] <- DataFusion; names the top 10 consumers in error messages
260+
[CometFairMemoryPool] <- delegates acquire/release to Spark over JNI
260261
```
261262

262263
Each decorator forwards every `MemoryPool` method to its inner pool, so `reserved()` at any level
263-
reports the base pool's number.
264+
reports the base pool's number. `SpillReplayPool` also turns one kind of refused `try_grow` into a
265+
`grow`; see
266+
[Final aggregates reading their spill files back](#final-aggregates-reading-their-spill-files-back).
264267

265268
### The unified pools
266269

@@ -346,6 +349,18 @@ hands every native plan in a task the same one and drops it when the task comple
346349
covers the whole task, so `CometExecIterator.close()` warns about memory still in use only when
347350
the task's last open native plan closes.
348351

352+
### Final aggregates reading their spill files back
353+
354+
`SpillReplayPool` (`spill_replay.rs`) wraps both Comet pools to work around
355+
[issue #6254](https://github.com/apache/datafusion-comet/issues/6254). Once one of DataFusion 55's
356+
final aggregates has spilled, it reads its spill files back through an aggregate that cannot spill,
357+
so a refused `try_grow` there fails the task. When the pool refuses such a request,
358+
`SpillReplayPool` records it with the pool's `grow` instead, which skips `CometFairMemoryPool`'s
359+
local checks and carries what Spark does not grant as overcommit. Every other refusal is passed on
360+
unchanged. `spill_replay.rs` describes how the wrapper recognizes these requests and what that relies
361+
on in DataFusion. [Issue #6583](https://github.com/apache/datafusion-comet/issues/6583) tracks
362+
removing it.
363+
349364
## How DataFusion consumes the pool
350365

351366
Native operators reserve through DataFusion's `MemoryConsumer` / `MemoryReservation` API:

‎native/core/src/execution/memory_pools/mod.rs‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,13 +20,15 @@ mod fair_pool;
2020
pub mod logging_pool;
2121
mod plan_pool;
2222
mod spark_memory;
23+
mod spill_replay;
2324
mod task_shared;
2425
mod unified_pool;
2526

2627
use datafusion::execution::memory_pool::{MemoryPool, TrackConsumersPool, UnboundedMemoryPool};
2728
use fair_pool::CometFairMemoryPool;
2829
use jni::objects::{Global, JObject};
2930
use spark_memory::SparkMemory;
31+
use spill_replay::SpillReplayPool;
3032
use std::num::NonZeroUsize;
3133
use std::sync::Arc;
3234
use unified_pool::CometUnifiedMemoryPool;
@@ -70,10 +72,16 @@ fn create_pool(
7072

7173
match pool_type {
7274
MemoryPoolType::GreedyUnified => acquire_task_shared_pool(task_attempt_id, || {
73-
tracked(CometUnifiedMemoryPool::with_spark(spark()))
75+
Arc::new(SpillReplayPool::new(
76+
task_attempt_id,
77+
tracked(CometUnifiedMemoryPool::with_spark(spark())),
78+
))
7479
}),
7580
MemoryPoolType::FairUnified => acquire_task_shared_pool(task_attempt_id, || {
76-
tracked(CometFairMemoryPool::with_spark(spark(), pool_size))
81+
Arc::new(SpillReplayPool::new(
82+
task_attempt_id,
83+
tracked(CometFairMemoryPool::with_spark(spark(), pool_size)),
84+
))
7785
}),
7886
MemoryPoolType::Unbounded => Arc::new(UnboundedMemoryPool::default()),
7987
}
@@ -85,6 +93,7 @@ fn create_pool(
8593
/// takes nothing from Spark or that the function did not create.
8694
pub(crate) fn overcommit(pool: &Arc<dyn MemoryPool>) -> usize {
8795
let pool = task_shared::unwrap_task_shared(pool).unwrap_or(pool);
96+
let pool = spill_replay::unwrap_spill_replay(pool).unwrap_or(pool);
8897
if let Some(tracked) = pool.downcast_ref::<TrackConsumersPool<CometUnifiedMemoryPool>>() {
8998
tracked.inner().overcommit()
9099
} else if let Some(tracked) = pool.downcast_ref::<TrackConsumersPool<CometFairMemoryPool>>() {

0 commit comments

Comments
 (0)