Skip to content

Commit d6d4be6

Browse files
committed
docs: say a fanout partition stays in FanoutPartitions, not open, until the task ends
The comment on `held_bytes` still said every partition stays open, which stopped being true once partitions could close early. What makes the held-back rows share one limit is that every partition, and its feed, stays in `partitions`.
1 parent 9270e17 commit d6d4be6

1 file changed

Lines changed: 4 additions & 4 deletions

File tree

‎native/core/src/execution/operators/iceberg_write.rs‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1128,10 +1128,10 @@ struct FanoutPartitions {
11281128
/// Where each partition value sits in `partitions`.
11291129
index: HashMap<IcebergStruct, usize>,
11301130
/// Bytes the feeds hold back for their dictionary choice, across all partitions. Every
1131-
/// partition stays open, so the rows they hold back share one limit: once they reach it, every
1132-
/// partition still holding makes its choice from the rows it has. A task fanning out to many
1133-
/// partitions that each get less than a page would otherwise hold all of its rows back,
1134-
/// uncompressed, until it closed.
1131+
/// partition stays in `partitions` until the task ends, so the rows they hold back share one
1132+
/// limit: once they reach it, every partition still holding makes its choice from the rows it
1133+
/// has. A task fanning out to many partitions that each get less than a page would otherwise
1134+
/// hold all of its rows back, uncompressed, until it closed.
11351135
///
11361136
/// Always the sum of the feeds' [`PartitionFeed::held_bytes`], kept in step wherever a feed
11371137
/// changes what it holds back: a push, the release once the limit is reached, and an early

0 commit comments

Comments
 (0)