Skip to content

Commit 853d1db

Browse files
Dandandanclaude
andcommitted
Add doc comments to PushDecoderStreamState and transition method
Co-Authored-By: Claude Opus 4.6 <[email protected]>
1 parent 870c72a commit 853d1db

1 file changed

Lines changed: 14 additions & 0 deletions

File tree

‎datafusion/datasource-parquet/src/opener.rs‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -680,6 +680,12 @@ impl FileOpener for ParquetOpener {
680680
}
681681
}
682682

683+
/// State for a stream that decodes a single Parquet file using a push-based decoder.
684+
///
685+
/// The [`transition`](Self::transition) method drives the decoder in a loop: it requests
686+
/// byte ranges from the [`AsyncFileReader`], pushes the fetched data into the
687+
/// [`ParquetPushDecoder`], and yields projected [`RecordBatch`]es until the file is
688+
/// fully consumed.
683689
struct PushDecoderStreamState {
684690
decoder: ParquetPushDecoder,
685691
reader: Box<dyn AsyncFileReader>,
@@ -692,6 +698,14 @@ struct PushDecoderStreamState {
692698
}
693699

694700
impl PushDecoderStreamState {
701+
/// Advances the decoder state machine until the next [`RecordBatch`] is
702+
/// produced, the file is fully consumed, or an error occurs.
703+
///
704+
/// On each iteration the decoder is polled via [`ParquetPushDecoder::try_decode`]:
705+
/// - [`NeedsData`](DecodeResult::NeedsData) – the requested byte ranges are
706+
/// fetched from the [`AsyncFileReader`] and fed back into the decoder.
707+
/// - [`Data`](DecodeResult::Data) – a decoded batch is projected and returned.
708+
/// - [`Finished`](DecodeResult::Finished) – signals end-of-stream (`None`).
695709
async fn transition(&mut self) -> Option<Result<RecordBatch>> {
696710
loop {
697711
match self.decoder.try_decode() {

0 commit comments

Comments
 (0)