forked from slingdata-io/sling-cli
-
Notifications
You must be signed in to change notification settings - Fork 0
additional empty chunk file #60
Copy link
Copy link
Closed
Description
The issue is that when the last row of the data hits exactly the row limit (100 rows / 10 = exactly 10 files), nextPipe() is called which creates a new empty reader. Then the loop ends, but that empty reader was already sent to the channel.
The file_max_rows: 10 is working now (10 rows per file × 10 files = 100 rows), but there's an extra empty file part.01.0011.csv with only the header. This is a bug. The empty 11th file is created because:
Row 100 is written to file 10
br.Counter >= rowLimit (10 >= 10) is true
nextPipe() is called, creating file 11 with just a header
The loop ends (no more rows)
File 11 remains with only the header
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels