-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Closed
Labels
bugSomething isn't workingSomething isn't working
Description
Describe the bug
DataFrame::write_csv/json/parquet isn't cancelling safe due to spawned tasks. See these lines: https://github.com/apache/arrow-datafusion/blob/master/datafusion/core/src/physical_plan/file_format/csv.rs#L300-L307
It's actually bad because impossible to wrap such functions to timeout or select! for example.
To Reproduce
Wrap it to select or timeout and check that spawned tasks are still working after cancelling by another branch.
Expected behavior
It should be safe-to-cancel these methods.
Additional context
Possible ways to fix:
- Use stop/cancel token/channel to stop spawned task on drop of outside variable. Like these: CancellationToken(with drop-guard) or stop-token
- Wrap spawned tasks(i.e
JoinHandle) to future withDropand callabortfor them.
Not sure if there are any other spawns which should be covered as well.
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working