-
-
Notifications
You must be signed in to change notification settings - Fork 750
Closed
Labels
Description
A few mechanics in worker.py are very well self-contained, and it would make a lot of sense to encapsulate them into a separate module/class in order to reduce complexity:
- memory limit and thresholds
- initialisation of the SpillBuffer
- memory_monitor(), which handles pause, unpause, and force evict from spillbuffer
From the Worker's perspective, Worker.data should be treated as an opaque MutableMapping.
This is just a refactoring, with no functional impact.
Consider deprecating init arguments to Worker using target, spill, pause threshold etc. in favour of dask.config.
This change is part of the wider epic
Related issues: