Skip to content

Assertion for SpillBuffer breaks usage of custom Worker(data=...) types #5909

@pentschev

Description

@pentschev

#5814 introduced https://github.com/dask/distributed/blob/main/distributed/worker.py#L3884 which I believe unintentionally breaks the flexibility of passing a custom type via Worker(data=...). In Dask-CUDA we use the data= kwarg to pass a specialized handler for spilling GPU memory, and it currently breaks one of our tests, see here.

Would it be possible to remove that assertion or make it somehow flexible so that a custom type wouldn't fail?

cc @crusaderky who's the author.

cc @quasiben @madsbk

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions