Skip to content

Invalid TaskState released #6191

@mrocklin

Description

@mrocklin

https://github.com/dask/distributed/runs/6148662781?check_suite_focus=true

Traceback (most recent call last):
  File "/Users/runner/work/distributed/distributed/distributed/worker.py", line 3991, in validate_task
    self.validate_task_released(ts)
  File "/Users/runner/work/distributed/distributed/distributed/worker.py", line 3962, in validate_task_released
    assert not any(ts.key in has_what for has_what in self.has_what.values())
AssertionError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/runner/work/distributed/distributed/distributed/core.py", line 625, in handle_stream
    handler(**merge(extra, msg))
  File "/Users/runner/work/distributed/distributed/distributed/worker.py", line 1727, in handle_free_keys
    self.transitions(recommendations, stimulus_id=stimulus_id)
  File "/Users/runner/work/distributed/distributed/distributed/worker.py", line 2635, in transitions
    self.validate_task(ts)
  File "/Users/runner/work/distributed/distributed/distributed/worker.py", line 3999, in validate_task
    raise AssertionError(
AssertionError: Invalid TaskState encountered for <TaskState 'ndarray-c380d2dc1cdf941cb55bb0ea78c929d7' released>.
Story:
[('ndarray-c380d2dc1cdf941cb55bb0ea78c929d7', 'ensure-task-exists', 'released', 'compute-task-1650839033.713147', 1650839033.725312)]

This came out of test_stress_scatter_death

cc @fjetter @crusaderky @gjoseph92

Metadata

Metadata

Assignees

No one assigned

    Labels

    deadlockThe cluster appears to not make any progress

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions