-
-
Notifications
You must be signed in to change notification settings - Fork 3
Description
We've been doing some work on high level graphs (HLG) to improve scheduler efficiency. This includes major changes relatively low down in the collections' handling of graphs, optimization, and the scheduler. All of the changes that we're making are designed to be backwards compatible for users, but still in the last release we ran into two issues. One of these we caught before release (thanks to @sofroniewn / Napari folks) and one we found afterwards (thanks to @bnaul).
These were both rapidly fixed, but highlight that we're likely to cause some downstream pain during this process, despite our best efforts. There are several more PRs waiting to be merged here, each of which might cause similar amounts of churn.
We spoke about this briefly in the maintenance meeting today and considered two options:
- Pausing on releases for a few weeks while we settle HLG things
- Continuing HLG work into a separate development branch and merging it in afterwards
Folks decided to go for option 1, which means that we're planning to release the fix for the str_graph issue (thanks @mabsbk), let that settle for a day, and then not release again for 2-3 weeks. I'm looking through the dask/distributed issue tracker to make sure that there isn't anything critical while I think @jsignell is doing the same for dask/dask.
This might also be a good time to make the switch to Calver (cc @jcrist )