We are currently intending to turn on scheduler-side queuing by default. This will give much better cluster memory use in most cases, and help users avoid the painful "running out of memory on simple workloads" experience #2602.

It does slow things down, but the increased stability and better out-of-the-box user experience seems worthwhile.

Specifically, this will mean changing the default worker-saturation value from inf to 1.0 or 1.1 or something. Advanced users who need to keep old behavior can get it back by changing this config, which will be documented #7203.

TODO:

• Transition queued->memory causes AssertionError #7200
• decide_worker_rootish_queuing_disabled assertion fails when retiring worker #7063
• AssertionError: waiting on released dep when worker-saturation < .inf #7204
• Queued tasks could deadlock in combination with secede #7223
• maybe Only use opened queue slots on running workers #7190
• Polish parsing of worker-saturation from config #7255
• Avoid expensive occupancy calculation when unused #7257
• decide_worker could be expensive on large clusters with queuing enabled #7246
• Un-skip test_nested_compute #7247
• Queuing docs #7203
• Queue by default #7279