Hand cluster shards off one at a time on shutdown - #8941

Summary

When Sharding shuts down, it now tries to hand each shard to the remaining runners before the final releaseAll, instead of leaving every lock to releaseAll and, if that never runs, to lease expiry. There's no new public API: disposing the Sharding layer, for example on SIGTERM, starts the handoff.

• No new shards. Once shutdown starts, the runner stops acquiring shards and doesn't register again. Registration and unregistration are serialized, so an in-flight registration can't advertise a runner that is shutting down.
• Per-shard release. Each held shard is released through the normal per-shard path, after its entities and singletons have stopped.
• Best effort. The wait is bounded by shardLockInterval. If it times out, releaseAll attempts to release the remaining locks. Neither step is guaranteed to finish before the process exits, so terminationGracePeriodSeconds should leave room for both.

Two related fixes:

• Interrupting a shard no longer waits forever on an entity whose id is also active on another shard.
• Singletons that are still stopping stay visible to shard release. Their teardown is awaited outside the singleton registry lock, so a singleton that registers other singletons can be closed without deadlocking.

Background

This started as a report with failing tests. Kubernetes rollouts send SIGTERM to every old runner at once. When a process died before releaseAll finished, new runners waited out the roughly 35-second lease, and runners that were still exiting picked up shards freed by their peers, only to hand them off again.

The first version proposed a public Sharding.drain for preStop hooks. Review replaced it with the shutdown-only change above. The v3 variant of the original tests is #8946.

Tests

Four regressions in packages/effect/test/cluster/Sharding.test.ts:

• Scope disposal waits for entity teardown, releases the held shard before releaseAll, and does not acquire a peer's newly freed shard.
• A singleton whose registration is already closing keeps its shard locked until teardown finishes; both held shards are released before releaseAll.
• A singleton that registers another singleton can close without deadlocking.
• An entity id active on two shards is interrupted on both.

Validation

After the tests-only reduction:

• Sharding.test.ts: 88/88 with default timeouts.
• pnpm check and pnpm lint pass.
• All four retained tests fail on the code they guard against: three on main before this PR, and the nested-singleton test on the earlier PR implementation that held the registry lock during teardown.

Earlier validation, before this reduction: 217/217 cluster tests and 109/109 platform-node cluster tests. Those broader suites were not rerun for the reduction. SQL integration suites and Bun/Deno runner tests were not run.

Closes EFF-1876