Gracefully drain environments before shutdown or update #35233

Summary

This discussion asks for a direction decision. Before a server shutdown or self-update, should T3 gracefully drain environments — finish or park in-flight turns safely, run configured settle/save behavior, then exit — instead of cutting agents mid-turn?

Desktop updates and restarts are common. #32038 was a prior attempt with detailed review on gate lifetime and client compatibility.

Why drain

Abrupt process death loses in-flight work and confuses remote clients that still think the environment is live. A drain path makes updates boring: live turns get a chance to park or save, then the process exits cleanly.

The valuable part is live sessions, not prompting every idle thread to “save work” on a routine desktop update (that turns one update into N agent turns). Admission to drain must be atomic so concurrent callers do not clobber each other’s shutdown scope, and a stuck drain needs a timeout / fail-open story.

Proposal

• Add a drain gate before shutdown/self-update: stop admitting new work, let in-flight turns complete or park, then exit.
• Target live sessions by default; make broad save-work prompts opt-in.
• Keep progress reporting backwards-compatible for older clients (do not break decode of update progress).
• Ship server drain first; desktop update UX can follow.

Reference implementation

#32038 is a prior attempt covering drain admission, prepared-run / queue interaction, and draining progress. Linked as evidence only; review notes there are the checklist for a smaller follow-up.

Question for maintainers

Yes, no, or not now: graceful drain before shutdown/self-update (default for live sessions, opt-in for idle save-work).