Skip to content

fix(cron): compute the session-title fallback lazily inside its recovery block - #146940

Open
princejain756 wants to merge 1 commit into
NousResearch:mainfrom
princejain756:fix/cron-title-fallback-lazy
Open

princejain756 wants to merge 1 commit into
NousResearch:mainfrom
princejain756:fix/cron-title-fallback-lazy

Conversation

@princejain756

Copy link
Copy Markdown

Summary

Fixes #146855 - when a cron job finishes, _finalize_cron_session persists a title before tearing the session down. The recovery fallback was built eagerly in the for-loop tuple, outside the inner try: a get_next_title_in_lineage call on an already-wedged _BoundedCronSessionDB raised there, escaped the except handler, and aborted run_job's finally before defer_agent_teardown ran. Per fired job that leaks the ephemeral agent's httpx clients, terminal sandboxes and MCP stdio children (the EMFILE class the block below explicitly guards against), plus a bogus failure notice.

The two fallbacks are now lambdas evaluated inside the per-iteration try, so a raising lineage lookup is caught like any other title failure and falls through to the bare id-stamped title. Finalization can no longer be interrupted by the title path.

Test plan

  • scripts/run_tests.sh tests/cron/test_finalize_cron_session_title.py - new regression test: all title paths + the lineage lookup raise, _finalize_cron_session must complete (proven red on unfixed code - RuntimeError: session finalization disabled escapes; green after)
  • scripts/run_tests.sh tests/cron/ - 1482 pass, 0 fail
  • python scripts/check - 13 checks ok (line-neutral change inside the over-cap file)

…ery block

The lineage fallback in _finalize_cron_session was evaluated eagerly in
the for-loop tuple, outside the inner try. When _BoundedCronSessionDB was
already wedged (e.g. after a title-write timeout), get_next_title_in_lineage
raised there, escaped the except handler, and aborted run_job's finally
before defer_agent_teardown ran — leaking the ephemeral agent's httpx
clients, terminal sandboxes and MCP stdio children per fired job (the
EMFILE class the block below explicitly guards against).

The fallbacks are now lambdas evaluated inside the per-iteration try, so
a raising lineage lookup falls through to the bare id-stamped title and
the rest of finalization always completes.

Fixes NousResearch#146855
@alt-glitch alt-glitch added area/sessions Session lifecycle, resume, persistence, history comp/cron Cron scheduler and job management P2 Medium — degraded but workaround exists type/bug Something isn't working labels Oct 30, 2026
@whyyagswhy

Copy link
Copy Markdown

Verified at head 95afb03d (fetched to refs/review/146940, SHA matches headRefOid 95afb03d7c37834b12e01ace06925e8f284b15f5), in a detached worktree at that head via scripts/run_tests.sh:

  • tests/cron/test_finalize_cron_session_title.py (PR's own test file) — 1 passed, 0 failed
  • Neighbor tests/cron/test_scheduler_session_title_nudge.py — 5 passed, 0 failed
  • ruff check on cron/scheduler.py: clean
  • Gaps: no full-suite run; coverage limited to the two files above.

@wsiczek

wsiczek commented Oct 31, 2026

Copy link
Copy Markdown

Reproduced the bug on main (c419cf7c) before looking at the fix, using your test as the probe — so the test's validity is established independently:

main + only your test file   → 1 failed  (RuntimeError from get_next_title_in_lineage escaping, line 22)
your branch (fix + test)     → 1 passed

Neighbouring cron session/title tests (test_cron_session_workdir.py, test_scheduler_cron_session_isolation.py, test_scheduler_session_title_nudge.py, test_sessiondb_init_hang.py): I get the same set of 10 pre-existing failures on main (all run_job/session-DB-init cases that don't survive a sandboxed worktree), and applying your branch changes nothing in that set. No regression attributable to this PR.

One thing you should know before a maintainer picks this up: it's a duplicate. #147138 (nikolas-sapa) fixes the same three lines. I ran your test against their branch: 10 passed — their fix satisfies your contract too. The two PRs also conflict textually (CONFLICT (content): cron/scheduler.py), so only one can land.

The only behavioural difference I can find: #147138 appends a final cron <job_id> candidate, so the non-blank guarantee from #50535 still holds when both the lineage lookup and the id-stamped write fail. Yours is 3 lines versus their 26 + a helper — if the project prefers the minimal diff, yours is the one; if it prefers the belt-and-braces candidate list, theirs is.

@princejain756

Copy link
Copy Markdown
Author

Thanks @whyyagswhy and @wsiczek for running the repro on main. Good catch on #147138 touching the same lines too. Totally fine with whichever version maintainers prefer.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/sessions Session lifecycle, resume, persistence, history comp/cron Cron scheduler and job management P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Cron session-title fallback runs outside the try block, so a title-write timeout skips agent teardown and leaks the whole ephemeral agent

4 participants