Skip to content

Don't let asyncio.run() shut down the DBOS executor - #925

Merged
kraftp merged 3 commits into
dbos-inc:mainfrom
parbhatkapila4:fix/asyncio-run-shuts-down-executor
Oct 27, 2026
Merged

kraftp merged 3 commits into
dbos-inc:mainfrom
parbhatkapila4:fix/asyncio-run-shuts-down-executor

Conversation

@parbhatkapila4

Copy link
Copy Markdown
Contributor

Fixes #924

Description of the fix
Calling a DBOS async API from an event loop that DBOS doesn't own, like asyncio.run(), shuts down the DBOS executor when that loop closes. After that, every start_workflow and queue dispatch raised "cannot schedule new futures after shutdown" until the process restarted.

Implementation
_configure_asyncio_thread_pool() sets the DBOS executor itself as the running loop's default executor, and asyncio.run() calls shutdown() on that executor on exit. Now the loop gets a small ThreadPoolExecutor subclass that submits to the DBOS executor and ignores shutdown(). It has to be a subclass because set_default_executor() only accepts a ThreadPoolExecutor. It's created once per executor and reused, and DBOS.destroy() still shuts down the real executor like before.

Testing

  • Added test_asyncio_run_keeps_dbos_executor to test_async.py. It calls a DBOS async API through asyncio.run(), then starts a sync workflow and runs an async workflow that uses asyncio.to_thread. On main, it fails with "cannot schedule new futures after shutdown". With this change, it passes on Postgres and SQLite.
  • test_async, test_async_workflow_management, test_client, test_streaming, test_concurrency_primitives, test_dbos, test_queue and test_scheduler pass on Postgres 16 (417 passed, 1 skipped).
  • black, isort, and mypy are clean, and pyright shows nothing new.

@kraftp kraftp left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good, thanks for the fix!

@parbhatkapila4

Copy link
Copy Markdown
Contributor Author

The 3.14 / Postgres failure is test_cancel_resume_queue (assert 3 == 2), which looks flaky on main. I ran it 40 times on Python 3.14 against Postgres: without this change, it failed once with the same assertion; with this change, 0 of 40.

Could you re-run that job?

@kraftp
kraftp merged commit 718d066 into dbos-inc:main Oct 27, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

asyncio.run() of any DBOS async API shuts down the DBOS executor

2 participants