Skip to content

Release the ydoc subscription in YRoom.stop() - #149

Closed
Thanatos9404 wants to merge 1 commit into
y-crdt:mainfrom
Thanatos9404:fix/release-ydoc-subscription
Closed

Thanatos9404 wants to merge 1 commit into
y-crdt:mainfrom
Thanatos9404:fix/release-ydoc-subscription

Conversation

@Thanatos9404

Copy link
Copy Markdown
Contributor

Fixes #146

YRoom.stop() unobserves the ydoc but keeps the Subscription object in self._subscription, so it lives as long as the room. A pycrdt subscription can't be dropped in another thread than the one that created it, and a stopped room is in practice freed by the cyclic garbage collector, which runs in whatever thread triggers it. When that is a worker thread, freeing the room gives RuntimeError: pycrdt::subscription::Subscription is unsendable, but is being dropped on another thread.

This sets self._subscription = None in stop() right after unobserving, so the subscription is freed there, in the event loop thread, and the room can later be freed in any thread.

Why not only unobserve the awareness callback (fix A in the issue)

I tried that first. The awareness cycle is real, but it isn't the only thing keeping a stopped room alive: the cancellation exceptions of the room's tasks keep frames that refer to the room in reference cycles. On asyncio I traced it to anyio's task_done callback (anyio 4.15.1), which keeps the CancelledError in a local variable while the exception's traceback holds that same frame; on Python 3.13, trio has a similar chain through its cancel frames. So with fix A alone the room can still only go away in a GC pass, and the subscription goes with it: on Python 3.13 the test below still fails with fix A alone on asyncio and trio (on 3.14 on asyncio), while on 3.10 fix A alone is enough. Dropping _subscription in stop() fixes it on all three, with or without fix A. Also, in pycrdt 0.14 Awareness is pure Python and doesn't hold a Subscription; the one in the error is the room's ydoc subscription.

Fix A as written in the issue would also break restarting a room with start() after stop() (the lower-level API from the YRoom docstring, which works today): the second run no longer broadcasts the server awareness, and the second stop() raises a KeyError. If you'd like stop() to remove the awareness callback too, it needs to be observed again in start(). Happy to add that here or in a follow-up.

Test

test_yroom_garbage_collected_in_another_thread disables the GC, starts and stops a room, drops it, runs gc.collect() in a separate thread, and checks that the room was freed and that nothing reached sys.unraisablehook. On main it fails on asyncio and trio with the error from the issue. It is skipped on PyPy, which doesn't free objects as soon as their last reference goes away, so the test's premise doesn't hold there.

Run locally on Windows 11:

  • pytest -v --timeout=60: 56 passed on Python 3.10 and 3.13 (editable install) and 3.14 (pip install --group test . as in CI). With main's yroom.py, the 2 new test cases fail (on 3.10, 3.13 and 3.14) and the other 54 pass (full runs on 3.10 and 3.13).
  • mypy src tests (Python 3.14, CI-style install): no issues.
  • pre-commit run --files src/pycrdt/websocket/yroom.py tests/test_yroom.py --hook-stage=manual: passed.

I couldn't run it on PyPy, since pycrdt 0.14 has no PyPy wheels and building it was out of reach here.

`stop()` unobserved the ydoc but kept the `Subscription` object in
`_subscription`. A stopped room is usually freed by the cyclic garbage
collector rather than by reference counting: the room observes its own
awareness with a bound method, and the cancellation exceptions of its
tasks keep their frames, which refer to the room, in reference cycles.
The collector runs in whatever thread triggers it, and a pycrdt
subscription cannot be dropped in another thread than the one that
created it, so freeing the room from a worker thread raised
"Subscription is unsendable, but is being dropped on another thread".

`stop()` now also drops `_subscription`, so the subscription is freed in
the thread that stops the room, whenever and wherever the room itself is
freed.

Fixes y-crdt#146
@davidbrochart

Copy link
Copy Markdown
Contributor

Thanks for the PR @Thanatos9404.
I think y-crdt/pycrdt#446 will fix this issue.

@Thanatos9404

Copy link
Copy Markdown
Contributor Author

Thanks @davidbrochart, fixing it in pycrdt is better: it also covers subscriptions this PR can't reach, such as the one Doc.events() keeps for the provider. I'll close this once #446 is merged, unless you'd like to keep the stop() change as well.

@davidbrochart

Copy link
Copy Markdown
Contributor

unless you'd like to keep the stop() change as well.

I don't think this changes anything, since garbage collection is not deterministic anyway (we don't know when/where the destructor will effectively be called).

@Thanatos9404

Copy link
Copy Markdown
Contributor Author

Fair enough. For what it's worth, the stop() change didn't depend on the collector: dropping the last reference frees the subscription right away, in the thread that calls stop(), which is why the new test passed every time. But y-crdt/pycrdt#446 fixes it for every subscription, so I'm closing this in favour of it. Thanks for looking!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RuntimeError: Subscription unsendable when dropped on wrong thread — caused by reference cycle in YRoom that forces GC-based reclamation

2 participants