Repository navigation
perf(mobile): stop replaying stored read state on every resume #9090
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
c43b44c
36204c2
f0cc23b
29be4ea
4eff9e4
eefce94
8244942
c620e9d
fb1ba45
b7f17da
388e627
56a161a
937bc7b
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
Initialize, reconnects and every live-subscription ready now request a snapshot through one loop. A request before a pass starts its read joins that read; a request during the read runs one more pass, so a queued cold-start pass can no longer swallow a later reconnect. Snapshot reads are numbered in start order and only the newest settled read decides whether the last snapshot succeeded, so an older read finishing late cannot clear a newer failure's retry. Drops the live generation and ready-transition checks, accepting an occasional extra snapshot read. Co-authored-by: studio <dc2fc481343672df31a689840c09299972e704dac592e0da6f18486646a7ee3e@buzz.block.builderlab.xyz> Co-Authored-By: Claude Opus 5.5 <[email protected]> Signed-off-by: Tom Brow <[email protected]>
- Loading branch information
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -147,10 +147,14 @@ class ReadStateManager { | |
| Future<void> _saved = Future.value(); | ||
| bool _saveScheduled = false; | ||
| Object? syncError; | ||
| // Whether the latest snapshot read succeeded. A publish that skips its own | ||
| // Whether the newest snapshot read succeeded. A publish that skips its own | ||
| // snapshot clears [syncError] only on this evidence. | ||
| bool _lastSnapshotSucceeded = false; | ||
| int _liveGeneration = 0; | ||
| // Snapshot reads numbered in start order. Only a read newer than every | ||
| // settled one decides [_lastSnapshotSucceeded], so an older read finishing | ||
| // late cannot clear the retry a newer failure needs. | ||
| int _snapshotsStarted = 0; | ||
| int _newestSettledSnapshot = 0; | ||
| final int Function() _now; | ||
| // Override keys already in this device's own slot, carried unchanged. | ||
| // See [isOverrideContext]. | ||
|
|
@@ -160,20 +164,13 @@ class ReadStateManager { | |
| void Function()? _unsubscribeLive; | ||
| bool _initialized = false; | ||
| bool _initialSeedComplete = false; | ||
| // Pending first fetch and subscribe. A reconnect while it runs waits for it | ||
| // so two passes cannot each install a live subscription and leak one. | ||
| Future<void>? _initializing; | ||
| // The reconnect pass running now, and the one waiting to start after it. | ||
| Future<void>? _runningResync; | ||
| Future<void>? _queuedResync; | ||
| // Whether the first fetch and subscribe both succeeded. A connect that | ||
| // lands while they run (the cold-start connect) then has nothing to redo. | ||
| bool _initialSyncSucceeded = false; | ||
|
|
||
| // The current live subscription has reported ready. A connect while it | ||
| // is not yet live is the connection it was waiting for; a connect after | ||
| // it went live is a reconnect. | ||
| bool _liveReady = false; | ||
| // The one sync loop, while it runs. Each pass resubscribes if asked, then | ||
| // reads the snapshot, so two passes never each install a subscription and | ||
| // leak one. A request before a pass starts its read joins that read; a | ||
| // request after runs one more pass. | ||
| Future<void>? _syncing; | ||
| bool _snapshotWanted = false; | ||
| bool _resubscribeWanted = false; | ||
|
|
||
| bool _disposed = false; | ||
| bool _isPublishing = false; | ||
|
|
@@ -227,16 +224,7 @@ class ReadStateManager { | |
| return; | ||
| } | ||
|
|
||
| final initializing = Completer<void>(); | ||
| _initializing = initializing.future; | ||
| try { | ||
| final subscribed = await _startLiveSubscription(); | ||
| final fetched = await _fetchAndMerge(); | ||
| _initialSyncSucceeded = fetched && subscribed; | ||
| } finally { | ||
| _initializing = null; | ||
| initializing.complete(); | ||
| } | ||
| await _requestSync(resubscribe: true); | ||
| if (_localContexts.isNotEmpty || | ||
| _pending.isNotEmpty || | ||
| _carriedOverrides.isNotEmpty) { | ||
|
|
@@ -296,56 +284,54 @@ class ReadStateManager { | |
| await _publish(); | ||
| } | ||
|
|
||
| /// Fetch and resubscribe after a reconnect. Passes run one at a time, so | ||
| /// each replaces the previous live subscription instead of leaking one. | ||
| /// Reconnects while a pass waits to start share that pass. | ||
| Future<void> reinitializeRemote() { | ||
| if (_disposed || !_remoteEnabled || !_initialized) return Future.value(); | ||
| final queued = _queuedResync; | ||
| if (queued != null) return queued; | ||
| final running = _runningResync; | ||
| final initializing = _initializing; | ||
| final reconnected = _liveReady; | ||
| final pass = Completer<void>(); | ||
| _queuedResync = pass.future; | ||
| unawaited(() async { | ||
| try { | ||
| if (running != null) { | ||
| await running; | ||
| } else if (initializing != null) { | ||
| await initializing; | ||
| // Connected while the first sync waited for its connection: it | ||
| // already used this one. | ||
| if (_initialSyncSucceeded && !reconnected) return; | ||
| } | ||
| if (_disposed) return; | ||
| if (identical(_queuedResync, pass.future)) _queuedResync = null; | ||
| _runningResync = pass.future; | ||
| await _resyncRemote(); | ||
| } catch (e) { | ||
| debugPrint('[ReadStateManager] reinitializeRemote failed: $e'); | ||
| } finally { | ||
| if (identical(_queuedResync, pass.future)) _queuedResync = null; | ||
| if (identical(_runningResync, pass.future)) _runningResync = null; | ||
| pass.complete(); | ||
| } | ||
| }()); | ||
| return pass.future; | ||
| } | ||
|
|
||
| Future<void> _resyncRemote() async { | ||
| /// Resubscribe and read the snapshot after a reconnect. | ||
| Future<void> reinitializeRemote() async { | ||
| if (_disposed || !_remoteEnabled || !_initialized) return; | ||
| debugPrint('[ReadStateManager] reinitializeRemote'); | ||
| if (_isPublishing) { | ||
| await _publishCompleter?.future; | ||
| } | ||
| _unsubscribeLive?.call(); | ||
| _unsubscribeLive = null; | ||
| await _startLiveSubscription(); | ||
| await _fetchAndMerge(); | ||
| await _requestSync(resubscribe: true); | ||
| _schedulePublish(); | ||
| _onChanged(); | ||
| } | ||
|
|
||
| /// Asks the sync loop for a snapshot read, after a resubscribe if | ||
| /// [resubscribe]. Completes when the loop has nothing left to do. | ||
| Future<void> _requestSync({bool resubscribe = false}) { | ||
| _snapshotWanted = true; | ||
| if (resubscribe) _resubscribeWanted = true; | ||
| final syncing = _syncing; | ||
| if (syncing != null) return syncing; | ||
| // Set before the loop starts: a subscription can report ready from | ||
| // inside subscribe, and that request must join this loop. | ||
| final done = Completer<void>(); | ||
| _syncing = done.future; | ||
| unawaited(_runSync(done)); | ||
| return done.future; | ||
| } | ||
|
|
||
| Future<void> _runSync(Completer<void> done) async { | ||
| try { | ||
| while ((_snapshotWanted || _resubscribeWanted) && !_disposed) { | ||
| if (_resubscribeWanted) { | ||
| _resubscribeWanted = false; | ||
| if (_isPublishing) await _publishCompleter?.future; | ||
| _unsubscribeLive?.call(); | ||
| _unsubscribeLive = null; | ||
| await _startLiveSubscription(); | ||
| if (_disposed) return; | ||
| } | ||
| _snapshotWanted = false; | ||
| // A failed read is retried by the next publish, which reads the | ||
| // snapshot in full after a failure. | ||
| if (!await _fetchAndMerge()) _schedulePublish(); | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Non-blocking, wording only. The retry chain here is one publish deep. If that publish's snapshot also fails, Related nit: the publish read at
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 Both done in c620e9d4b5. The comment now says a failed read waits for the next trigger (the publish it schedules, or a later ready, resume or reconnect), and |
||
| } | ||
| } catch (e) { | ||
| debugPrint('[ReadStateManager] sync failed: $e'); | ||
| } finally { | ||
| _syncing = null; | ||
| done.complete(); | ||
| } | ||
| } | ||
|
|
||
| void dispose({bool flushPending = true}) { | ||
| if (_disposed) return; | ||
| _disposed = true; | ||
|
|
@@ -382,23 +368,32 @@ class ReadStateManager { | |
| } | ||
|
|
||
| Future<bool> _fetchAndMerge() async { | ||
| final read = ++_snapshotsStarted; | ||
| try { | ||
| // Snapshot transport validates completeness and fences retired sessions. | ||
| final events = await _relaySession!.fetchReadStateSnapshot(); | ||
| if (_disposed || !_servesThisCommunity) return false; | ||
| _lastSnapshotSucceeded = true; | ||
| _settleSnapshot(read, succeeded: true); | ||
| _mergeEvents(events); | ||
| await _persistLocalState(); | ||
| _onChanged(); | ||
| return true; | ||
| } catch (error) { | ||
| _lastSnapshotSucceeded = false; | ||
| syncError = error; | ||
| if (_settleSnapshot(read, succeeded: false)) syncError = error; | ||
| debugPrint('[ReadStateManager] snapshot failed: $error'); | ||
| return false; | ||
| } | ||
| } | ||
|
|
||
| /// Records snapshot read [read]'s outcome unless a newer read already | ||
| /// settled. Returns whether it was recorded. | ||
| bool _settleSnapshot(int read, {required bool succeeded}) { | ||
| if (read < _newestSettledSnapshot) return false; | ||
| _newestSettledSnapshot = read; | ||
| _lastSnapshotSucceeded = succeeded; | ||
| return true; | ||
| } | ||
|
|
||
| void _mergeEvents(List<NostrEvent> events) { | ||
| if (!_servesThisCommunity) return; | ||
| for (final event in events) { | ||
|
|
@@ -468,9 +463,6 @@ class ReadStateManager { | |
|
|
||
| Future<bool> _startLiveSubscription() async { | ||
| try { | ||
| final generation = ++_liveGeneration; | ||
| var returned = false; | ||
| _liveReady = false; | ||
| final unsub = await _relaySession!.subscribeWithStatus( | ||
| NostrFilter( | ||
| kinds: const [EventKind.readState], | ||
|
|
@@ -483,25 +475,17 @@ class ReadStateManager { | |
| limit: 0, | ||
|
brow marked this conversation as resolved.
Resolved
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When initialization starts before WebSocket authentication completes, or EOSE takes over 500 ms, AGENTS.md reference: AGENTS.md:L235-L238 Useful? React with 👍 / 👎. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When the relay sends a retryable AGENTS.md reference: AGENTS.md:L235-L238 Useful? React with 👍 / 👎.
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 Blocker: read marks published while this subscription is CLOSED-and-retrying are never received.
Scenario: the relay closes this REQ with a rate-limit CLOSED, and the phone's socket stays up. During the backoff, the desktop marks a channel read. The retry registers with Fix: follow the existing pattern for
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 [P2] Keep replay-derived timestamp bounds off read-state notifications The retained subscription keeps
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 Fixed in 388e627. |
||
| ), | ||
| _handleIncomingEvent, | ||
| // The snapshot after this call covers marks published before the REQ | ||
| // went live, unless it goes live later: after the 500 ms fallback | ||
| // (slow AUTH on a cold start) or after a transient CLOSED's retry. | ||
| // With limit 0 nothing replays the gap, so read the snapshot then. | ||
| // A repeated ready while already live (a reconnect can send the | ||
| // same REQ twice) leaves no gap, so it reads nothing. | ||
| // With limit 0 nothing replays marks published before the REQ went | ||
| // live, so read the snapshot every time it goes live: its EOSE, which | ||
| // can come after the 500 ms fallback on a slow AUTH, on a reconnect's | ||
| // replay, or after a CLOSED's retry. The sync loop folds a ready during its own subscribe | ||
| // into the read that follows. | ||
| onStatusChanged: (status) { | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 Blocker: on cold start, a peer mark published before the held REQ registers is never received. If Scenario: the user cold-launches the phone with slow AUTH. The snapshot reads old state, then the desktop marks a channel read, then AUTH completes and the REQ registers. The phone shows that channel unread until the next resume or a publish that reads the snapshot. The same 500 ms fallback applies to the subscription Fix: also catch up when the first |
||
| if (generation != _liveGeneration) return; | ||
| if (status == RelaySubscriptionStatus.ready) { | ||
| if (!_liveReady && returned) { | ||
| unawaited(_catchUpAfterLateReady(generation)); | ||
| } | ||
| _liveReady = true; | ||
| } else { | ||
| _liveReady = false; | ||
| if (status == RelaySubscriptionStatus.ready && !_disposed) { | ||
| unawaited(_requestSync()); | ||
| } | ||
| }, | ||
| ); | ||
| returned = true; | ||
| if (_disposed) { | ||
| unsub.call(); | ||
| return false; | ||
|
|
@@ -515,19 +499,6 @@ class ReadStateManager { | |
| } | ||
| } | ||
|
|
||
| /// Reads the snapshot after the live subscription went live late, once | ||
| /// any pass already running is done. Skipped when a reconnect pass has | ||
| /// replaced the subscription, since that pass read the snapshot after it. | ||
| Future<void> _catchUpAfterLateReady(int generation) async { | ||
| final pass = _queuedResync ?? _runningResync ?? _initializing; | ||
| if (pass != null) await pass; | ||
| if (_disposed || !_servesThisCommunity) return; | ||
| if (generation != _liveGeneration) return; | ||
| // A failed read is retried by the next publish, which reads the | ||
| // snapshot in full after a failure. | ||
| if (!await _fetchAndMerge()) _schedulePublish(); | ||
| } | ||
|
|
||
| void _handleIncomingEvent(NostrEvent event) { | ||
| if (_disposed || !_servesThisCommunity) return; | ||
| _mergeEvents([event]); | ||
|
|
@@ -648,8 +619,9 @@ class ReadStateManager { | |
| final checkConnection = _relaySession?.captureConnectionFence(); | ||
| var snapshot = <NostrEvent>[]; | ||
| if (_relaySession != null) { | ||
| final read = ++_snapshotsStarted; | ||
| final events = await _relaySession.fetchReadStateSnapshot(); | ||
| _lastSnapshotSucceeded = true; | ||
| _settleSnapshot(read, succeeded: true); | ||
| if (!_servesThisCommunity || | ||
| !_ownsJournal || | ||
| (_disposed && !allowDisposed)) { | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Blocker: every clean reconnect reads the full snapshot twice.
The provider's
connectedlistener runs synchronously insidestate =in_handleConnected, before_replayLiveSubscriptions. So this resubscribe sendsREQ l-2on the new socket, and the replay right after it sendsl-2again (marked ambiguous in_sendLiveReq). Both EOSEs emitready. The first one releasessubscribeand the loop starts snapshot A. The second lands while A downloads, sets_snapshotWanted, and the loop reads the whole snapshot again.Repro: a real
RelaySessionNotifierwith a recording socket and a gated fakefetchReadStateSnapshot. CallreinitializeRemote(), thendebugReplayLiveSubscriptions(), then EOSE each REQ, the second while the read is held.eefce94ba0[l-2, l-2]824494224a[l-2, l-2]A separate reviewer probe got the same result through the provider listener. With only this resubscribe removed, it got one REQ (
[l-1]) and one read.That is a full read-state download (discovery plus POST) on every resume, on the accounts this PR is meant to help. The PR text's "Each resume reads the read-state snapshot once over HTTP" no longer holds.
Suggested fix (simplest): let the session own resubscription. It already keeps live subscriptions across reconnects and replays them, and a subscription opened before AUTH is held and sent on connect. Pass
onClosedto mark the subscription lost, and resubscribe here only when it is lost (terminal CLOSED, or a failed subscribe). Otherwise the replay'sreadyalready asks the loop for the read. That also removes_resubscribeWanted, the wait for the publish before resubscribing, and the cold start's extra pass, and it stops a terminal CLOSED from dropping the subscription silently. The alternative is to restore "a repeated ready while already live reads nothing", reset onretryingand on each new subscribe. That works, but it keeps the duplicate REQ.Please cover it with a test through the real session replay. The
_FakeRelaySessionpath can't show the duplicate EOSE.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🤖 Fixed in c620e9d4b5, with your suggested fix. A reconnect no longer resubscribes: the session's replay of the live subscription is the one REQ on the new socket, and its ready asks the sync loop for the one snapshot read. The manager opens a new subscription only after losing one (
onClosedon a terminal CLOSED, or a failed subscribe), on the next reconnect. That also removed the resubscribe flag and the wait for an in-flight publish; the sync loop now just opens the subscription when it is missing. No generation or ready-while-live check came back.New journey test
a clean reconnect through the real session replays one REQ and reads the snapshot once, and a lost subscription is reopeneddrives the realRelaySessionNotifierwith a recording socket:reinitializeRemote(), thendebugReplayLiveSubscriptions(), then EOSE while the read is held. It fails against 824494224a ([l-2, l-2], two reads) and passes now ([l-1], one read). It also checks that after a terminal CLOSED the next reconnect opensl-2and reads the snapshot. In that rare lost case the replay can still send the new REQ a second time.The cold start keeps its occasional extra read when ready lands after the 500 ms fallback.