Skip to content

Commit c97c85d

Browse files
browsquare-tombstudio
authored
perf(mobile): load only the directory when search opens (#9466)
Opening search on a list that is already loaded refreshed the whole channel list to get the open-channel directory: the membership rosters (kind 39002), member channel metadata, hidden DMs, Huddle starts, DM names and the latest messages of every joined channel, then the directory pages. Only the directory was what search needed; everything else was already in the list and kept current by live updates. Search now reads only the directory pages and merges them into the current list: every channel you are not a member of is rebuilt from the fresh directory (latest revision per channel, open and non-DM only, the same rules as a full refresh), and joined channels keep their entries untouched. When the list has not loaded yet, the full refresh runs as before. Races: - A directory-only load is fenced by scope alone (community, identity, disposal), not by the list-refresh generation, so it neither retires an in-flight membership refresh nor is retired by one. A community or identity switch retires it: its response is discarded and it no longer owns its loading status, so returning to the first community (before or after the old request ends) lets search load the directory again. - Directory loads are ordered among themselves: only the newest one writes the directory and its load status, so an older full refresh cannot overwrite a newer search load and vice versa. - A list refresh that merged an older directory and then installs after a search load landed re-merges the newer directory, so the open channel search just found does not disappear. - A list refresh no longer resets a live directory-only load's status to idle (that reset existed because a refresh used to discard the in-flight directory, which it no longer does for this kind of load). A load retired by a scope switch is not live, so the reset still applies to the status it left behind. ## Results iPhone 12 mini, profile build, Builderlab account. Typing a query into search on a loaded list, more than 30 seconds after the last directory load (two runs each, same numbers both times): | | HTTP per search | What it read | | --- | --- | --- | | main | 3.59 MB | rosters 1.53 MB, channel metadata and directory 1.26 MB, latest messages 0.80 MB | | This change | 1.26 MB | directory pages only | Unjoined open channels still appear in results with their Open badge. ## Regressions The tradeoff we accept: on a search open, joined channels' member counts, unread badges and latest-message times are not re-read; they show what the last full refresh found, as updated by live events since. Live join and leave notices, metadata changes and new messages still apply as they arrive, and the next ordinary refresh (launch, reconnect, resume, pull to refresh) re-reads everything. The case that differs is a change that live delivery missed on a healthy connection (for example a roster change made while a live chunk was briefly refused): before, opening search happened to repair it; now the next ordinary refresh does. Nothing else changes: open channels are listed under the same visibility rules, a failed directory load still reports its error to search and keeps the cached discovery, and the first load is unchanged. ## Testing - Full mobile suite passes; `flutter analyze` clean. - New tests: a search load on a loaded list issues only directory queries (no roster, member metadata, hidden-DM, Huddle or message reads), lists the new open channel, drops one that became private and keeps the joined channel object unchanged; a membership refresh during the load is kept and does not reset its status; a list refresh that merged the directory before a search load landed keeps the newer directory; a community switch discards the response; returning to the first community, both while the old request is in flight and after it ended, loads the directory again. - Automated device run on the iPhone 12 mini: launch, two search sessions typing a query, results list joined and unjoined open channels, close and reopen search. Rerun on f216dd6 (after the scope-switch fix) with the same results: 1.26 MB HTTP per query, unjoined open channels marked Open. --------- Signed-off-by: Tom Brow <[email protected]> Co-authored-by: Tom Brow <[email protected]> Co-authored-by: studio <dc2fc481343672df31a689840c09299972e704dac592e0da6f18486646a7ee3e@buzz.block.builderlab.xyz>
1 parent 5f1cd67 commit c97c85d

3 files changed

Lines changed: 476 additions & 31 deletions

File tree

‎mobile/lib/features/channels/channel_directory.dart‎

Lines changed: 164 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -136,8 +136,40 @@ class _StaleChannelRefresh implements Exception {
136136
/// This token is captured once at the start of every ordinary, directory, and
137137
/// reconnect refresh. It is re-checked after each relay await, so an older
138138
/// request cannot regain ownership by reaching subscription setup last.
139-
class _ChannelRefreshFence {
139+
/// What a directory load checks before each write: the scope it serves.
140+
abstract interface class _DirectoryLoadFence {
141+
/// Relay-and-identity scope that started the load.
142+
String get scope;
143+
144+
/// Throws [_StaleChannelRefresh] once the load must write nothing.
145+
void ensureCurrent();
146+
}
147+
148+
/// Fences a directory-only load to its scope alone.
149+
///
150+
/// Unlike [_ChannelRefreshFence] it holds no refresh generation: a directory
151+
/// read does not depend on memberships, so it neither retires an in-flight
152+
/// list refresh nor is retired by one. A community or identity switch, or
153+
/// disposal, still retires it, so its response never reaches another scope.
154+
class _DirectoryOnlyFence implements _DirectoryLoadFence {
155+
@override
156+
final String scope;
157+
158+
final _ChannelRefreshCoordinator _coordinator;
159+
160+
_DirectoryOnlyFence(this._coordinator, this.scope);
161+
162+
@override
163+
void ensureCurrent() {
164+
if (!_coordinator.isMounted() || scope != _coordinator.currentScope()) {
165+
throw const _StaleChannelRefresh();
166+
}
167+
}
168+
}
169+
170+
class _ChannelRefreshFence implements _DirectoryLoadFence {
140171
/// Relay-and-identity scope that started the refresh.
172+
@override
141173
final String scope;
142174

143175
final _ChannelRefreshCoordinator _coordinator;
@@ -155,6 +187,7 @@ class _ChannelRefreshFence {
155187
///
156188
/// Call after every await and immediately before every write to metadata,
157189
/// cache, load status, subscriptions or provider state.
190+
@override
158191
void ensureCurrent() {
159192
if (!isCurrent) throw const _StaleChannelRefresh();
160193
}
@@ -390,6 +423,16 @@ class _ChannelRefreshCoordinator {
390423
/// Generation of the most recently issued or retired request.
391424
int get generation => _generation;
392425

426+
// Directory loads are ordered among themselves: only the newest one writes
427+
// the directory or its status. A directory-only load is not retired by a
428+
// list refresh, so [beginRefresh] must leave its loading status alone.
429+
int _directoryGeneration = 0;
430+
int? _directoryOnlyGeneration;
431+
432+
/// Starts a directory-only load for the active scope.
433+
_DirectoryOnlyFence beginDirectoryOnly() =>
434+
_DirectoryOnlyFence(this, currentScope());
435+
393436
_ChannelRefreshCoordinator({
394437
required this.isMounted,
395438
required this.currentScope,
@@ -416,6 +459,16 @@ class _ChannelRefreshCoordinator {
416459
/// wire cannot be written into the new scope.
417460
void retireInFlight() => _generation++;
418461

462+
/// Retires every in-flight request, directory-only loads included, for a
463+
/// relay or identity change. A directory-only load survives ordinary list
464+
/// refreshes, but not a scope change: its loading status must not look
465+
/// owned when the user returns to the old scope.
466+
void retireScope() {
467+
_generation++;
468+
_directoryGeneration++;
469+
_directoryOnlyGeneration = null;
470+
}
471+
419472
/// Starts a fenced refresh without issuing the directory query.
420473
///
421474
/// Used by callers that must carry the scope across later awaits even when
@@ -433,7 +486,8 @@ class _ChannelRefreshCoordinator {
433486
fence.ensureCurrent();
434487
if (!fetchesDirectory) {
435488
final status = loadStatus();
436-
if (status.isLoading(scope)) {
489+
if (status.isLoading(scope) &&
490+
_directoryOnlyGeneration != _directoryGeneration) {
437491
// This refresh takes ownership of the shared generation, so the
438492
// in-flight directory response will be discarded. Mark it idle, not
439493
// error: nothing failed, so a caller may load it again, and search
@@ -456,18 +510,25 @@ class _ChannelRefreshCoordinator {
456510
/// Fetches the directory under [fence], or throws if the fence is retired.
457511
///
458512
/// Returns null when the request failed inside the current scope, which means
459-
/// "retain the cached discovery". The fence is re-checked after the await and
460-
/// before every write, on both the success and the failure path.
513+
/// "retain the cached discovery", or when a newer directory load started,
514+
/// which then owns the directory and its status. The fence is re-checked
515+
/// after the await and before every write, on both the success and the
516+
/// failure path.
461517
Future<List<NostrEvent>?> loadDirectory(
462518
RelaySessionNotifier session,
463-
_ChannelRefreshFence fence,
519+
_DirectoryLoadFence fence,
464520
) async {
521+
final directoryGeneration = ++_directoryGeneration;
522+
_directoryOnlyGeneration = fence is _DirectoryOnlyFence
523+
? directoryGeneration
524+
: null;
465525
loadStatus().markLoading(fence.scope);
466526
final List<NostrEvent> metas;
467527
try {
468528
metas = await _fetchChannelDirectoryMetas(session);
469529
} catch (error, stackTrace) {
470530
fence.ensureCurrent();
531+
if (directoryGeneration != _directoryGeneration) return null;
471532
loadStatus().markError(fence.scope);
472533
debugPrint(
473534
'[ChannelsNotifier] channel directory refresh failed; retaining '
@@ -476,6 +537,7 @@ class _ChannelRefreshCoordinator {
476537
return null;
477538
}
478539
fence.ensureCurrent();
540+
if (directoryGeneration != _directoryGeneration) return null;
479541
loadStatus().markLoaded(fence.scope);
480542
return metas;
481543
}
@@ -621,3 +683,100 @@ extension ObservedThreadActivity on ChannelsNotifier {
621683
}
622684
}
623685
}
686+
687+
/// Directory-only loads and the list edits they share with full refreshes.
688+
///
689+
/// Lives in this part file because `channels_provider.dart` sits against the
690+
/// repository-wide 1200-line file ceiling enforced by `just file-size-check`.
691+
extension _ChannelsNotifierDirectory on ChannelsNotifier {
692+
/// Downloads only the open-channel directory and merges it into the list
693+
/// [current] returns once it lands, for a list that already has its
694+
/// memberships. Joined channels keep their entries; live updates keep them
695+
/// current. Returns null when nothing should be written: the load failed
696+
/// (its status says so), a newer directory load replaced it, the scope
697+
/// changed, or there is no list to merge into.
698+
Future<List<Channel>?> _loadDirectoryOnly(
699+
List<Channel>? Function() current,
700+
) async {
701+
final fence = _refreshCoordinator.beginDirectoryOnly();
702+
try {
703+
final metas = await _refreshCoordinator.loadDirectory(
704+
_lifecycleRef.read(relaySessionProvider.notifier),
705+
fence,
706+
);
707+
if (metas == null) return null;
708+
_directoryMetas = metas;
709+
_directoryMetasScope = fence.scope;
710+
} on _StaleChannelRefresh {
711+
return null;
712+
}
713+
final previous = current();
714+
if (previous == null) return null;
715+
final channels = [...previous];
716+
_replaceDirectoryChannels(channels);
717+
_sortChannels(channels);
718+
_invalidateChangedDetails(channels, previous);
719+
return channels;
720+
}
721+
722+
/// Rebuilds every channel in [channels] the user is not a member of from
723+
/// [_directoryMetas], with a full refresh's visibility rules: the latest
724+
/// revision of each open, non-DM channel. Joined channels are untouched.
725+
void _replaceDirectoryChannels(List<Channel> channels) {
726+
final previous = {
727+
for (final channel in channels)
728+
if (!channel.isMember) channel.id: channel,
729+
};
730+
channels.removeWhere((channel) => !channel.isMember);
731+
final memberIds = {for (final channel in channels) channel.id};
732+
for (final event in _latestChannelEvents(_directoryMetas, kind: 39000)) {
733+
final data = ChannelData.fromEvent(event);
734+
if (memberIds.contains(data.id) ||
735+
data.visibility == 'private' ||
736+
data.channelType == 'dm') {
737+
continue;
738+
}
739+
final channel = _channelFromMeta(event, data, isMember: false);
740+
final lastMessageAt = previous[data.id]?.lastMessageAt;
741+
channels.add(
742+
lastMessageAt == null
743+
? channel
744+
: channel.copyWith(lastMessageAt: lastMessageAt),
745+
);
746+
}
747+
}
748+
749+
/// Invalidates the details of channels whose identity or archive state
750+
/// changed from [previous], so opening one does not show stale Solo or
751+
/// archive state. Returns [previous] by id.
752+
Map<String, Channel> _invalidateChangedDetails(
753+
List<Channel> channels,
754+
List<Channel>? previous,
755+
) {
756+
final prevById = <String, Channel>{
757+
for (final c in previous ?? const <Channel>[]) c.id: c,
758+
};
759+
for (final channel in channels) {
760+
final prev = prevById[channel.id];
761+
if (prev == null ||
762+
prev.isArchived != channel.isArchived ||
763+
prev.channelType != channel.channelType ||
764+
prev.visibility != channel.visibility ||
765+
prev.createdBy != channel.createdBy ||
766+
!listEquals(prev.labels, channel.labels)) {
767+
_lifecycleRef.invalidate(channelDetailsProvider(channel.id));
768+
}
769+
}
770+
return prevById;
771+
}
772+
}
773+
774+
/// Orders the list by channel type, then case-insensitively by name to match
775+
/// desktop's `localeCompare` ordering.
776+
void _sortChannels(List<Channel> channels) => channels.sort((left, right) {
777+
final typeOrder =
778+
(_channelTypeOrder[left.channelType] ?? 99) -
779+
(_channelTypeOrder[right.channelType] ?? 99);
780+
if (typeOrder != 0) return typeOrder;
781+
return left.name.toLowerCase().compareTo(right.name.toLowerCase());
782+
});

‎mobile/lib/features/channels/channels_provider.dart‎

Lines changed: 22 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -227,7 +227,7 @@ class ChannelsNotifier extends AsyncNotifier<List<Channel>> {
227227
_confirmedJoins.clear();
228228
// Retire any in-flight directory request: its response describes the
229229
// previous relay or identity and must not reach this scope's state.
230-
_refreshCoordinator.retireInFlight();
230+
_refreshCoordinator.retireScope();
231231
}
232232
final snapshot =
233233
(_hasLoaded ? state.value : null) ??
@@ -429,6 +429,7 @@ class ChannelsNotifier extends AsyncNotifier<List<Channel>> {
429429
_directoryMetasScope = fence.scope;
430430
}
431431
}
432+
final directoryMetas = _directoryMetas;
432433

433434
// Merge and dedupe by `d` tag. Kind:39000 is parameterized-replaceable,
434435
// but stale revisions from before the relay's d_tag backfill can linger.
@@ -579,14 +580,11 @@ class ChannelsNotifier extends AsyncNotifier<List<Channel>> {
579580
}
580581
}
581582

582-
channels.sort((left, right) {
583-
final typeOrder =
584-
(_channelTypeOrder[left.channelType] ?? 99) -
585-
(_channelTypeOrder[right.channelType] ?? 99);
586-
if (typeOrder != 0) return typeOrder;
587-
// Case-insensitive to match desktop's `localeCompare` ordering.
588-
return left.name.toLowerCase().compareTo(right.name.toLowerCase());
589-
});
583+
// A directory-only load that landed during the awaits above is newer.
584+
if (!identical(directoryMetas, _directoryMetas)) {
585+
_replaceDirectoryChannels(channels);
586+
}
587+
_sortChannels(channels);
590588

591589
// Details are a separate cache. Invalidate changes that would otherwise
592590
// overwrite fresh Solo identity or archive state when opening a channel.
@@ -596,20 +594,7 @@ class ChannelsNotifier extends AsyncNotifier<List<Channel>> {
596594
// after the await.
597595
fence.ensureCurrent();
598596

599-
final prevById = <String, Channel>{
600-
for (final c in state.value ?? const <Channel>[]) c.id: c,
601-
};
602-
for (final channel in channels) {
603-
final prev = prevById[channel.id];
604-
if (prev == null ||
605-
prev.isArchived != channel.isArchived ||
606-
prev.channelType != channel.channelType ||
607-
prev.visibility != channel.visibility ||
608-
prev.createdBy != channel.createdBy ||
609-
!listEquals(prev.labels, channel.labels)) {
610-
ref.invalidate(channelDetailsProvider(channel.id));
611-
}
612-
}
597+
final prevById = _invalidateChangedDetails(channels, state.value);
613598

614599
// Layer live delivery onto the already-built snapshot without making EOSE
615600
// part of the provider's readiness path. Every refresh queues a generation;
@@ -1080,11 +1065,14 @@ class ChannelsNotifier extends AsyncNotifier<List<Channel>> {
10801065
fresh)) {
10811066
return;
10821067
}
1083-
await retryDirectory();
1068+
await retryDirectory(directoryOnly: true);
10841069
}
10851070

10861071
/// Retries channel discovery while retaining the current channel list.
1087-
Future<void> retryDirectory() async {
1072+
///
1073+
/// With [directoryOnly], a list that has loaded its memberships reads only
1074+
/// the directory again; joined channels keep their entries.
1075+
Future<void> retryDirectory({bool directoryOnly = false}) async {
10881076
final directoryStatus = ref.read(
10891077
channelDirectoryLoadStatusProvider.notifier,
10901078
);
@@ -1115,6 +1103,15 @@ class ChannelsNotifier extends AsyncNotifier<List<Channel>> {
11151103
}
11161104
if (!ref.mounted || generation != _buildGeneration) return;
11171105
final previousChannels = state.value;
1106+
if (directoryOnly && _hasLoaded && previousChannels != null) {
1107+
// The list has its memberships, so only the directory is read again.
1108+
final saveSnapshot = _saveSnapshot;
1109+
final channels = await _loadDirectoryOnly(() => state.value);
1110+
if (channels == null) return;
1111+
state = AsyncData(channels);
1112+
saveSnapshot?.call(channels, _hiddenDmIds);
1113+
return;
1114+
}
11181115
try {
11191116
state = AsyncData(
11201117
await _fetch(

0 commit comments

Comments
 (0)