Skip to content

Commit 5e52fa9

Browse files
browsquare-tomb
andauthored
perf(mobile): download the identity archive list only when it changed (#9445)
Every reconnect refetches the community's identity archive list (kind 13535) so the agent pickers stay current. On Block's Builderlab that list is about 228 KB, and it is downloaded again on every resume even though it rarely changes. The provider now remembers the last verified list per community and relay authority, and asks the relay only for a newer one (`since` the known list's `created_at` plus one). When nothing newer exists it reuses the known list, as the same set instance so watchers (mention candidates, search, the Agents page, channel details, the members sheet) don't rebuild. A newer list replaces it after the same verification as before. A list that fails any check, including its signature, yields no archived identities and drops the remembered one. ### Validation Provider tests: a refresh with an unchanged list sends `since`, keeps the known set and leaves the hidden set's listeners untouched; a newer list replaces it and becomes the new baseline; every invalid list (wrong signer, missing `-` marker, bad signature) starting from a warm cache drops the remembered list, so a following empty answer hides nobody. Each new step fails with its fix reverted. The existing verification, community-switch and deadline tests pass with the relay fake now honoring `since`. Full mobile test suite (4803 passed) and `flutter analyze` pass. On an iPhone 12 mini signed in to Block's Builderlab (instrumented profile build), a resume after 60 s in the background downloaded the 228 KB list each time before this change. ### Regressions **Accepted tradeoff: a same-second list can be missed.** The cursor assumes each list the relay publishes is stamped strictly later than the previous one. The relay nearly always does this (`publish_nipia_archival_list` stamps past the current head and republishes on drift), but two lists published within one second can share a `created_at`, and NIP-IA then keeps the one with the lower id. If the app already holds the other one, it misses the replacement until a later list arrives (the next archive or unarchive) or the app restarts. Meanwhile a just-archived or just-unarchived agent can show wrongly in pickers and search, never in history. We accept this for the bandwidth and resume-time win; a relay follow-up that stamps each list under the replacement lock and makes the rule a NIP-IA MUST removes the case. The same bound applies if a relay's database is restored to an older list: the app keeps its newer one until the relay publishes a list newer than it, or the app restarts. A failed or timed-out fetch still hides nobody, as before. --------- Signed-off-by: Tom Brow <[email protected]> Co-authored-by: Tom Brow <[email protected]>
1 parent 9e40bf5 commit 5e52fa9

2 files changed

Lines changed: 148 additions & 27 deletions

File tree

‎mobile/lib/shared/identity_archives/identity_archives.dart‎

Lines changed: 50 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ import 'package:buzz/shared/crypto/event_signing.dart';
66
import 'package:hooks_riverpod/hooks_riverpod.dart';
77
import 'package:http/http.dart' as http;
88
import 'package:buzz/shared/http/app_http_client.dart';
9+
import 'package:nostr/nostr.dart' as nostr;
910

1011
import '../relay/relay.dart';
1112
import '../me/me_sync.dart';
@@ -16,6 +17,14 @@ final relayInformationClientProvider = Provider<http.Client>((ref) {
1617
return client;
1718
});
1819

20+
/// The last verified archive list per community and authority, so a refresh
21+
/// asks the relay only for a newer list instead of downloading it again.
22+
class _ArchiveCache {
23+
final entries = <(String, String), ({int createdAt, Set<String> archived})>{};
24+
}
25+
26+
final _archiveCacheProvider = Provider<_ArchiveCache>((_) => _ArchiveCache());
27+
1928
/// The identities this relay's NIP-IA archive list (kind 13535) names, never
2029
/// the viewer: an archive stays visible to its subject. Empty when the list
2130
/// cannot be read. This is presentation, never ownership or admission.
@@ -61,12 +70,37 @@ final identityArchivesProvider = FutureProvider<Set<String>>((ref) async {
6170
!RegExp(r'^[0-9a-f]{64}$').hasMatch(authority)) {
6271
return {};
6372
}
73+
final cache = ref.read(_archiveCacheProvider).entries;
74+
final key = (sync.community, authority);
75+
final known = cache[key];
76+
// Ask only for a list newer than the known one. This assumes each list is
77+
// stamped strictly later than the previous one, which the relay does
78+
// except when two lists are published in the same second; then the
79+
// second can be missed until a later list arrives or the app restarts.
6480
final events = await sync.session.fetchHistory(
65-
NostrFilter(kinds: const [13535], authors: [authority], limit: 1),
81+
NostrFilter(
82+
kinds: const [13535],
83+
authors: [authority],
84+
since: known == null ? null : known.createdAt + 1,
85+
limit: 1,
86+
),
6687
timeout: const Duration(seconds: 3),
6788
);
6889
sync.check();
69-
return (await _parseArchives(events, authority))..remove(sync.pubkey);
90+
// Hand back the known instance when it does not name the viewer, so an
91+
// unchanged list does not rebuild every watcher.
92+
Set<String> visible(Set<String> archived) => archived.contains(sync.pubkey)
93+
? ({...archived}..remove(sync.pubkey))
94+
: archived;
95+
if (known != null && events.isEmpty) return visible(known.archived);
96+
final archived = await _parseArchives(events, authority);
97+
if (events.length != 1 || archived == null) {
98+
cache.remove(key);
99+
return {};
100+
}
101+
final shared = Set<String>.unmodifiable(archived);
102+
cache[key] = (createdAt: events.single.createdAt, archived: shared);
103+
return visible(shared);
70104
} catch (_) {
71105
return {};
72106
} finally {
@@ -98,23 +132,30 @@ final hiddenArchivedIdentitiesProvider = Provider<Set<String>>((ref) {
98132
return known.$2;
99133
});
100134

101-
Future<Set<String>> _parseArchives(List<NostrEvent> events, String authority) =>
102-
Isolate.run(() => _archived(events, authority));
135+
/// The archived identities, or null when [events] is not one valid list.
136+
Future<Set<String>?> _parseArchives(
137+
List<NostrEvent> events,
138+
String authority,
139+
) => Isolate.run(() => _archived(events, authority));
103140

104-
Set<String> _archived(List<NostrEvent> events, String authority) {
105-
if (events.length != 1) return {};
141+
Set<String>? _archived(List<NostrEvent> events, String authority) {
142+
if (events.length != 1) return null;
106143
final event = events.single;
107144
final markers = event.tags.where((tag) => tag.isNotEmpty && tag.first == '-');
108145
if (event.kind != 13535 ||
109146
event.pubkey != authority ||
110147
event.content.isNotEmpty ||
111148
markers.length != 1 ||
112149
markers.single.length != 1) {
113-
return {};
150+
return null;
114151
}
115152
final encoded = jsonEncode(event.toJson());
116-
if (utf8.encode(encoded).length > 2 * 1024 * 1024) return {};
117-
verifiedNostrEventFromJson(encoded);
153+
if (utf8.encode(encoded).length > 2 * 1024 * 1024) return null;
154+
try {
155+
verifiedNostrEventFromJson(encoded);
156+
} on nostr.EventValidationException {
157+
return null;
158+
}
118159
return {
119160
for (final tag in event.tags)
120161
if (tag.length >= 2 &&

‎mobile/test/shared/identity_archives/identity_archives_test.dart‎

Lines changed: 98 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -18,12 +18,14 @@ void main() {
1818
final authority = nostr.Keys.generate();
1919
final impostor = nostr.Keys.generate();
2020
final target = 'a' * 64;
21+
var createdAt = 1700000000;
2122
NostrEvent snapshot(nostr.Keys signer, {bool marker = true}) =>
2223
NostrEvent.fromJson(
2324
nostr.Event.from(
2425
kind: 13535,
2526
content: '',
2627
secretKey: signer.secret,
28+
createdAt: createdAt++,
2729
tags: [
2830
if (marker) ['-'],
2931
['p', target],
@@ -54,12 +56,22 @@ void main() {
5456
addTearDown(client.close);
5557
expect(await container.read(identityArchivesProvider.future), {target});
5658
expect(relay.author, authority.public);
57-
for (final invalid in [
58-
snapshot(impostor),
59-
snapshot(authority, marker: false),
60-
NostrEvent.fromJson(snapshot(authority).toJson()..['sig'] = '0' * 128),
59+
for (final invalid in <NostrEvent Function()>[
60+
() => snapshot(impostor),
61+
() => snapshot(authority, marker: false),
62+
() => NostrEvent.fromJson(
63+
snapshot(authority).toJson()..['sig'] = '0' * 128,
64+
),
6165
]) {
62-
relay.event = invalid;
66+
// Each case starts from a warm cache, so it must also drop the known
67+
// list: once the relay has nothing newer, nobody stays hidden.
68+
relay.event = snapshot(authority);
69+
container.invalidate(identityArchivesProvider);
70+
expect(await container.read(identityArchivesProvider.future), {target});
71+
relay.event = invalid();
72+
container.invalidate(identityArchivesProvider);
73+
expect(await container.read(identityArchivesProvider.future), isEmpty);
74+
relay.event = null;
6375
container.invalidate(identityArchivesProvider);
6476
expect(await container.read(identityArchivesProvider.future), isEmpty);
6577
}
@@ -77,17 +89,19 @@ void main() {
7789
final authority = nostr.Keys.generate();
7890
final archivedInA = 'a' * 64;
7991
final archivedInB = 'b' * 64;
80-
NostrEvent snapshot(List<String> keys) => NostrEvent.fromJson(
81-
nostr.Event.from(
82-
kind: 13535,
83-
content: '',
84-
secretKey: authority.secret,
85-
tags: [
86-
['-'],
87-
for (final key in keys) ['p', key],
88-
],
89-
).toMap(),
90-
);
92+
NostrEvent snapshot(List<String> keys, {int? createdAt}) =>
93+
NostrEvent.fromJson(
94+
nostr.Event.from(
95+
kind: 13535,
96+
content: '',
97+
secretKey: authority.secret,
98+
createdAt: createdAt,
99+
tags: [
100+
['-'],
101+
for (final key in keys) ['p', key],
102+
],
103+
).toMap(),
104+
);
91105
// Community A's list also names the viewer, who must stay visible.
92106
final relay = _ArchiveRelay(snapshot([archivedInA, viewer.public]));
93107
final communityB = Completer<void>();
@@ -110,7 +124,10 @@ void main() {
110124
);
111125
addTearDown(container.dispose);
112126
addTearDown(client.close);
113-
container.listen(hiddenArchivedIdentitiesProvider, (_, _) {});
127+
var hiddenChanges = 0;
128+
container.listen(hiddenArchivedIdentitiesProvider, (_, _) {
129+
hiddenChanges++;
130+
});
114131
Future<Set<String>> settled() async {
115132
await container.read(identityArchivesProvider.future);
116133
await pumpEventQueue();
@@ -122,6 +139,16 @@ void main() {
122139
container.invalidate(identityArchivesProvider);
123140
expect(container.read(hiddenArchivedIdentitiesProvider), {archivedInA});
124141
expect(await settled(), {archivedInA});
142+
// An unchanged list leaves the hidden set alone for every watcher.
143+
relay.event = snapshot([
144+
archivedInA,
145+
], createdAt: relay.event!.createdAt + 10);
146+
container.invalidate(identityArchivesProvider);
147+
expect(await settled(), {archivedInA});
148+
final changes = hiddenChanges;
149+
container.invalidate(identityArchivesProvider);
150+
expect(await settled(), {archivedInA});
151+
expect(hiddenChanges, changes);
125152

126153
// Switch community and account: A's list no longer applies, so nobody
127154
// is hidden until B's list arrives.
@@ -136,6 +163,55 @@ void main() {
136163
},
137164
);
138165

166+
test('a refresh downloads the list again only when it changed', () async {
167+
final owner = nostr.Keys.generate();
168+
final authority = nostr.Keys.generate();
169+
NostrEvent snapshot(String target, int createdAt) => NostrEvent.fromJson(
170+
nostr.Event.from(
171+
kind: 13535,
172+
content: '',
173+
secretKey: authority.secret,
174+
createdAt: createdAt,
175+
tags: [
176+
['-'],
177+
['p', target],
178+
],
179+
).toMap(),
180+
);
181+
final relay = _ArchiveRelay(snapshot('a' * 64, 1700000000));
182+
final client = MockClient(
183+
(_) async => http.Response(jsonEncode({'self': authority.public}), 200),
184+
);
185+
final container = ProviderContainer(
186+
overrides: [
187+
meSyncProvider.overrideWithValue(
188+
MeSync(
189+
relay,
190+
RelayConfig(baseUrl: 'https://relay.test', nsec: owner.nsec),
191+
),
192+
),
193+
relayInformationClientProvider.overrideWithValue(client),
194+
],
195+
);
196+
addTearDown(container.dispose);
197+
addTearDown(client.close);
198+
expect(await container.read(identityArchivesProvider.future), {'a' * 64});
199+
expect(relay.since, isNull);
200+
201+
// Unchanged: the relay has nothing newer, and the known list stands.
202+
container.invalidate(identityArchivesProvider);
203+
expect(await container.read(identityArchivesProvider.future), {'a' * 64});
204+
expect(relay.since, 1700000001);
205+
206+
// Replaced: the newer list wins and becomes the new baseline.
207+
relay.event = snapshot('b' * 64, 1700000100);
208+
container.invalidate(identityArchivesProvider);
209+
expect(await container.read(identityArchivesProvider.future), {'b' * 64});
210+
container.invalidate(identityArchivesProvider);
211+
expect(await container.read(identityArchivesProvider.future), {'b' * 64});
212+
expect(relay.since, 1700000101);
213+
});
214+
139215
test('slow metadata has a total deadline and cancels its stream', () async {
140216
final owner = nostr.Keys.generate();
141217
final relay = _ArchiveRelay(null);
@@ -226,14 +302,18 @@ class _ArchiveRelay extends RelaySessionNotifier {
226302
_ArchiveRelay(this.event);
227303
NostrEvent? event;
228304
String? author;
305+
int? since;
229306
@override
230307
Future<List<NostrEvent>> fetchHistory(
231308
NostrFilter filter, {
232309
Duration timeout = const Duration(seconds: 8),
233310
}) async {
234311
expect(filter.kinds, [13535]);
235312
author = filter.authors!.single;
236-
return [?event];
313+
since = filter.since;
314+
final event = this.event;
315+
if (event == null || event.createdAt < (filter.since ?? 0)) return [];
316+
return [event];
237317
}
238318
}
239319

0 commit comments

Comments
 (0)