Skip to content

Commit 5e36d65

Browse files
browstudio
andauthored
fix(mobile): inflate gzip read-state snapshot responses (#9089)
On relays behind a CDN that gzips responses (the Builderlab relay does), mobile read state never synced. Every read-state snapshot fetch, and therefore every read-mark publish, failed with `FormatException: Unexpected extension byte`, so read marks made on the phone never reached other devices and each reconnect retried and failed again. The relay query transport deliberately leaves gzip bodies compressed so they can be inflated off the UI isolate (see `RelayHttpQueryClient`), but the snapshot path decoded the raw bytes with `utf8.decode`. It now uses `relayResponseText`, like the other query callers, for both the discovery and the snapshot responses. Heads-up for reviewers: with the snapshot working, a resume also pays for the full snapshot download on top of the existing read-state replay on the live subscription. A follow-up PR removes that replay. ### Validation - Added test: `inflates gzip discovery and snapshot bodies` serves both responses gzipped. Without the fix it fails with the same `Unexpected extension byte` error seen on the device. - Full mobile suite: all pass except the known stale `fullscreen_onboarding_test` case, which fails on main too. - iPhone 12 mini (Profile build, Builderlab community): before, every resume logged `snapshot failed` and `publish failed (journal retained)`. After, the snapshot on resume and the snapshot inside a publish both completed with no errors. - Automated app run on the same phone with this change: Home backgrounded past the reconnect grace period and resumed, then search and a large channel opened. No read-state errors logged. Read marks were not compared across devices. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Tom Brow <[email protected]> Co-authored-by: Tom Brow <[email protected]> Co-authored-by: studio <dc2fc481343672df31a689840c09299972e704dac592e0da6f18486646a7ee3e@buzz.block.builderlab.xyz>
1 parent 50c10dc commit 5e36d65

4 files changed

Lines changed: 135 additions & 13 deletions

File tree

‎mobile/lib/shared/relay/read_state_snapshot.dart‎

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,12 @@ import 'package:flutter/foundation.dart';
55
import 'package:nostr/nostr.dart' as nostr;
66

77
import 'nostr_models.dart';
8+
import 'relay_query_response.dart';
89

910
/// Snapshot bytes and the authenticated scope they must match.
1011
typedef ReadStateSnapshotInput = ({
11-
String text,
12+
Uint8List bytes,
13+
String? contentType,
1214
String viewer,
1315
String community,
1416
});
@@ -24,6 +26,9 @@ Future<List<NostrEvent>> decodeReadStateSnapshot(
2426

2527
/// Hard bounds of the version-1 atomic snapshot protocol.
2628
const readStateSnapshotMaxBytes = 8 * 1024 * 1024;
29+
30+
/// The events plus room for the snapshot envelope around them.
31+
const readStateSnapshotMaxEnvelopeBytes = readStateSnapshotMaxBytes + 4096;
2732
const readStateSnapshotMaxEvents = 4096;
2833
final _hex64 = RegExp(r'^[0-9a-f]{64}$');
2934
final _hex128 = RegExp(r'^[0-9a-f]{128}$');
@@ -45,7 +50,13 @@ String readStateSnapshotCommunity(Object? descriptor) {
4550
/// Validates the entire snapshot atomically, on a worker isolate via compute.
4651
/// No decryption or read-state namespace selection occurs in this transport.
4752
List<NostrEvent> validateReadStateSnapshot(ReadStateSnapshotInput input) {
48-
final raw = jsonDecode(input.text);
53+
final raw = jsonDecode(
54+
relayResponseText(
55+
input.bytes,
56+
input.contentType,
57+
maxInflatedBytes: readStateSnapshotMaxEnvelopeBytes,
58+
),
59+
);
4960
if (raw is! Map<String, dynamic> ||
5061
raw['read_state_snapshot'] != 1 ||
5162
raw['complete'] != true ||

‎mobile/lib/shared/relay/relay_query_response.dart‎

Lines changed: 46 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -59,16 +59,58 @@ List<NostrEvent> parseRelayQueryEvents(Uint8List bytes, String? contentType) {
5959
/// passes through. JSON never starts with the gzip magic bytes, so the bytes
6060
/// decide, not the `Content-Encoding` header a transport may leave behind.
6161
/// The charset rules are those of [http.Response.body].
62-
String relayResponseText(Uint8List bytes, String? contentType) {
63-
final inflated = _isGzip(bytes)
64-
? Uint8List.fromList(gzip.decode(bytes))
65-
: bytes;
62+
///
63+
/// With [maxInflatedBytes], a gzip body that expands past it throws a
64+
/// [FormatException] as soon as it does, so a small compressed body cannot
65+
/// expand without limit. Transport byte caps only see the compressed size.
66+
String relayResponseText(
67+
Uint8List bytes,
68+
String? contentType, {
69+
int? maxInflatedBytes,
70+
}) {
71+
final inflated = _isGzip(bytes) ? _inflate(bytes, maxInflatedBytes) : bytes;
6672
return http.Response.bytes(
6773
inflated,
6874
200,
6975
headers: {'content-type': ?contentType},
7076
).body;
7177
}
7278

79+
Uint8List _inflate(Uint8List bytes, int? maxBytes) {
80+
if (maxBytes == null) return Uint8List.fromList(gzip.decode(bytes));
81+
final output = BytesBuilder(copy: false);
82+
final input = gzip.decoder.startChunkedConversion(
83+
_CappedByteSink(output, maxBytes),
84+
);
85+
// Small input chunks bound how far one chunk can overshoot the cap.
86+
const chunk = 4096;
87+
for (var start = 0; start < bytes.length; start += chunk) {
88+
final end = start + chunk < bytes.length ? start + chunk : bytes.length;
89+
input.add(Uint8List.sublistView(bytes, start, end));
90+
}
91+
input.close();
92+
return output.takeBytes();
93+
}
94+
95+
class _CappedByteSink implements Sink<List<int>> {
96+
_CappedByteSink(this._output, this._maxBytes);
97+
98+
final BytesBuilder _output;
99+
final int _maxBytes;
100+
101+
@override
102+
void add(List<int> data) {
103+
if (_output.length + data.length > _maxBytes) {
104+
throw FormatException(
105+
'relay response exceeds $_maxBytes bytes after inflating',
106+
);
107+
}
108+
_output.add(data);
109+
}
110+
111+
@override
112+
void close() {}
113+
}
114+
73115
bool _isGzip(Uint8List bytes) =>
74116
bytes.length > 2 && bytes[0] == 0x1f && bytes[1] == 0x8b;

‎mobile/lib/shared/relay/relay_session_snapshot.dart‎

Lines changed: 24 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ extension on RelaySessionNotifier {
2424
}
2525
}
2626

27-
Future<String> send(http.Request request, int maxBytes) async {
27+
Future<http.Response> send(http.Request request, int maxBytes) async {
2828
final response = await _httpQueryClient.sendBounded(
2929
request,
3030
maxBytes: maxBytes,
@@ -38,18 +38,33 @@ extension on RelaySessionNotifier {
3838
),
3939
);
4040
checkContext();
41-
final text = utf8.decode(response.bodyBytes);
4241
if (response.statusCode < 200 || response.statusCode >= 300) {
42+
final text = relayResponseText(
43+
response.bodyBytes,
44+
response.headers['content-type'],
45+
maxInflatedBytes: maxBytes,
46+
);
4347
_activateRateLimitGateFromHttpError(text);
4448
throw RelayException(response.statusCode, text);
4549
}
46-
return text;
50+
return response;
4751
}
4852

4953
final origin = Uri.parse(config.baseUrl);
5054
final discovery = http.Request('GET', origin.resolve('/'))
5155
..headers['Accept'] = 'application/nostr+json';
52-
final info = jsonDecode(await send(discovery, 64 * 1024));
56+
// The query transport leaves gzip bodies compressed (see
57+
// RelayHttpQueryClient), and the relay's CDN gzips both responses. The
58+
// inflated size is capped too: the transport cap only sees the
59+
// compressed bytes.
60+
final discovered = await send(discovery, 64 * 1024);
61+
final info = jsonDecode(
62+
relayResponseText(
63+
discovered.bodyBytes,
64+
discovered.headers['content-type'],
65+
maxInflatedBytes: 64 * 1024,
66+
),
67+
);
5368
final descriptor = info is Map<String, dynamic>
5469
? info['read_state_snapshot']
5570
: null;
@@ -76,10 +91,12 @@ extension on RelaySessionNotifier {
7691
nsec: nsec,
7792
),
7893
});
79-
final text = await send(request, readStateSnapshotMaxBytes + 4096);
80-
// Parsing and Schnorr verification must not block Flutter's UI isolate.
94+
final response = await send(request, readStateSnapshotMaxEnvelopeBytes);
95+
// Inflating, parsing and Schnorr verification must not block Flutter's
96+
// UI isolate.
8197
final events = await _snapshotDecoder((
82-
text: text,
98+
bytes: response.bodyBytes,
99+
contentType: response.headers['content-type'],
83100
viewer: viewer,
84101
community: community,
85102
));

‎mobile/test/shared/relay/read_state_snapshot_transport_test.dart‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
11
import 'dart:async';
22
import 'dart:convert';
3+
import 'dart:io';
34

45
import 'package:buzz/shared/auth/auth_provider.dart';
56
import 'package:buzz/shared/hosted/builderlab_account.dart';
67
import 'package:buzz/shared/hosted/hosted_relay_auth.dart';
8+
import 'package:buzz/shared/relay/read_state_snapshot.dart';
79
import 'package:buzz/shared/relay/relay.dart';
810
import 'package:flutter_test/flutter_test.dart';
911
import 'package:hooks_riverpod/hooks_riverpod.dart';
@@ -113,6 +115,56 @@ void main() {
113115
},
114116
);
115117

118+
test('inflates gzip discovery and snapshot bodies', () async {
119+
// The query transport keeps gzip bodies compressed, and the relay's CDN
120+
// gzips both responses. Decoding the raw bytes failed every snapshot.
121+
http.StreamedResponse gzipped(String body) => http.StreamedResponse(
122+
Stream.value(gzip.encode(utf8.encode(body))),
123+
200,
124+
headers: {'content-type': 'application/json', 'content-encoding': 'gzip'},
125+
);
126+
final h = harness(
127+
_StreamClient(
128+
(request) async => gzipped(
129+
request.method == 'GET' ? discovery : jsonEncode(envelope()),
130+
),
131+
),
132+
);
133+
final events = await h.session.fetchReadStateSnapshot();
134+
expect(events.single.id, event()['id']);
135+
});
136+
137+
test('rejects a gzip body that inflates past the snapshot cap', () async {
138+
// Transport caps see only compressed bytes; a few KB of gzip can expand
139+
// to many megabytes.
140+
final padded = utf8.encode(
141+
'${jsonEncode(envelope())}${' ' * 2 * readStateSnapshotMaxBytes}',
142+
);
143+
final h = harness(
144+
_StreamClient(
145+
(request) async => http.StreamedResponse(
146+
Stream.value(
147+
gzip.encode(
148+
request.method == 'GET' ? utf8.encode(discovery) : padded,
149+
),
150+
),
151+
200,
152+
headers: {'content-type': 'application/json'},
153+
),
154+
),
155+
);
156+
await expectLater(
157+
h.session.fetchReadStateSnapshot(),
158+
throwsA(
159+
isA<FormatException>().having(
160+
(error) => error.message,
161+
'message',
162+
contains('after inflating'),
163+
),
164+
),
165+
);
166+
});
167+
116168
test(
117169
'accepts complete empty state and unrelated own kind-30078 coordinates',
118170
() async {

0 commit comments

Comments
 (0)