Skip to content

Commit 20a778e

Browse files
matt2eclaude
andauthored
feat(communities): sync known communities through Builderlab accounts (#644)
Signed-off-by: Matt Toohey <[email protected]> Co-authored-by: Claude Fable 5.1 <[email protected]>
1 parent b7bcb1d commit 20a778e

24 files changed

Lines changed: 2285 additions & 34 deletions

‎docs/communities.md‎

Lines changed: 127 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,8 @@
33
The app opens into Personal space with no joined community. The shell, local
44
profile editor, Home, Settings and plugin management do not wait for a relay.
55
`features/communities/service.ts` owns the local identity's default profile,
6-
saved memberships and optional selection. `features/relay/session.ts` still owns
6+
saved memberships, optional selection and the [known-community sync
7+
state](#known-communities). `features/relay/session.ts` still owns
78
each community's queries, live subscriptions, projections and durable outbox.
89

910
## Try it
@@ -114,6 +115,86 @@ empty state there. Try drafting in A, switching to B, then returning to A.
114115
Selected channels, drafts and reading offsets are partitioned by the canonical
115116
community origin and viewer; channel IDs alone are not sufficient keys.
116117

118+
## Known communities
119+
120+
The device record also carries which saved destinations the account service
121+
last reported holding and the uploads still owed to it, in the same single
122+
write as the memberships, so a join or leave and its upload intent cannot be
123+
split by a failed or interrupted save. Destinations are addressed the way the
124+
service spells them, `wss://host[:port]` with a lowercase host, no default
125+
port and no trailing slash; a configured alias resolves to its origin first and
126+
never leaves the device. Records from before this field existed, or a field
127+
this reader cannot understand, read as empty, and a malformed entry is dropped
128+
on its own.
129+
130+
The account's list is a set: the service's add and remove are idempotent, so
131+
the outbox holds at most one intent per destination, the latest. Joining queues
132+
an add and leaving a removal, and a newer intent replaces the older one
133+
outright, since whichever edit lands last is the state. An intent the latest
134+
word already satisfies (the pending intent, or the service holding the
135+
destination) queues nothing, so re-running a join the service already holds
136+
sends nothing; removing a destination the service is not known to hold still
137+
queues, in case another device saved it. An accepted upload moves the
138+
destination into or out of the known set and drops the intent, unless a
139+
different one has replaced it meanwhile, which is then sent in turn. This is
140+
what makes a lost answer safe: the retry simply sends whatever intent stands,
141+
so joining then leaving (or leaving then rejoining) across a lost
142+
acknowledgement converges on the latest intent. A complete list from the
143+
service adds destinations saved on another device under their host name
144+
without opening a session, and queues an upload for saved memberships the
145+
service has never seen, which is how a device list from before sync existed
146+
reaches the account; a pending intent for a destination speaks for it in that
147+
merge. Removals made on another device are ignored for now, a deliberate
148+
interim choice: a saved membership the service knew but no longer lists stays
149+
on this device, and sync never forgets a membership. So that this device does
150+
not undo the other's removal by uploading the membership again, the known set
151+
is cumulative for its account (everything the service has listed or accepted
152+
an add for, less the removals it accepted from here) rather than the last list:
153+
the omitted membership stays known, and so off the queue, on every later
154+
sign-in. Leaving it here still propagates. Until removals are followed, the
155+
devices of one account can disagree about a community, and an add queued while
156+
the service was unreachable can likewise land after another device's removal.
157+
Joining and leaving never wait for sync. The `knownCommunities` capability
158+
exposes the record, the queue and these writes to one owner, the bundled
159+
Builderlab plugin.
160+
161+
The list is account-scoped, but this state is still kept under the device
162+
record's viewer key (`buzz-client.v1:<viewer>`), where it outlives a
163+
Builderlab sign-out. So the record also names the account whose list the
164+
known set describes, and a merge for a different account (or for a record
165+
that names none) starts from nothing known, with the outbox kept, so every
166+
saved membership the new account's list lacks is queued for it. Signing back
167+
in as the same account keeps the set, so a membership it once held and no
168+
longer lists is still neither forgotten nor uploaded again. Moving the state
169+
to live with the Builderlab account is a follow-up;
170+
until then a device whose identity changes starts from an empty record and
171+
re-merges on its next sign-in.
172+
173+
The plugin starts a sync owner only where signing in is possible: the native
174+
app with a configured Builderlab service. The owner syncs only while signed in.
175+
Each sign-in reads the complete list, merges it as above, then sends each
176+
destination's latest intent one at a time. A newly queued intent, the window
177+
coming online or becoming visible runs the drain again at once; once the list
178+
has merged, a trigger never reads it again before the next sign-in. A failed
179+
request, or a device record that would not take the answer, retries at 1, 2,
180+
4… seconds, capped at a minute, re-sending the intent that then stands; a
181+
timeout, a rate limit and a server error are such failures. The service's
182+
refusals are never retried as sent: an address it refuses, an account that is
183+
full or any other client error parks that one intent until it is replaced or
184+
the next sign-in while the rest continue, an
185+
account that cannot sync or a list this client cannot read (a shape the two
186+
sides no longer share) stops until then, and a session the service has ended
187+
signs the plugin out. A service that answers any of the routes with 404 has no
188+
known-communities routes yet, or not all of them: sync stops until the next
189+
sign-in. The owner reports nothing to the app, so a halted or parked sync is
190+
not shown anywhere yet; each refusal is named once in a console warning (the
191+
route, the address and the reason), since an account without the routes'
192+
server capability is refused on every sign-in and would otherwise leave no
193+
trace on the device. A listed address this device could never save is left
194+
out and counted the same way, so one bad row cannot stop the account's sync.
195+
Signing out or disabling the plugin abandons the request in flight and leaves
196+
the queue intact.
197+
117198
## Session lifetime
118199

119200
Selecting a saved membership lazily acquires its session. Sessions survive page
@@ -234,7 +315,47 @@ leaving (membership removal, Personal space fallback, session disposal, purged
234315
device state, a leave that keeps device state for the banned answer, the last
235316
community, a purge that leaves named failures, and a native device record that
236317
will not save, which throws before the snapshot, session, record or device state
237-
change). `CommunityPicker.test.tsx` covers the leave flow end to end against the
318+
change), that join and leave each write the membership change and its queued
319+
upload in one record write which a restart restores, applying a server list
320+
(additions under the host name, nothing forgotten, one record write) and a
321+
native sync write that will not save.
322+
It also covers that re-running a join the service already holds queues nothing
323+
while a join it does not hold queues an add.
324+
`known-communities.test.ts` covers the queue rules on their own: one latest
325+
intent per destination with a newer intent replacing the older, nothing queued
326+
when the latest word already agrees, acknowledgements moving a destination in
327+
and out of the known set while keeping an intent that replaced the settled one,
328+
both lost-acknowledgement orders converging on the latest intent as a plain
329+
retry, every list-merge branch including the omitted membership kept and not
330+
uploaded again, the pre-sync migration upload and deference to the pending
331+
intent, and the
332+
tolerant reader of saved state.
333+
`src/bundled/builderlab/known-communities/client.test.ts` covers the account
334+
service's three routes against the native host transport: the exact request
335+
shapes, the device's spelling of listed addresses, omitted repeated fields,
336+
unsaveable addresses dropped and counted, rows of another shape and a success
337+
without a JSON object body answered as an invalid response, every named refusal
338+
in JSON and in the framework's plain text, unnamed client errors returned as
339+
rejections on every route while timeouts, rate limits and server errors remain
340+
failures, a 401 that ends the session and an unreachable service.
341+
`sync.test.ts` covers the owner under a controlled clock: sign-in and a
342+
loading record, the complete list merged once per sign-in and an intent queued
343+
while it is in flight, the never-uploaded rule on a later sign-in and a
344+
membership removed elsewhere kept and never re-sent across the sign-ins after
345+
it, one upload in flight at a time in queue order, identical retries at each
346+
backoff boundary and the reset after a success, a device record that will not
347+
take an answer backing off and replaying, the `online` and `visibilitychange`
348+
triggers and no second list once synced, parked and halting refusals
349+
including rejections by status, an unreadable list (through the real client,
350+
keeping every membership) and the 404 on the list or an upload for a service
351+
without the routes, addresses the list left out, a parked add replaced by the
352+
removal that then goes out and re-sent on the next sign-in, a withdrawn parked
353+
intent sent when queued anew, both lost-acknowledgement orders against a set
354+
service, an abandoned upload on sign-out that keeps the queue, a session ended
355+
by the service and disposal. `index.test.tsx` runs the real plugin wiring
356+
through to the list request under the signed-in credential, and that an
357+
unconfigured build starts no owner.
358+
`CommunityPicker.test.tsx` covers the leave flow end to end against the
238359
broker route: confirm, publish, then remove; cancel; refusals and timeouts that
239360
keep the membership; the not-a-member answers (purging) and the banned answer
240361
(keeping device state); a device record that will not save after the relay
@@ -246,8 +367,8 @@ per-store failure report. The broker and native adapter tests sign the exact
246367
leave shape and pass through only the relay's known refusals.
247368
`broker.test.ts` runs real localhost HTTP with signed fixture events to verify
248369
multi-community routing, profile publication, invite claims, and a captured send
249-
after opening another community. `destination.test.ts` checks normalization and
250-
rejection; `broker-url.test.ts` exercises the real middleware with isolated signing
370+
after opening another community. `destination.test.ts` checks normalization,
371+
rejection and the account service's `wss://` address; `broker-url.test.ts` exercises the real middleware with isolated signing
251372
keys and upstream fixtures, including registration, cross-origin guards, all route
252373
sinks and captured sends. Service tests cover arbitrary membership persistence,
253374
selected-only restore, retry registration and equivalent-URL selection. Existing relay tests cover connection generations,
@@ -256,7 +377,8 @@ late responses, delivery and revocation.
256377
`native-join.test.tsx` mounts the real dialog, community service and native adapter
257378
with fixture IPC to cover claim/profile response loss, acknowledged but superseded
258379
or missing profiles, read and persistence failures, interrupted setup, alias
259-
recovery and selected-only restart. `join-journal.test.ts` covers alias addition,
380+
recovery, selected-only restart and the completed join's queued upload in its
381+
device record. `join-journal.test.ts` covers alias addition,
260382
removal and unresolved legacy records across restarts. `native-api.test.ts`
261383
and `relay/native.test.ts` cover routing, verification, live auth, capacity,
262384
receipt correlation and expired-event readback. `app/services.test.ts` exercises

‎src/app/CommunityAddress.test.tsx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ const communities: CommunityReader = {
2020
profile: { name: "", picture: "" },
2121
memberships: [],
2222
selected: null,
23+
sync: { known: [], outbox: [] },
2324
}),
2425
};
2526

‎src/app/ProfileSettings.avatar.test.tsx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,7 @@ function setup(
5353
status: "ready",
5454
relayAvailable: true,
5555
viewer: "a".repeat(64),
56+
sync: { known: [], outbox: [] },
5657
selected,
5758
memberships: [
5859
{ id: a, name: "Alpha" },

‎src/app/Settings.test.tsx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -119,6 +119,7 @@ const communityState = {
119119
profile: { name: "Buzz User", picture: "" },
120120
memberships: [{ id: "primary", name: "Primary" }],
121121
selected: "primary",
122+
sync: { known: [], outbox: [] },
122123
};
123124
const host = {
124125
plugins: { subscribe: () => () => {}, snapshot: () => pluginState },

‎src/app/services.test.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -262,6 +262,7 @@ it("seeds and persists the configured relay through the real app composition", a
262262
profile: { name: "", picture: "", about: "" },
263263
memberships: [membership],
264264
selected: membership.id,
265+
sync: { known: [], outbox: [] },
265266
});
266267
});
267268

@@ -481,6 +482,7 @@ it("seeds a packaged native community after onboarding and preserves the user's
481482
profile: { name: "", picture: "", about: "" },
482483
memberships: [membership],
483484
selected: membership.id,
485+
sync: { known: [], outbox: [] },
484486
});
485487
// A later build's default cannot change the saved selection.
486488
await services.dispose();

‎src/app/services.ts‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,12 @@ export function createServices() {
7676
snapshot: communities.snapshot,
7777
subscribe: communities.subscribe,
7878
});
79+
ctx.provide("knownCommunities", {
80+
snapshot: communities.snapshot,
81+
subscribe: communities.subscribe,
82+
pending: communities.pendingSync,
83+
apply: communities.applySync,
84+
});
7985
const relay = communities.relay;
8086
const adminAccess = createAdminAccess(relay, communities, nativeRelayStaff());
8187
ctx.provide("adminAccess", adminAccess);

‎src/bundled/builderlab/README.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -100,6 +100,21 @@ agents are not migrated into this type.
100100
name. Retry the same name after an uncertain failure within the server's
101101
seven-day replay window. Instructions are limited to 20,000 characters,
102102
affect newly accepted tasks and are never published to the relay.
103+
- `known-communities/` keeps the account's community list in step with this
104+
device through the `knownCommunities` capability while signed in, in native
105+
builds with a configured service. The list is an account-scoped set behind
106+
three idempotent JSON routes under `/v1/buzz/known-communities/`: `list`
107+
(`{}` → `{ communities: [{ community_url }] }`), `add` and `remove` (each
108+
`{ community_url }`; adding a saved destination or removing an absent one
109+
succeeds, and adds past about 1,000 saved destinations are refused with
110+
`limit_reached`). Each sign-in merges the complete list (adding what it
111+
lists; a saved membership it omits is kept and not uploaded again, since
112+
removals made on another device are ignored for now), then sends each
113+
destination's latest queued join or leave one at a time, retrying failures
114+
as sent. A list this client cannot read halts sync until the next sign-in,
115+
as does a 404 on any route (not yet deployed); a listed address the device
116+
could never save is dropped and counted, never a halt. See
117+
[communities](../../../docs/communities.md#known-communities).
103118

104119
The plugin uses the `BUZZ_BUILDERLAB_URL` [build input](../../../docs/configuration.md#builderlab-url-build-input)
105120
as its server address and appends `/api/goose`.

‎src/bundled/builderlab/agents/community-testing.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,7 @@ export function communityFixture(
7777
viewer,
7878
selected,
7979
memberships: [],
80+
sync: { known: [], outbox: [] },
8081
profile: { name: "", picture: "", about: "" },
8182
};
8283
const listeners = new Set<() => void>();

0 commit comments

Comments
 (0)