Skip to content

Commit cb65ce1

Browse files
tlongwell-blockEvaMariSamiMax
authored
Multi-tenant Buzz relay: community_id as a server-resolved key (comprehensive rewrite) (block#1321)
Signed-off-by: tlongwell-block <[email protected]> Signed-off-by: npub1jh9wn95s0472h86ahapupaf7m6kx4v9sx2n0atj2hltcfer8k06s5n3pyf <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@sprout-oss.stage.blox.sqprod.co> Signed-off-by: Tyler Longwell <[email protected]> Signed-off-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@sprout-oss.stage.blox.sqprod.co> Signed-off-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Mari <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Sami <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Max <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Quinn <96f056ad5f2305c8ddf637dc65d048aa4c12d7daeb8867690e34fca46b0ef64c@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Dawn <c6237ef84fa537c78dcee78efd2d4e59f728859c7f194da42ac51ededfa0be05@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Tyler Longwell <[email protected]> Co-authored-by: Sami <[email protected]> Co-authored-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@sprout-oss.stage.blox.sqprod.co>
1 parent a82ca21 commit cb65ce1

201 files changed

Lines changed: 23390 additions & 15412 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/ci.yml‎

Lines changed: 56 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -305,7 +305,7 @@ jobs:
305305
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
306306
- uses: cashapp/activate-hermit@e49f5cb4dd64ff0b0b659d1d8df499595451155a # v1
307307
- name: Start integration services
308-
run: docker compose up -d postgres redis typesense minio minio-init
308+
run: docker compose up -d postgres redis minio minio-init
309309
- name: Get pnpm store directory
310310
id: pnpm-cache
311311
run: echo "STORE_PATH=$(pnpm store path --silent)" >> "$GITHUB_OUTPUT"
@@ -357,22 +357,45 @@ jobs:
357357
}
358358
wait_healthy "Postgres" "buzz-postgres"
359359
wait_healthy "Redis" "buzz-redis"
360-
wait_healthy "Typesense" "buzz-typesense"
361360
wait_healthy "MinIO" "buzz-minio"
362361
- name: Download relay binary
363362
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
364363
with:
365364
name: desktop-e2e-relay
366365
path: target/ci
366+
- name: Apply schema and seed deployment community
367+
# MT: the relay resolves each request's tenant from the communities host
368+
# map and fails closed on an unmapped host. The channel reconciler binds
369+
# the deployment community ONCE at boot (outside its retry loop) and
370+
# exits permanently on an unmapped host, so the 'localhost:3000'
371+
# community MUST exist before the relay starts — the retry loop only
372+
# handles late-seeded channels, not a late-seeded community. The relay
373+
# migrates at boot via BUZZ_AUTO_MIGRATE, but that's too late for the
374+
# pre-boot seed, so apply the schema here first (then drop AUTO_MIGRATE
375+
# below). lower(host) is the unique index → ON CONFLICT target. psql
376+
# isn't on PATH in hermit → exec into the buzz-postgres container.
377+
env:
378+
PGHOST: localhost
379+
PGPORT: "5432"
380+
PGUSER: buzz
381+
PGPASSWORD: buzz_dev
382+
PGDATABASE: buzz
383+
run: |
384+
./bin/pgschema apply --file schema/schema.sql --auto-approve
385+
docker exec -i -e PGPASSWORD=buzz_dev buzz-postgres \
386+
psql -U buzz -d buzz -v ON_ERROR_STOP=1 < scripts/attach-schema-partitions.sql
387+
docker exec -e PGPASSWORD=buzz_dev buzz-postgres \
388+
psql -U buzz -d buzz -qtA -c "
389+
INSERT INTO communities (id, host)
390+
VALUES ('00000000-0000-4000-8000-00000000c0de', 'localhost:3000')
391+
ON CONFLICT (lower(host)) DO NOTHING
392+
;"
367393
- name: Start relay
368394
run: |
369395
chmod +x ./target/ci/buzz-relay
370396
nohup env \
371397
DATABASE_URL=postgres://buzz:buzz_dev@localhost:5432/buzz \
372-
BUZZ_AUTO_MIGRATE=true \
373398
REDIS_URL=redis://localhost:6379 \
374-
TYPESENSE_URL=http://localhost:8108 \
375-
TYPESENSE_API_KEY=buzz_dev_key \
376399
RELAY_URL=ws://localhost:3000 \
377400
BUZZ_BIND_ADDR=0.0.0.0:3000 \
378401
BUZZ_REQUIRE_AUTH_TOKEN=false \
@@ -448,7 +471,7 @@ jobs:
448471
with:
449472
save-if: ${{ github.event_name != 'pull_request' }}
450473
- name: Start integration services
451-
run: docker compose up -d postgres redis typesense minio minio-init
474+
run: docker compose up -d postgres redis minio minio-init
452475
- name: Wait for integration services
453476
run: |
454477
wait_healthy() {
@@ -467,22 +490,45 @@ jobs:
467490
}
468491
wait_healthy "Postgres" "buzz-postgres"
469492
wait_healthy "Redis" "buzz-redis"
470-
wait_healthy "Typesense" "buzz-typesense"
471493
wait_healthy "MinIO" "buzz-minio"
472494
- name: Download relay binary
473495
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
474496
with:
475497
name: desktop-e2e-relay
476498
path: target/ci
499+
- name: Apply schema and seed deployment community
500+
# MT: the relay resolves each request's tenant from the communities host
501+
# map and fails closed on an unmapped host. The reminder scheduler binds
502+
# the deployment community ONCE at boot and exits permanently on an
503+
# unmapped host (no retry, unlike the channel reconciler), so the
504+
# 'localhost:3000' community MUST exist before the relay starts — seeding
505+
# after boot leaves the scheduler dead. The relay migrates at boot via
506+
# BUZZ_AUTO_MIGRATE, but that's too late for the pre-boot seed, so apply
507+
# the schema here first (then drop AUTO_MIGRATE below). lower(host) is the
508+
# unique index → ON CONFLICT target. psql isn't on PATH in hermit → exec
509+
# into the buzz-postgres container.
510+
env:
511+
PGHOST: localhost
512+
PGPORT: "5432"
513+
PGUSER: buzz
514+
PGPASSWORD: buzz_dev
515+
PGDATABASE: buzz
516+
run: |
517+
./bin/pgschema apply --file schema/schema.sql --auto-approve
518+
docker exec -i -e PGPASSWORD=buzz_dev buzz-postgres \
519+
psql -U buzz -d buzz -v ON_ERROR_STOP=1 < scripts/attach-schema-partitions.sql
520+
docker exec -e PGPASSWORD=buzz_dev buzz-postgres \
521+
psql -U buzz -d buzz -qtA -c "
522+
INSERT INTO communities (id, host)
523+
VALUES ('00000000-0000-4000-8000-00000000c0de', 'localhost:3000')
524+
ON CONFLICT (lower(host)) DO NOTHING
525+
;"
477526
- name: Start relay
478527
run: |
479528
chmod +x ./target/ci/buzz-relay
480529
nohup env \
481530
DATABASE_URL=postgres://buzz:buzz_dev@localhost:5432/buzz \
482-
BUZZ_AUTO_MIGRATE=true \
483531
REDIS_URL=redis://localhost:6379 \
484-
TYPESENSE_URL=http://localhost:8108 \
485-
TYPESENSE_API_KEY=buzz_dev_key \
486532
RELAY_URL=ws://localhost:3000 \
487533
BUZZ_BIND_ADDR=0.0.0.0:3000 \
488534
BUZZ_REQUIRE_AUTH_TOKEN=false \

‎AGENTS.md‎

Lines changed: 10 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ crates/
4242
buzz-db # Postgres event store and data access layer
4343
buzz-auth # Authentication and authorization
4444
buzz-pubsub # Redis pub/sub fan-out, presence, typing indicators
45-
buzz-search # Typesense-backed full-text search
45+
buzz-search # Postgres FTS full-text search
4646
buzz-audit # Hash-chain audit log
4747
buzz-media # Blossom/S3 media storage
4848
# Agent surface
@@ -52,7 +52,6 @@ crates/
5252
buzz-persona # Agent persona packs
5353
buzz-workflow # YAML-as-code workflow engine (evalexpr conditions)
5454
# Clients + interop
55-
buzz-proxy # Nostr client compatibility proxy (NIP-28)
5655
buzz-pair-relay # Ephemeral sidecar relay for NIP-AB device pairing
5756
buzz-pairing-cli # CLI for NIP-AB device pairing interop testing
5857
git-sign-nostr # Sign git objects with a Nostr key
@@ -116,24 +115,21 @@ Additional rules:
116115

117116
## Key Patterns
118117

119-
**Dual API surface**: Buzz exposes both a REST API and a NIP-29 WebSocket
120-
relay. Both paths converge on shared DB functions in `buzz-db`. When adding
121-
a feature, implement the shared DB logic first, then wire up both surfaces.
118+
**Nostr-first HTTP surface**: Buzz's primary API is NIP-29 over WebSocket. The relay also exposes a narrow HTTP surface: NIP-11/NIP-05 metadata, `POST /events`, `POST /query`, `POST /count`, workflow webhooks at `/hooks/{id}`, Blossom media, git smart HTTP, git policy hooks, and health probes. These HTTP paths all preserve the same host-derived community boundary.
122119

123-
**Prefer Nostr events over new REST endpoints**: For new feature work, model
120+
**Prefer Nostr events over new HTTP endpoints**: For new feature work, model
124121
the operation as a Nostr event (new kind in `buzz-core/src/kind.rs`, handler
125-
in `buzz-relay`) rather than adding a new REST endpoint. REST is reserved
126-
for things that genuinely need an HTTP-only surface: media upload/download
127-
(Blossom), OAuth callbacks, health checks, and the existing read endpoints
128-
that proxy DB queries. Two helpful endpoints already exist and rarely need
129-
to be duplicated:
122+
in `buzz-relay`) rather than adding endpoint-specific JSON APIs. HTTP is
123+
reserved for things that genuinely need an HTTP-only surface: media upload/download
124+
(Blossom), webhooks, git smart HTTP, NIP-11/NIP-05 metadata, health checks,
125+
and the generic Nostr bridge endpoints:
130126

131127
- `POST /events` — submit any signed event (same path the WebSocket uses).
132128
- `POST /query` — Nostr REQ filters over HTTP. NIP-50 `search` filters
133-
are routed to `buzz-search` (Typesense-backed) automatically.
129+
are routed to `buzz-search` (Postgres FTS) automatically.
134130
- `POST /count` — Nostr COUNT filters over HTTP.
135131

136-
If you find yourself reaching for a new REST endpoint, first check whether
132+
If you find yourself reaching for a new HTTP endpoint, first check whether
137133
an event kind would do the job — it usually will, and you get realtime
138134
fan-out, NIP-29 scoping, and the existing auth pipeline for free.
139135

@@ -207,9 +203,6 @@ just test # full integration suite (requires Postgres + Redis)
207203

208204
E2E tests live in `crates/buzz-test-client/tests/`:
209205
- `e2e_relay.rs` — WebSocket relay protocol
210-
- `e2e_rest_api.rs` — REST endpoint coverage
211-
- `e2e_tokens.rs` — auth token flows
212-
- `e2e_workflows.rs` — workflow engine
213206
- `e2e_media.rs` — media upload/download (Blossom)
214207
- `e2e_media_extended.rs` — extended media scenarios
215208
- `e2e_nostr_interop.rs` — Nostr interop (NIP-50 search, NIP-10 threads, NIP-17 gift wraps)
@@ -554,7 +547,7 @@ just mobile-dev
554547

555548
## See Also
556549

557-
- [CONTRIBUTING.md](CONTRIBUTING.md) — setup, code style, PR process, how to add event kinds / CLI subcommands / API endpoints
550+
- [CONTRIBUTING.md](CONTRIBUTING.md) — setup, code style, PR process, how to add event kinds / CLI subcommands / HTTP endpoints
558551
- [TESTING.md](TESTING.md) — multi-agent E2E test guide
559552
- [ARCHITECTURE.md](ARCHITECTURE.md) — system design and component relationships
560553
- [RELEASING.md](RELEASING.md) — release process: `release-desktop`, `release-relay`, `release-mobile`, auto-tag, internal builds

0 commit comments

Comments
 (0)