Skip to content

Commit eefce94

Browse files
authored
Merge branch 'main' into tomb/mobile-read-state-live-no-replay
2 parents 4eff9e4 + 8cce12f commit eefce94

497 files changed

Lines changed: 62788 additions & 11865 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.env.example‎

Lines changed: 12 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -31,9 +31,6 @@ PGDATABASE=buzz
3131
# Redis 7
3232
# -----------------------------------------------------------------------------
3333
REDIS_URL=redis://localhost:6379
34-
# Max connections in the relay's shared Redis pool (default 16).
35-
# BUZZ_REDIS_POOL_SIZE=16
36-
3734
# Max connections in each of the relay's Postgres pools — writer and, when
3835
# READ_DATABASE_URL is set, reader (default 50).
3936
# BUZZ_DB_POOL_SIZE=50
@@ -188,6 +185,10 @@ BUZZ_PUSH_GATEWAY_DELIVERY_URL=https://push.buzz.xyz/v1/deliveries/apns
188185
# BUZZ_GIT_PACK_CACHE_MAX_BYTES=5368709120
189186
# BUZZ_GIT_PACK_CACHE_MAX_CONCURRENT_POPULATIONS=2
190187

188+
# Channel media revocation: off (default), shadow, enforce. Restart to change.
189+
# Apply migration 0086 first; see docs/operator-private-media.md.
190+
BUZZ_MEDIA_READ_MODE=off
191+
191192
# -----------------------------------------------------------------------------
192193
# S3-Compatible Object Storage (media + Git/CAS)
193194
# -----------------------------------------------------------------------------
@@ -473,6 +474,11 @@ RUST_LOG=buzz_relay=debug,buzz_datastore=info,buzz_db=debug,buzz_auth=debug,buzz
473474
# key and NIP-FI enforcement. Profiles are historical records and are never
474475
# retracted.
475476

476-
# New irreversible community FI acceptances stay disabled until the fleet's
477-
# enforcement and activation baseline is verified. Disabling preserves receipts.
478-
BUZZ_NIP_FI_UPGRADES_ENABLED=false
477+
# Permit community FI policy enables and disables only after the whole fleet
478+
# supports policy management. A closed gate preserves receipt recovery/status.
479+
BUZZ_NIP_FI_CHANGES=false
480+
481+
# NIP-CL compact membership discovery. Deploy fleet-wide dark before enabling.
482+
BUZZ_MEMBERSHIP_SNAPSHOT_ENABLED=false
483+
# Independent listed-open public membership lookup; also deploy disabled.
484+
BUZZ_MEMBERSHIP_PUBLIC_ENABLED=false

‎.github/workflows/_ci-clients.yml‎

Lines changed: 13 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -137,17 +137,27 @@ jobs:
137137
- name: Test native secp256k1 package
138138
if: matrix.lane == 'checks'
139139
run: cd mobile/packages/buzz_secp256k1 && dart pub get && dart test
140-
- name: Test configured authentication domains
140+
- name: Test internal build
141141
if: matrix.lane == 'checks'
142142
run: >-
143143
cd mobile && flutter test
144144
--dart-define=BUZZ_PUSH_GATEWAY_URL=https://push.example
145-
--dart-define=BUZZ_AUTH_DOMAINS=buzz.xyz,builderlab.xyz,builderlabstaging.xyz
145+
--dart-define=BUZZ_DISTRIBUTION=internal
146+
--dart-define=BUZZ_BUILDERLAB_URL=https://id.example.org
147+
--dart-define=BUZZ_BUILDERLAB_RELAYS=buzz.example.org
148+
test/features/onboarding/internal_build_entry_test.dart
149+
- name: Test public build
150+
if: matrix.lane == 'checks'
151+
run: >-
152+
cd mobile && flutter test
153+
--dart-define=BUZZ_PUSH_GATEWAY_URL=https://push.example
154+
--dart-define=BUZZ_DISTRIBUTION=public
155+
--dart-define=BUZZ_BUILDERLAB_URL=https://app.builderlab.xyz
156+
--dart-define=BUZZ_BUILDERLAB_RELAYS=communities.buzz.xyz
146157
test/shared/auth/community_entry_requirements_test.dart
147158
test/features/agents/agent_creation_link_journey_test.dart
148159
test/features/invites/invite_join_provider_test.dart
149160
test/features/onboarding/onboarding_entry_routes_test.dart
150-
test/shared/hosted/hosted_environment_test.dart
151161
test/shared/hosted/self_hosted_entry_test.dart
152162
test/features/agents/agents_signed_out_prompt_test.dart
153163
test/features/agents/agents_restore_prompt_test.dart

‎.github/workflows/_ci-relay.yml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -122,6 +122,7 @@ jobs:
122122
--cargo-profile ci \
123123
-p buzz-db \
124124
-p buzz-relay \
125+
-p buzz-relay-mesh \
125126
-p buzz-test-client \
126127
--lib \
127128
--test e2e_event_reminder \
@@ -574,6 +575,14 @@ jobs:
574575
--archive-file target/ci/backend-integration-tests.tar.zst \
575576
-E 'package(buzz-relay) and test(/^api::nip_fi::route_integration_tests::external_infra_redis::/)' \
576577
--run-ignored ignored-only
578+
- name: Mesh ready-registry index regressions
579+
env:
580+
BUZZ_TEST_REDIS_URL: redis://localhost:6379
581+
run: |
582+
cargo nextest run \
583+
--archive-file target/ci/backend-integration-tests.tar.zst \
584+
-E 'package(buzz-relay-mesh) and test(/^registry::tests::external_infra_redis::/)' \
585+
--run-ignored ignored-only
577586
- name: Workflow message provenance unit tests
578587
# The relay's workflow_sink suite is not selected by the infra-free
579588
# unit job. Its ignored database cases run in the isolated PostgreSQL
@@ -654,6 +663,8 @@ jobs:
654663
merge-multiple: true
655664
path: target/ci
656665
- name: Start relay
666+
env:
667+
BUZZ_MEDIA_READ_MODE: enforce
657668
run: |
658669
chmod +x ./target/ci/buzz-relay ./target/ci/git-credential-nostr
659670
./scripts/start-relay-for-tests.sh --no-build

‎.github/workflows/auto-tag-on-release-pr-merge.yml‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,10 @@ name: Auto-tag on Release PR Merge
1111
# any internal PR that bumps deploy/charts/buzz/Chart.yaml `version`
1212
# → tag chart-v<v> → helm-chart.yml (helm chart)
1313
#
14-
# Mobile candidate tags do not come from merged PRs. Operators create immutable
15-
# mobile-v<v>-rc.N tags directly from remote main with scripts/mobile-release.sh,
16-
# then hand the exact tag to buzz-releases.
14+
# Mobile candidate tags do not come from merged PRs or from this App. Release
15+
# managers push immutable mobile-v<v>-rc.N tags themselves with
16+
# scripts/mobile-release.sh, gated by a dedicated mobile tag ruleset whose only
17+
# bypass is the mobile release team, then hand the exact tag to buzz-releases.
1718
#
1819
# Release tags are created with a short-lived token from the dedicated
1920
# buzz-release-bot GitHub App. GitHub attributes the ref creation to that

‎.github/workflows/ci.yml‎

Lines changed: 1 addition & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -106,13 +106,10 @@ jobs:
106106
- 'scripts/mobile-release-source.sh'
107107
- 'scripts/mobile-worktree-overrides.sh'
108108
- 'scripts/mobile-worktree-clean.sh'
109-
- 'scripts/publish-mobile-release-candidate.sh'
110109
- 'scripts/release-rulesets.sh'
111110
- 'scripts/test-mobile-release-contract.sh'
112-
- 'scripts/test-mobile-release-candidate-publisher.sh'
113111
- 'scripts/test-mobile-worktree-overrides.sh'
114112
- 'scripts/ci-hermit-retry.sh'
115-
- '.github/workflows/mobile-release-candidate.yml'
116113
- '.github/workflows/ci.yml'
117114
- '.github/workflows/_ci-*.yml'
118115
- name: Select runtime suites conservatively at the PR file-list limit
@@ -136,9 +133,7 @@ jobs:
136133
scripts/test-oss-desktop-promotion.sh
137134
scripts/test-oss-desktop-promotion-behavior.sh
138135
- name: Mobile release contract
139-
run: |
140-
scripts/test-mobile-release-contract.sh
141-
scripts/test-mobile-release-candidate-publisher.sh
136+
run: scripts/test-mobile-release-contract.sh
142137
- name: Desktop instance environment contract
143138
run: scripts/test-desktop-instance-detection.sh
144139
- name: Mobile worktree identity contract

‎.github/workflows/mobile-release-candidate.yml‎

Lines changed: 0 additions & 75 deletions
This file was deleted.

‎AGENTS.md‎

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -73,26 +73,26 @@ confirmation, not app runs.
7373
- Agents post reviews as comments, never Request Changes. Humans decide which
7474
findings must be fixed.
7575

76+
## Open-source boundary
77+
78+
This repository is public and contains the relay, NIP specifications, legacy desktop app, and mobile clients. Never commit values that only make sense inside Block: the employee-only bb-block community and its hosts (`block.builderlab.xyz`, `buzz.block.builderlab.xyz`, `*.communities.block.xyz`), private GitHub organizations or repositories, organization-specific SSH users, internal hosts, cloud accounts, IAM roles or buckets, internal ticket keys, `go/` links, internal Slack channels, or real employees' names and personal setups. Public Builderlab (`app.builderlab.xyz`, `*.communities.builderlab.xyz`) is supported and fine.
79+
80+
Distributor-specific values belong in the relevant release configuration: build-time settings for clients and deployment configuration for the relay. If a setting is missing, add a generic one; never hardcode a Block-only value or special-case a host or organization.
81+
82+
The rule covers functionality, not only identifiers. Every feature here must be generic: useful to any operator or distributor, with no knowledge of one company's internal services, workflows or identity setup. Functionality that only serves Block lives in Block's private repositories (release builds, deployment configuration, private plugins) and reaches this code only through generic settings or documented extension points that never name, detect or special-case a specific organization, plugin or service. Review judges a change's purpose and approach, not just its strings.
83+
84+
Tests, fixtures, docs, and examples use example values (`example.org`, `example-org`, fictional names).
85+
86+
Reviewers treat a violation as a blocking finding.
87+
7688
## Ecosystem
7789

78-
Buzz spans six repos. This one (`block/buzz`) is the OSS source for the relay, mobile, CLI, and the deprecated Buzz 0.X desktop app. [block/buzz-app](https://github.com/block/buzz-app) is the OSS source for the Buzz 1.0 desktop and web app. The others handle internal builds and deployment:
90+
This repository (`block/buzz`) is the OSS source for the relay, mobile, CLI, and the deprecated Buzz 0.X desktop app. [block/buzz-app](https://github.com/block/buzz-app) is the OSS source for the Buzz 1.0 desktop and web app. Block's internal builds and deployments live in private repositories.
7991

8092
| Repo | Purpose |
8193
|------|---------|
8294
| [block/buzz](https://github.com/block/buzz) | OSS source — relay, mobile app, CLI, agent harness, deprecated Buzz 0.X desktop app |
8395
| [block/buzz-app](https://github.com/block/buzz-app) | OSS source — Buzz 1.0 desktop and web app; all new desktop feature work |
84-
| [squareup/buzz-releases](https://github.com/squareup/buzz-releases) | Buildkite pipelines producing Block-signed macOS + iOS builds with `-block` desktop version suffix |
85-
| [squareup/sprout-oss](https://github.com/squareup/sprout-oss) | CI pipeline building the relay Docker image and pushing to internal ECR |
86-
| [squareup/block-coder-tf-stacks](https://github.com/squareup/block-coder-tf-stacks) | Terraform + ArgoCD deploying the relay to the staging Kubernetes cluster |
87-
| [squareup/sprout-backend-blox](https://github.com/squareup/sprout-backend-blox) | Desktop backend provider script connecting Blox workstation agents to the relay |
88-
89-
```
90-
block/buzz (source)
91-
├─► buzz-releases (desktop + mobile builds → Artifactory, GitHub, Mobile Releases)
92-
├─► sprout-oss (relay Docker image → ECR)
93-
│ └─► block-coder-tf-stacks (Helm chart → ArgoCD → staging cluster)
94-
└─── sprout-backend-blox (Blox compute provider for Desktop agent launch)
95-
```
9696

9797
See [RELEASING.md](RELEASING.md) for the desktop release flow and
9898
[CONTRIBUTING.md § Ecosystem](CONTRIBUTING.md#ecosystem) for contributor

‎ARCHITECTURE.md‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -550,7 +550,7 @@ Subscriber → dedicated PubSub → PSUBSCRIBE buzz:channel:*
550550

551551
The subscriber uses a **dedicated** `redis::aio::PubSub` connection — not from the pool. This is intentional: pool connections cannot hold `PSUBSCRIBE` state.
552552

553-
**Current state:** The subscriber loop is spawned in `buzz-relay/src/main.rs` and populates the broadcast channel. A consumer task subscribes via `pubsub.subscribe_local()`, calls `sub_registry.fan_out()` on each received event, and delivers matches to local WebSocket connections via `conn_manager.send_to()`. Multi-node fan-out is now wired end-to-end. Local-echo deduplication is implemented via `AppState.local_event_ids` — events published by the local relay instance are tracked and skipped when received via the Redis round-trip.
553+
**Current state:** The subscriber loop is spawned in `buzz-relay/src/main.rs` and populates the broadcast channel. A dispatcher task subscribes via `pubsub.subscribe_local()` and routes each event to one of 16 delivery workers by `(community, topic)`, so one channel's events keep their order while channels deliver concurrently (`handlers/multinode_fanout.rs`). Each worker calls `sub_registry.fan_out()`, applies the access gate, and delivers matches to local WebSocket connections via `conn_manager.send_to()`. Live delivery never drops an event silently: when a worker's bounded queue is full, or an event is not delivered within 2 s of this node reading it from Redis, that event's local recipients get `CLOSED "resync: ..."` for syncs, are fenced from every further live EVENT frame as soon as the loss is detected (so a client's resume cursor stops moving), and are closed so clients catch up from their cursor; when the dispatcher itself lags the broadcast, every subscribed connection is treated the same way. Only the fence is immediate: closes are paced (at most 5,000/s per node) so a loss cannot become a reconnect storm. Clients resume from the newest `created_at` they received minus 5 s, so a closed client replays a missed event only while the 2 s deadline plus the event's writer clock skew and publish delay stays under that 5 s; an ordinary reconnect has the same limit. A dispatcher that falls behind without overflowing the broadcast detects a loss late, and dispatcher lag loses events it never sees; neither is bounded by the deadline. Multi-node fan-out is now wired end-to-end. Local-echo deduplication is implemented via `AppState.local_event_ids` — events published by the local relay instance are tracked and skipped when received via the Redis round-trip.
554554

555555
**Reconnection:** exponential backoff 1s → 30s (`backoff_secs * 2`). Backoff resets to 1s only after a clean stream end, not on each reconnect attempt.
556556

@@ -579,6 +579,12 @@ multi-community mode every query filter includes `community_id`, so the shared
579579
`events` table is infrastructure, not a cross-community result space; the relay
580580
re-authorizes every candidate hit before returning it.
581581

582+
A database built from `migrations/0008_fresh_install_search_allowlist.sql` indexes
583+
only an allowlist of kinds, and so does one converted by
584+
`scripts/maintenance/nip_rs_search_allowlist.sql`. Such a database does not index
585+
kind 30180 (custom emoji entries) until an operator runs the updated maintenance
586+
script, which must happen before the emoji migration rollout.
587+
582588
**Key behaviors:**
583589
- `SearchService::new(pool)` wraps a `PgPool`; `search(&SearchQuery)` runs a
584590
parameterized FTS query against the `events.search_tsv` GIN index and returns

‎CONTEXT.md‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Buzz domain language
22

3-
Shared terminology for Buzz product behavior. This glossary currently covers mobile [push suppression](#push-suppression); it is not a complete model of Buzz.
3+
Shared terminology for Buzz product behavior. This glossary currently covers mobile [push suppression](#push-suppression) and [mobile builds](#mobile-builds); it is not a complete model of Buzz.
44

55
## Notifications
66

@@ -51,3 +51,17 @@ A [suppression activity signal](#suppression-activity-signal) that extends how l
5151
### Inactivity cutoff
5252

5353
The time of the last observed [desktop activity](#desktop-activity) plus the duration specified in the spec, when [desktop inactivity](#desktop-inactivity) begins. New desktop activity moves this cutoff forward.
54+
55+
## Mobile builds
56+
57+
### Internal build
58+
59+
A mobile app an organization distributes only to its own members, for example through device management. It uses only that organization's Builderlab and reaches only that organization's community.
60+
61+
_Avoid_: Dogfood build, enterprise build, org build
62+
63+
### Public build
64+
65+
The mobile app distributed through the App Store and TestFlight. It uses only the public Builderlab and cannot reach any organization's community.
66+
67+
_Avoid_: App Store build, external build

0 commit comments

Comments
 (0)