Skip to content

Commit 17126af

Browse files
npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67dTyler Longwell
andcommitted
Wire mesh inbound consumers at boot + BUZZ_MESH_DEMO_ECHO smoke consumer
The dispatcher's register_* slots existed but had no callers: with BUZZ_MESH=on, inbound datagrams and session streams were accepted by the runtime and dropped at the dispatcher ("before handler registration"). This lands the boot wiring so consumers actually run live. - mesh_boot::wire_mesh_consumers (called once from main.rs via MeshHandle::wire_consumers, right after boot_mesh): * RealtimeMedia datagrams -> MeshAudioRouter::on_media_datagram, constructed WITH the handle's shared audio_fence (new MeshAudioRouter::with_fence) so the datagram hot path and huddle teardown (GenerationFloor::forget, Dawn's lane) enforce exactly one floor per session. Test pins the invariant end-to-end through the dispatcher. * HuddleControl streams -> HuddleControlAcceptor::accept_inbound (spawned; structural checks then Redis-fenced control loop). * ReliableStream streams -> ReliableStreamRouter::accept_inbound, then the demo echo consumer when BUZZ_MESH_DEMO_ECHO=on, else accept/log/close (no product session consumer wired yet). - BUZZ_MESH_DEMO_ECHO (config.mesh_demo_echo, strict opt-in like BUZZ_MESH): owner-side echo of every validated Data frame — a transport/session-routing smoke for cross-pod evidence runs; each echoed frame proves fenced validation (Redis directory included) and both mesh directions. Not a product flow. Owner-side lease renewal is intentionally NOT spawned here: spawn_observable_renewer attaches on the join path's ReliableJoin::Owned arm (the acquiring pod renews), which lands with the first product session consumer. Inbound acceptance never owns a lease. The huddle-lane renewer (Dawn's next increment) is likewise a seam, not wired. BUZZ_MESH=off remains byte-identical: boot_mesh returns None before any of this runs, and BUZZ_MESH_DEMO_ECHO defaults off. Validation: cargo fmt; clippy --all-targets clean; buzz-relay 513 passed / 2 ignored; buzz-relay-mesh 32 passed. Co-authored-by: Tyler Longwell <[email protected]> Signed-off-by: Tyler Longwell <[email protected]>
1 parent 3582fc7 commit 17126af

4 files changed

Lines changed: 258 additions & 1 deletion

File tree

‎crates/buzz-relay/src/audio/mesh.rs‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -166,9 +166,22 @@ impl MeshAudioRouter {
166166
/// Construct a router over this pod's rooms, tagged with the local runtime
167167
/// identity (used to distinguish owner vs non-owner delivery paths).
168168
pub fn new(rooms: Arc<AudioRoomManager>, local_runtime_id: RuntimeId) -> Self {
169+
Self::with_fence(rooms, local_runtime_id, Arc::new(GenerationFloor::new()))
170+
}
171+
172+
/// Construct a router that enforces an externally owned generation floor.
173+
///
174+
/// Used by the boot wiring (`mesh_boot::wire_mesh_consumers`) so the
175+
/// datagram hot path and session teardown (`GenerationFloor::forget`,
176+
/// reached via `MeshHandle::audio_fence`) consult exactly one floor.
177+
pub fn with_fence(
178+
rooms: Arc<AudioRoomManager>,
179+
local_runtime_id: RuntimeId,
180+
fence: Arc<GenerationFloor>,
181+
) -> Self {
169182
Self {
170183
rooms,
171-
fence: Arc::new(GenerationFloor::new()),
184+
fence,
172185
local_runtime_id,
173186
}
174187
}

‎crates/buzz-relay/src/config.rs‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -100,6 +100,14 @@ pub struct Config {
100100
/// no-regression rollout.
101101
pub mesh: buzz_relay_mesh::MeshConfig,
102102

103+
/// Testbed-only reliable-stream echo consumer (`BUZZ_MESH_DEMO_ECHO`).
104+
/// When `on`, the owner side of an inbound reliable mesh stream echoes
105+
/// every validated `Data` frame back to the sender — a transport/
106+
/// session-routing smoke for cross-pod evidence runs, NOT a product flow.
107+
/// Same strict opt-in as `BUZZ_MESH`; default off means inbound reliable
108+
/// streams are accepted, logged, and closed (no session consumer yet).
109+
pub mesh_demo_echo: bool,
110+
103111
/// Optional hex-encoded pubkey of the relay owner.
104112
/// When set, this pubkey is automatically bootstrapped into `relay_members`
105113
/// with the `owner` role on first startup.
@@ -264,6 +272,12 @@ impl Config {
264272
registry_refresh: std::time::Duration::from_secs(15),
265273
};
266274

275+
// Demo echo opt-in: same strict pattern as BUZZ_MESH — explicit
276+
// `on`/`true`/`1` only, anything else (absent, `off`, typos) is off.
277+
let mesh_demo_echo = std::env::var("BUZZ_MESH_DEMO_ECHO")
278+
.map(|v| v.eq_ignore_ascii_case("on") || v == "true" || v == "1")
279+
.unwrap_or(false);
280+
267281
let allow_nip_oa_auth = std::env::var("BUZZ_ALLOW_NIP_OA_AUTH")
268282
.map(|v| v == "true" || v == "1")
269283
.unwrap_or(false);
@@ -456,6 +470,7 @@ impl Config {
456470
require_relay_membership,
457471
huddle_audio_available,
458472
mesh,
473+
mesh_demo_echo,
459474
relay_owner_pubkey,
460475
allow_nip_oa_auth,
461476
media,

‎crates/buzz-relay/src/main.rs‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -348,6 +348,10 @@ async fn main() -> anyhow::Result<()> {
348348
.await?
349349
{
350350
let runtime_id = handle.local_runtime_id;
351+
// Register the per-profile inbound consumers (huddle datagram fan-in,
352+
// HuddleControl accept loop, reliable-stream accept + optional
353+
// BUZZ_MESH_DEMO_ECHO) before peers can route traffic here.
354+
handle.wire_consumers(Arc::clone(&state.audio_rooms), state.config.mesh_demo_echo);
351355
if state.mesh.set(handle).is_err() {
352356
unreachable!("mesh handle is set exactly once, right here");
353357
}

‎crates/buzz-relay/src/mesh_boot.rs‎

Lines changed: 225 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ use buzz_relay_mesh::{
3232

3333
use crate::config::Config;
3434
use crate::tunnel::directory::SessionDirectory;
35+
use crate::tunnel::reliable::{ReliableFrame, ReliableInbound, ReliableStreamRouter};
3536

3637
/// Handler for one inbound session-stream profile. Called on the accept task;
3738
/// implementations must hand off promptly (spawn) rather than block.
@@ -144,6 +145,13 @@ pub struct MeshHandle {
144145
/// dispatcher itself is already installed as the transport's single
145146
/// inbound slot by [`boot_mesh`].
146147
pub dispatcher: MeshInboundDispatcher,
148+
/// The huddle media generation floor shared by the datagram fan-in path
149+
/// and session teardown (`GenerationFloor::forget`). Owned here;
150+
/// [`wire_mesh_consumers`] constructs the [`MeshAudioRouter`] with this
151+
/// same `Arc` so both halves enforce exactly one floor per session.
152+
///
153+
/// [`MeshAudioRouter`]: crate::audio::mesh::MeshAudioRouter
154+
pub audio_fence: Arc<crate::audio::mesh::GenerationFloor>,
147155
/// The running mesh (status snapshots, shutdown).
148156
runtime: MeshRuntime,
149157
}
@@ -153,6 +161,154 @@ impl MeshHandle {
153161
pub fn status(&self) -> MeshStatus {
154162
self.runtime.membership().status()
155163
}
164+
165+
/// Register the per-profile inbound consumers on this handle's dispatcher.
166+
///
167+
/// Called once from `main.rs` right after [`boot_mesh`]; see
168+
/// [`wire_mesh_consumers`] for what gets wired.
169+
pub fn wire_consumers(&self, rooms: Arc<crate::audio::AudioRoomManager>, demo_echo: bool) {
170+
wire_mesh_consumers(
171+
&self.dispatcher,
172+
self.directory.clone(),
173+
Arc::clone(&self.transport),
174+
self.local_runtime_id,
175+
Arc::clone(&self.audio_fence),
176+
rooms,
177+
demo_echo,
178+
)
179+
}
180+
}
181+
182+
/// Wire the three inbound mesh lanes to their consumers.
183+
///
184+
/// - `RealtimeMedia` datagrams → [`MeshAudioRouter`] fan-in, constructed with
185+
/// the handle's shared `audio_fence` so the datagram hot path and huddle
186+
/// teardown (`GenerationFloor::forget`) enforce exactly one floor.
187+
/// - `HuddleControl` streams → [`HuddleControlAcceptor::accept_inbound`]
188+
/// (owner-side register/unregister control loop).
189+
/// - `ReliableStream` streams → [`ReliableStreamRouter::accept_inbound`], then
190+
/// either the `BUZZ_MESH_DEMO_ECHO` consumer (testbed evidence runs: echo
191+
/// every validated `Data` frame back) or accept/log/close (default — no
192+
/// product session consumer is wired yet).
193+
///
194+
/// Owner-side lease renewal is NOT spawned here: `spawn_observable_renewer`
195+
/// attaches on the *join* path's [`ReliableJoin::Owned`] arm (the pod that
196+
/// acquires the lease renews it), which lands with the first product session
197+
/// consumer. Inbound acceptance never owns a lease, so it never renews one.
198+
///
199+
/// Takes parts rather than `&MeshHandle` so tests can wire a dispatcher
200+
/// without standing up a live `MeshRuntime`.
201+
///
202+
/// [`MeshAudioRouter`]: crate::audio::mesh::MeshAudioRouter
203+
/// [`HuddleControlAcceptor::accept_inbound`]: crate::audio::join::HuddleControlAcceptor::accept_inbound
204+
/// [`ReliableJoin::Owned`]: crate::tunnel::reliable::ReliableJoin::Owned
205+
#[allow(clippy::too_many_arguments)] // boot-only parts bundle, one caller + tests
206+
pub fn wire_mesh_consumers(
207+
dispatcher: &MeshInboundDispatcher,
208+
directory: SessionDirectory,
209+
transport: Arc<dyn RelayPeerTransport>,
210+
local_runtime_id: RuntimeId,
211+
audio_fence: Arc<crate::audio::mesh::GenerationFloor>,
212+
rooms: Arc<crate::audio::AudioRoomManager>,
213+
demo_echo: bool,
214+
) {
215+
// RealtimeMedia datagrams: huddle media fan-in over the shared fence.
216+
// `on_media_datagram` is synchronous and non-blocking (fence check +
217+
// local room delivery), so it runs inline on the accept task.
218+
let audio_router = crate::audio::mesh::MeshAudioRouter::with_fence(
219+
Arc::clone(&rooms),
220+
local_runtime_id,
221+
audio_fence,
222+
);
223+
dispatcher.register_datagrams(Box::new(move |_from, dgram| {
224+
audio_router.on_media_datagram(&dgram);
225+
}));
226+
227+
// HuddleControl streams: owner-side peer registration for cross-pod
228+
// huddles. The acceptor validates structurally, then Redis-fences every
229+
// stateful frame in its control loop.
230+
let acceptor = Arc::new(crate::audio::join::HuddleControlAcceptor::new(
231+
rooms,
232+
Arc::clone(&transport),
233+
Arc::new(directory.clone()),
234+
local_runtime_id,
235+
));
236+
dispatcher.register_huddle_control(Box::new(move |from, hello, stream| {
237+
let acceptor = Arc::clone(&acceptor);
238+
tokio::spawn(async move {
239+
if let Err(e) = acceptor.accept_inbound(from, hello, stream).await {
240+
tracing::warn!(peer = %from, "huddle-control stream ended with error: {e}");
241+
}
242+
});
243+
}));
244+
245+
// ReliableStream streams: owner-side accept. Every session is fence-
246+
// validated on accept; what happens after depends on the wired consumer.
247+
let reliable = Arc::new(ReliableStreamRouter::new(
248+
directory,
249+
transport,
250+
local_runtime_id,
251+
));
252+
dispatcher.register_reliable_stream(Box::new(move |from, hello, stream| {
253+
let router = Arc::clone(&reliable);
254+
tokio::spawn(async move {
255+
let inbound = match router.accept_inbound(from, hello, stream).await {
256+
Ok(inbound) => inbound,
257+
Err(e) => {
258+
tracing::warn!(peer = %from, "reliable stream rejected: {e}");
259+
return;
260+
}
261+
};
262+
if demo_echo {
263+
run_demo_echo(router.directory().clone(), inbound).await;
264+
} else {
265+
tracing::info!(
266+
session_id = %inbound.fenced.session_id,
267+
peer = %inbound.from,
268+
"reliable stream accepted; no session consumer wired — closing"
269+
);
270+
}
271+
});
272+
}));
273+
}
274+
275+
/// `BUZZ_MESH_DEMO_ECHO` consumer: echo every validated `Data` frame back to
276+
/// the sender. A transport/session-routing smoke for cross-pod evidence runs —
277+
/// each echoed frame proves fenced validation (Redis directory hit included)
278+
/// on the owner and delivery in both mesh directions. Not a product flow.
279+
async fn run_demo_echo(directory: SessionDirectory, inbound: ReliableInbound) {
280+
let session_id = inbound.fenced.session_id;
281+
let peer = inbound.from;
282+
let mut stream = inbound.stream;
283+
tracing::info!(%session_id, %peer, "mesh demo echo: session open");
284+
loop {
285+
match stream.recv_validated(&directory).await {
286+
Ok(Some(ReliableFrame::Data(payload))) => {
287+
// recv_validated latched the community from the frame it just
288+
// validated, so it is always present here.
289+
let Some(community_id) = stream.community_id() else {
290+
tracing::warn!(%session_id, "demo echo: no community after validated Data");
291+
return;
292+
};
293+
if let Err(e) = stream.send_bytes(community_id, &payload).await {
294+
tracing::warn!(%session_id, "demo echo: send failed: {e}");
295+
return;
296+
}
297+
}
298+
Ok(Some(ReliableFrame::Goodbye(reason))) => {
299+
tracing::info!(%session_id, ?reason, "mesh demo echo: goodbye");
300+
return;
301+
}
302+
Ok(None) => {
303+
tracing::info!(%session_id, "mesh demo echo: stream closed");
304+
return;
305+
}
306+
Err(e) => {
307+
tracing::warn!(%session_id, "mesh demo echo: recv failed: {e}");
308+
return;
309+
}
310+
}
311+
}
156312
}
157313

158314
/// Wire protocol version advertised in registry/gossip records.
@@ -298,6 +454,7 @@ pub async fn boot_mesh(
298454
membership: membership_arc,
299455
local_runtime_id: runtime_id,
300456
dispatcher,
457+
audio_fence: Arc::new(crate::audio::mesh::GenerationFloor::new()),
301458
runtime,
302459
}))
303460
}
@@ -460,4 +617,72 @@ mod tests {
460617
);
461618
assert_eq!(*first.lock().unwrap(), 1);
462619
}
620+
621+
/// The load-bearing wiring invariant: the datagram consumer registered by
622+
/// `wire_mesh_consumers` enforces the SAME `GenerationFloor` that
623+
/// `MeshHandle.audio_fence` exposes to huddle teardown. A datagram
624+
/// arriving through the dispatcher must advance the shared floor so a
625+
/// later `forget` on the handle's fence actually clears the hot path's
626+
/// suppression state.
627+
#[tokio::test]
628+
async fn wired_datagram_consumer_shares_the_handle_fence() {
629+
struct NoopTransport;
630+
impl buzz_relay_mesh::RelayPeerTransport for NoopTransport {
631+
fn send_datagram(&self, _to: RuntimeId, _dgram: MeshDatagram) -> Result<(), MeshError> {
632+
Ok(())
633+
}
634+
fn open_session_stream(
635+
&self,
636+
_to: RuntimeId,
637+
_hello: StreamHello,
638+
) -> std::pin::Pin<
639+
Box<dyn std::future::Future<Output = Result<MeshStream, MeshError>> + Send + '_>,
640+
> {
641+
Box::pin(async { Err(MeshError::Transport("unused".into())) })
642+
}
643+
fn set_inbound(&self, _handler: Box<dyn InboundHandler>) {}
644+
}
645+
646+
let dispatcher = MeshInboundDispatcher::default();
647+
let fence = Arc::new(crate::audio::mesh::GenerationFloor::new());
648+
let pool = deadpool_redis::Config::from_url("redis://127.0.0.1:1") // never dialed
649+
.create_pool(Some(deadpool_redis::Runtime::Tokio1))
650+
.unwrap();
651+
wire_mesh_consumers(
652+
&dispatcher,
653+
SessionDirectory::new(pool),
654+
Arc::new(NoopTransport),
655+
rid(9),
656+
Arc::clone(&fence),
657+
Arc::new(crate::audio::AudioRoomManager::new()),
658+
false,
659+
);
660+
661+
let session = uuid::Uuid::new_v4();
662+
dispatcher.on_datagram(
663+
rid(1),
664+
MeshDatagram {
665+
fenced: FencedHeader {
666+
session_id: session,
667+
generation: 7,
668+
owner_runtime_id: rid(9),
669+
},
670+
seq: 0,
671+
payload: vec![0, 1, 2],
672+
},
673+
);
674+
675+
// The shared fence observed the datagram's generation: a stale check
676+
// through the HANDLE's Arc is rejected, proving one floor, not two.
677+
assert_eq!(
678+
fence.check(session, 6),
679+
crate::audio::mesh::FenceVerdict::RejectStale { known: 7 }
680+
);
681+
// And `forget` through the handle's Arc clears the hot path's floor.
682+
fence.forget(session);
683+
assert!(matches!(
684+
fence.check(session, 1),
685+
crate::audio::mesh::FenceVerdict::Accept { .. }
686+
));
687+
}
463688
}

0 commit comments

Comments
 (0)