Repository navigation
perf(huddle): isolate level churn, tighten speech boundaries (#5825, #6397, #5671) - #304
Conversation
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
E2E verification — huddle context split (#275)Drove the real desktop frontend through the Playwright E2E mock bridge (headed Chromium, seeded active huddle with two participants), capturing console Levels context is still live after the splitEmitting a speaker level through the bridge moves Huddle surfaces render and lifecycle still worksHeadphones control in the channel header starts a huddle; Console: no Not covered: |
CI note: desktop E2E failures are pre-existing on
|
| run | result |
|---|---|
branch 68fa5f89 (project-commit-detail + overscroll-boundary + message-feedback-snapshots, smoke) |
8 failed / 4 passed / 4 skipped |
base 9bffb8ca (same three specs, same command) |
8 failed / 4 passed / 4 skipped — same 8 test titles |
Failing specs (project-commit-detail, message-feedback-snapshots, messaging, mission-inbox, needs-restart-screenshots, onboarding-agent-defaults, overscroll-boundary, persistent-agent-audience, edit-agent, entity-link-recipient-cards, inbox-reactions, profile, parity-ancestor-island) exercise projects, inbox, onboarding and agent-profile surfaces. This PR touches only desktop/src/features/huddle/**, desktop/src-tauri/src/huddle/stt*.rs and docs/crew/HERMES.md; the integration profile.spec.ts failures assert agent instruction permissions ("Who can send instructions" → expected Anyone, got Only npub1mock… (owner)), which is the surface changed by #271 landing on main just before this branch.
Per the fork guardrails I have not touched any CI configuration.
DONEIssue #275 is implemented end to end; nothing is left half-written on the branch.
Gates (all run locally, full logs attached below):
Evidence on the session box ( CI seen (not fixed, per the guardrails): IN PROGRESSNothing. The branch is committed ( TODO / NEXT
|
Upstream sync of block#5825 and block#6397 plus the always-on part of block#5671. block#5825 — 20 Hz speaker levels and the ~30 Hz mic analyser lived in the same context value as huddle lifecycle state, so every tick re-rendered every `useHuddle()` consumer (and their subtrees) across the whole app. Levels now live in a separate `HuddleLevelsContext` (`useHuddleLevels()`), the mic analyser moves to `useMicLevelAnalyser`, and the lifecycle value is memoized. Meters still see every tick; ordinary consumers see none (20 -> 0 re-renders per second of huddle audio). block#6397 — endpointing was a single threshold plus a silence counter, so a one-frame blip opened a segment, a dipping vowel started the countdown, and word onsets/offsets were clipped. Speech boundaries now live in a pure `VadEndpoint`: onset needs consecutive above-threshold frames, confirmation prepends bounded pre-roll, offset uses a lower threshold (hysteresis), flush keeps only a hangover tail, and reset drops buffered audio so nothing leaks between segments. Short segments now log why they were dropped. block#5671 — only the push-to-talk grouping fix is portable here: a held shortcut no longer silence-flushes mid-hold when the mic is also manually open (`vad_flush_allowed`). The env-gated STT/TTS levers live in TTS internals this branch does not touch; documented as such in docs/crew/HERMES.md, and BUZZ_STT_FLUSH_MS stays removed. Fixes #275 Signed-off-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
20b59de to
87af09f
Compare
Summary
Upstream sync of Huddle perf work: block#5825 (render churn), block#6397 (speech
boundaries), plus the one always-on piece of block#5671 that is portable to
Crew. No IA surface changes — channel-first IA untouched.
block#5825 — level churn no longer re-renders the app. Speaker levels arrive
~20 Hz and the local mic analyser ticks ~30 Hz; both lived in the single
HuddleContextValue, so every tick re-rendered everyuseHuddle()consumerand its subtree. Split:
micLevel/activeSpeakers/speakerLevelsmoved offHuddleContextValueintouseHuddleLevels(); the inline analyser effect moved verbatim intouseMicLevelAnalyser(same constants, same gate/attack behaviour). The threemeter consumers (
HuddleBar,HuddleProfileControl,HuddleRoomHeader) readthe levels context and still see every tick.
block#6397 — speech boundaries. Endpointing was one threshold plus a silence
counter, so a single high frame opened a segment, a dipping vowel started the
countdown, and onsets/offsets got clipped. Boundary policy now lives in a pure
VadEndpointstate machine inhuddle/stt.rs:ONSET_CONFIRM_FRAMESconsecutive frames aboveVAD_ONSET_THRESHOLD; confirmation prepends up toPRE_ROLL_FRAMESofbuffered audio (bounded
VecDeque) exactly once;VAD_OFFSET_THRESHOLD(hysteresis);HANGOVER_FRAMESof the trailing silence, andreset()drops pre-roll/candidate audio, so nothing leaks into the next segment;
(
SegmentDisposition::DropShort).Unchanged by design: TTS suppression + cooldown, the 30 s OOM guard,
model-independence (Earshot 1.1.0), and
SILENCE_FLUSH_FRAMES(300 ms).block#5671 — divergence, deliberate. The upstream latency levers
(
BUZZ_STT_SPECULATIVE,BUZZ_TTS_STREAMING,BUZZ_TTS_EMIT_FRAMES,BUZZ_STT_THREADS,BUZZ_TTS_THREADS) are default-off env gates insideTTS/STT internals this branch does not touch — porting them here would mean
importing upstream's whole TTS pipeline rewrite, which is out of this issue's
scope and would collide with the sibling sync PRs. What is portable is the
push-to-talk grouping fix: a held shortcut is an explicit "I am not done
talking", so a VAD pause no longer splits the hold even when the microphone is
also manually open (
vad_flush_allowed). Previouslyptt_active.is_none() || manually_openallowed exactly that split.BUZZ_STT_FLUSH_MSstays removed.docs/crew/HERMES.mddocuments the state sonobody sets a variable that does nothing in Crew.
stt.rscrossed the 1000-line ratchet, so its tests moved to a siblingstt_tests.rs(the repo's existing*_tests.rspattern) instead of raisingany baseline.
Related issue
Fixes #275. Part of the Buzz 0.5.18 sync stack (#285); guardrails per #278.
Upstream refs: block#5825, block#5671, block#6397. No duplicate
Crew PR found.
Testing
TDD, RED before GREEN:
pnpm --filter buzz checkpnpm --filter buzz typecheckpnpm --filter buzz testjust check-compilecargo clippy(tauri lib + tests)just test-unitbuzz-cli(command-inventory snapshots) andbuzz-db(embedded migration snapshot) — crates this diff does not touchuseHuddleLevels is not a function; Rust unresolvedVadEndpoint/SegmentDisposition/threshold imports, then unresolvedvad_flush_allowed.huddleLevelsContext.test.mjs2/2;cargo test --lib huddle::stt11/11 (onset confirmation, pre-roll once, hysteresis, hangover-only flush, no
cross-segment leakage, visible short-segment drop, held-PTT grouping).
20 speaker-level events = 1 s of huddle audio, counting
useHuddle()consumer re-renders: before 20 → after 0; level meters still re-render on
all 20 ticks.
No visual change — full-window E2E captures (sidebar + list + channel panel)
before and after are identical:
DONE
Issue #275 is implemented end to end; nothing is left half-written on the branch.
HuddleLevelsContext+useHuddleLevels()split out ofHuddleContext; mic analyser extracted verbatim todesktop/src/features/huddle/lib/useMicLevelAnalyser.ts; ordinary provider valuememoised;
HuddleBar,HuddleProfileControl,HuddleRoomHeaderread the levelscontext. Regression test:
desktop/src/features/huddle/huddleLevelsContext.test.mjs(2/2) mounts the real provider and asserts ordinary consumers see 0 re-renders
across 20 level ticks while meters see all 20.
VadEndpointstate machine indesktop/src-tauri/src/huddle/stt.rs(onset confirmation, boundedVecDequepre-roll, onset/offset hysteresis, hangover-only flush,
reset()leak prevention,visible
SegmentDisposition::DropShort), tests instt_tests.rs(11/11). TTSsuppression, cooldown, the 30 s OOM guard and model independence are unchanged.
(
vad_flush_allowed); the env-gated levers are documented as inert in Crew indocs/crew/HERMES.md.BUZZ_STT_FLUSH_MSstays removed.Gates (all run locally, full logs attached below):
pnpm --filter buzz checkpnpm --filter buzz typecheckpnpm --filter buzz testjust check-compilecargo clippy(tauri lib + tests)just test-unitbuzz-cliandbuzz-dbsnapshot tests — crates this diff does not touchEvidence on the session box (
/home/ubuntu/evidence-275/):red-frontend-huddle-levels.log,green-frontend-huddle-levels.log,red-rust-vad-tests.log,red-rust-vad-flush-allowed.log,green-rust-vad-tests.log,render-churn-before.log(20 renders),render-churn-after.log(0 renders),gate-check.log,gate-typecheck.log,gate-frontend-test.log,gate-test-unit.log,gate-check-compile.log,ci-smoke-branch-local.log,ci-smoke-base-main-local.log, and full-window capturesscreens/before-huddle-perf.png/screens/after-huddle-perf.png(embedded above).E2E drive evidence and the CI parity analysis are in the two PR comments.
CI seen (not fixed, per the guardrails):
Desktop Fast,Desktop Rust,CI Policy,Desktop Release Candidate,macOS ARM Packagegreen.Desktop Smoke E2EandDesktop E2E Integrationfailed, and reproduce identically on the basecommit
9bffb8cain a clean worktree (8 failed / 4 passed / 4 skipped on both, sametest titles) — pre-existing on
main, details in the "CI note" comment. Later runsmay also show as cancelled: the parent session is serialising CI on purpose.
IN PROGRESS
Nothing. The branch is committed (
git commit -s), pushed, and the parent's merge ofmain(mobile PR #305) is already in at20b59dee; GitHub reports no conflicts.TODO / NEXT
huddle regression appears:
HuddleRoomHeaderrenders only in the huddle companionwindow, which the E2E mock bridge does not open as a separate surface; and the
huddle-bar mute state could not be asserted because
micConnectedis false underthe mock (no error was thrown in either case).
VAD_ONSET_THRESHOLD0.6 /VAD_OFFSET_THRESHOLD0.35,ONSET_CONFIRM_FRAMES3,PRE_ROLL_FRAMES8,HANGOVER_FRAMES6) are tunedagainst the tests, not against recorded speech — worth a listening pass on real
audio before release, especially for soft onsets.
desktop/src-tauri/src/huddle/stt.rsor
desktop/src/features/huddle/HuddleContext.tsx; note thatstt.rstests weremoved to a sibling
stt_tests.rsto stay under the file-size ratchet, so aconflicting branch that still has tests inline will need that move applied.
just test-unitstays red until the unrelatedbuzz-cli/buzz-dbsnapshots areregenerated by their owners.
Link to Devin session: https://app.devin.ai/sessions/d62a99b203d14687bd3b0501230eb68b
Requested by: @oscarlehuu