Tempo Sync Modulation - No Pipelines by BuffMcBigHuge · Pull Request #703 · daydreamlive/scope

BuffMcBigHuge · 2026-03-17T13:21:45Z

Tempo Sync & Modulation

Note: The additional "hello world" pipelines for tempo sync have been removed from this PR, they can still be found here: #672.

Summary

Adds tempo synchronization and beat-synced parameter modulation to Scope. Pipelines can lock to Ableton Link or MIDI clock, quantize discrete parameter changes to beat boundaries, and continuously modulate parameters (e.g. noise_scale, denoising_steps) in sync with the beat. Target users are VDJs running Scope alongside Ableton Live, Resolume, or similar applications.

What's in this PR

Core

Tempo sync — Ableton Link and MIDI clock support with optional dependencies (aalink, mido/python-rtmidi). When neither is installed, the UI shows install hints and the feature degrades gracefully.
Parameter scheduling — Discrete parameter changes (prompt switches, denoising adjustments) can be quantized to beat/bar boundaries with configurable lookahead to compensate for pipeline latency.
Parameter modulation — Continuous per-frame modulation of numeric parameters driven by beat phase. Pipeline-agnostic: operates on kwargs before they reach any pipeline.

Frontend

Tempo Sync panel — Source toggle (Link/MIDI), BPM input, beats per bar, live beat indicator, quantize mode, lookahead, beat cache reset rate, prompt cycle rate.
Modulation section — Target selector (from schema ui.modulatable fields), wave shape (sine, cosine, triangle, saw, square, pulse decay), rate (½ beat → 4 bars), depth.
Prompt cycling — Automatically advance through timeline prompts on beat/bar boundaries when enabled.

Modulatable pipelines

LongLive, Krea, MemFlow, RewardForcing, StreamDiffusionV2 expose denoising_steps and (LongLive) noise_scale as modulatable via ui_field_config(modulatable=True, ...).

Architecture

Tempo Sources (Link / MIDI / Client-forwarded)
    → TempoSync Manager (BeatState, 15Hz notification loop)
    → ParameterScheduler (quantize discrete changes)
    → ModulationEngine (per-frame param modulation)
    → PipelineProcessor (inject beat state + modulated params)
    → Pipeline

Beat state: bpm, beat_phase, bar_position, beat_count, is_playing injected into every pipeline call.
Cloud mode: Browser forwards beat state over WebRTC data channel; server prefers client state when fresh (< 2s).
Modulation: Config sent as modulations in parameter updates; Pydantic-validated on receipt.

REST API

Method	Path	Purpose
GET	`/api/v1/tempo/status`	Current tempo status and beat state
POST	`/api/v1/tempo/enable`	Enable tempo sync (Link or MIDI)
POST	`/api/v1/tempo/disable`	Disable tempo sync
POST	`/api/v1/tempo/set_tempo`	Set BPM (Link only)
GET	`/api/v1/tempo/sources`	Available tempo sources and capabilities

Modulation: No dedicated REST API. Config is sent as modulations in WebRTC parameter updates.

How to try it

Install optional deps (for Link or MIDI):

uv sync --group link   # Ableton Link
uv sync --group midi   # MIDI clock

Run Scope and start a stream.
Open the Tempo Sync panel in the sidebar. Enable tempo sync, choose Link or MIDI, set BPM.
Quantize mode: Set to "Beat" or "Bar" so changes like prompt switches land on the beat. Adjust lookahead if latency is visible.
Modulation: In the modulation section, pick a target (e.g. Noise Scale), choose shape/rate/depth, and toggle ON.
Prompt cycling: Add multiple prompts to the timeline, set promptCycleRate to "Bar" or "Beat", and prompts will advance automatically.
Demo pipelines: Use metronome to test lookahead or mod-scope to visualize modulation.

Testing

ParameterScheduler: tests/test_parameter_scheduler.py — adversarial tests for boundary math, concurrency, lookahead, and edge cases.
E2E: Tests passed against fal.ai preview deployment.

Optional dependencies

[project.optional-dependencies]
link = ["aalink>=0.1.1"]
midi = ["mido>=1.3.0", "python-rtmidi>=1.5.0"]

Breaking changes

None. Tempo sync and modulation are additive. Pipelines that don't use beat kwargs are unaffected.

Documentation

docs/tempo-sync-review.md — Implementation review, architecture, API, edge cases, and future work.

Summary by CodeRabbit

New Features
- Tempo synchronization via Ableton Link and MIDI Clock support
- Beat-synced parameter modulation with waveform shapes, depth, and rate controls
- Parameter scheduling aligned to beat boundaries with quantization modes
- Beat-driven prompt cycling
Bug Fixes
- Enhanced noise controller state initialization handling

…_playing. Signed-off-by: BuffMcBigHuge <[email protected]>

Signed-off-by: BuffMcBigHuge <[email protected]>

…_playing. Signed-off-by: BuffMcBigHuge <[email protected]>

Signed-off-by: BuffMcBigHuge <[email protected]>

Signed-off-by: RyanOnTheInside <[email protected]> Signed-off-by: BuffMcBigHuge <[email protected]>

Signed-off-by: BuffMcBigHuge <[email protected]>

…steps and others. Signed-off-by: BuffMcBigHuge <[email protected]>

Signed-off-by: BuffMcBigHuge <[email protected]>

…odulation-2 Signed-off-by: BuffMcBigHuge <[email protected]>

Signed-off-by: BuffMcBigHuge <[email protected]>

Resolve conflicts in pyproject.toml, frame_processor.py, pipeline_processor.py, and uv.lock. Thread tempo_sync and modulation_engine through the new graph executor. Co-Authored-By: Claude Opus 4.6 <[email protected]> Signed-off-by: Rafał Leszko <[email protected]>

…ockerfiles MIDI requires local hardware access so libasound2 is not useful in containers. Also corrects uv sync --group to --extra across docs. Co-Authored-By: Claude Opus 4.6 <[email protected]> Signed-off-by: Rafal Leszko <[email protected]>

Remove isCloudMode and onOpenLoRAsSettings props that were added to DownloadDialog solely to suppress TypeScript errors in StreamPage, but were never actually used by the component. Co-Authored-By: Claude Opus 4.6 <[email protected]> Signed-off-by: Rafał Leszko <[email protected]>

transition (object) and denoising_step_list (array) were not caught by the typeof === "boolean" / "string" checks, so they bypassed beat quantization. Use a DISCRETE_PARAM_KEYS allowlist for key-based detection of known discrete parameters. Co-Authored-By: Claude Opus 4.6 <[email protected]> Signed-off-by: Rafał Leszko <[email protected]>

Co-Authored-By: Claude Opus 4.6 <[email protected]> Signed-off-by: Rafał Leszko <[email protected]>

Moves the 5 tempo endpoints from app.py into tempo_router.py, following the same APIRouter pattern as mcp_router.py. Co-Authored-By: Claude Opus 4.6 <[email protected]> Signed-off-by: Rafał Leszko <[email protected]>

mcp>=1.0.0 is already in core dependencies, so the optional-dependencies entry was unnecessary. Co-Authored-By: Claude Opus 4.6 <[email protected]> Signed-off-by: Rafał Leszko <[email protected]>

Make TempoSource an ABC with abstract methods, add _enabled_lock for thread-safe access, prune dead notification senders, unify install hints to use `uv sync --extra`, and clean up redundant .get() fallbacks. Co-Authored-By: Claude Opus 4.6 <[email protected]> Signed-off-by: Rafał Leszko <[email protected]>

Signed-off-by: Rafał Leszko <[email protected]>

Extract beat boundary calculation to a standalone get_beat_boundary() in tempo_sync.py and consolidate inline beat state handling into a dedicated _apply_tempo_sync() method. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> Signed-off-by: Rafał Leszko <[email protected]>

… warnings - Use `self.enabled` property (with lock) in `get_status()` instead of bare `self._enabled` - Add `PipelineProcessor.set_beat_cache_reset_rate()` to encapsulate private attribute access - Downgrade noisy per-frame "Extra params" log from info to debug - Convert f-string logger calls to %-style lazy formatting across tempo sync modules - Fix ESLint exhaustive-deps warnings in StreamPage prompt cycle effect Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> Signed-off-by: Rafał Leszko <[email protected]>

Signed-off-by: BuffMcBigHuge <[email protected]>

coderabbitai · 2026-03-17T13:21:53Z

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 71bfc75e-1eb1-49d2-b59d-de9b6edafa32

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch marco/feat/beatsync-modulation-3

📝 Coding Plan

Generate coding plan for human review comments

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Tip

You can disable sequence diagrams in the walkthrough.

Disable the reviews.sequence_diagrams setting to disable sequence diagrams in the walkthrough.

Signed-off-by: BuffMcBigHuge <[email protected]>

github-actions · 2026-03-17T13:36:40Z

🚀 fal.ai Preview Deployment


App ID	`daydream/scope-pr-703--preview`
WebSocket	`wss://fal.run/daydream/scope-pr-703--preview/ws`
Commit	`8df1182`

Testing

Connect to this preview deployment by running this on your branch:

uv run build && SCOPE_CLOUD_APP_ID="daydream/scope-pr-703--preview/ws" uv run daydream-scope

🧪 E2E tests will run automatically against this deployment.

github-actions · 2026-03-17T13:44:47Z

✅ E2E Tests passed


Status	passed
fal App	`daydream/scope-pr-703--preview`
Run	View logs

Test Artifacts

Check the workflow run for screenshots.

The OSC SSE EventSource subscription was accidentally deleted when the tempo sync useEffect was added, breaking OSC parameter control. The dmx_server variable was also dropped from the global declaration in lifespan(), causing get_dmx_server() to always return None. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> Signed-off-by: Rafał Leszko <[email protected]>

…ck race - Use 2**31-1 as upper bound for random seed to stay within signed 32-bit range - Add forward pass in modulate_step_list to fix strictly descending invariant when values are clamped to hi; fall back to clamped original if range is too narrow for distinct values - Unify _source_lock and _enabled_lock into a single _state_lock in TempoSync so enable/disable/get_beat_state see consistent _source + _enabled state Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> Signed-off-by: Rafał Leszko <[email protected]>

Wrap `import mido` in try/except like the Link adapter does, preventing ImportError when the module is imported without mido installed. Also move _quantize_mode and _lookahead_ms reads/writes under the scheduler lock to eliminate a TOCTOU race where the mode could change between the initial check and delay computation. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> Signed-off-by: Rafał Leszko <[email protected]>

Move overflow scrolling from InputAndControlsPanel to the parent container so Tempo Sync and Outputs scroll with Input & Controls instead of pushing it off-screen. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> Signed-off-by: Rafał Leszko <[email protected]>

Add --extra link to uv sync in all three platform setup paths so aalink is installed automatically in the desktop app. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> Signed-off-by: Rafał Leszko <[email protected]>

uv run performs its own sync before executing, which can remove aalink installed during setup. Pass --extra link to uv run as well. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> Signed-off-by: Rafał Leszko <[email protected]>

BuffMcBigHuge and others added 30 commits March 11, 2026 20:22

Tempo sync api with bpm, beat_phase, bar_position, beat_count, and is…

6c38b58

…_playing. Signed-off-by: BuffMcBigHuge <[email protected]>

Small refactor.

d350ebb

Signed-off-by: BuffMcBigHuge <[email protected]>

Tempo sync api with bpm, beat_phase, bar_position, beat_count, and is…

d8d79f7

…_playing. Signed-off-by: BuffMcBigHuge <[email protected]>

Small refactor.

be99359

Signed-off-by: BuffMcBigHuge <[email protected]>

feat: beatsync with timed params

c31b0f0

Signed-off-by: RyanOnTheInside <[email protected]> Signed-off-by: BuffMcBigHuge <[email protected]>

Modulation engine and pipeline.

9d5bfac

Signed-off-by: BuffMcBigHuge <[email protected]>

Development and experimentation of pipeline modulation with prompts, …

5718478

…steps and others. Signed-off-by: BuffMcBigHuge <[email protected]>

Linting.

5093ab6

Signed-off-by: BuffMcBigHuge <[email protected]>

Missing tempo sync panel.

4163333

Signed-off-by: BuffMcBigHuge <[email protected]>

Signing, merges, repair from bad rebase.

393179e

Signed-off-by: BuffMcBigHuge <[email protected]>

Small fix.

fef4917

Signed-off-by: BuffMcBigHuge <[email protected]>

Fix small rebase issue.

88aa841

Signed-off-by: BuffMcBigHuge <[email protected]>

Update to tempo sync document.

83ca5e6

Signed-off-by: BuffMcBigHuge <[email protected]>

Merge remote-tracking branch 'origin/main' into marco/feat/beatsync-m…

8f70ecc

…odulation-2 Signed-off-by: BuffMcBigHuge <[email protected]>

Code cleanup and refactor.

c35ce31

Signed-off-by: BuffMcBigHuge <[email protected]>

Linting/formatting.

61d5b6c

Signed-off-by: BuffMcBigHuge <[email protected]>

Better handling.

8d677c3

Signed-off-by: BuffMcBigHuge <[email protected]>

Add libasound2 to Dockerfiles

ad852a5

Update doc and CLAUDE.md

c37ba3c

Revert unintended package-lock.json changes

28aef8a

Co-Authored-By: Claude Opus 4.6 <[email protected]> Signed-off-by: Rafał Leszko <[email protected]>

Extract tempo sync endpoints into dedicated router

01bc93c

Moves the 5 tempo endpoints from app.py into tempo_router.py, following the same APIRouter pattern as mcp_router.py. Co-Authored-By: Claude Opus 4.6 <[email protected]> Signed-off-by: Rafał Leszko <[email protected]>

Remove redundant mcp optional dependency group

c316a96

mcp>=1.0.0 is already in core dependencies, so the optional-dependencies entry was unnecessary. Co-Authored-By: Claude Opus 4.6 <[email protected]> Signed-off-by: Rafał Leszko <[email protected]>

Update uv.lock

24e8594

Signed-off-by: Rafał Leszko <[email protected]>

leszko and others added 3 commits March 16, 2026 11:45

Merge branch 'main' into marco/feat/beatsync-modulation-2

9866f59

Removal of demo pipelines.

1daf001

Signed-off-by: BuffMcBigHuge <[email protected]>

Fix to clamping of of modulation result, cache state invalidation.

f20d1c4

Signed-off-by: BuffMcBigHuge <[email protected]>

Merge branch 'main' into marco/feat/beatsync-modulation-3

bac97b2

Signed-off-by: BuffMcBigHuge <[email protected]>

BuffMcBigHuge changed the title ~~Marco/feat/beatsync modulation 3~~ Tempo Sync Modulation - No Pipelines Mar 17, 2026

BuffMcBigHuge mentioned this pull request Mar 17, 2026

Tempo Sync Modulation #672

Closed

BuffMcBigHuge marked this pull request as ready for review March 17, 2026 13:29

BuffMcBigHuge requested a review from leszko March 17, 2026 13:31

leszko and others added 5 commits March 17, 2026 15:45

feat: include Ableton Link in Electron app builds

468b070

Add --extra link to uv sync in all three platform setup paths so aalink is installed automatically in the desktop app. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> Signed-off-by: Rafał Leszko <[email protected]>

leszko approved these changes Mar 17, 2026

View reviewed changes

leszko merged commit 01a173c into main Mar 17, 2026
7 of 8 checks passed

BuffMcBigHuge mentioned this pull request Mar 18, 2026

Fix: tempo sync modulation interfering with VACE noise scale. #716

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tempo Sync Modulation - No Pipelines#703

Tempo Sync Modulation - No Pipelines#703
leszko merged 40 commits intomainfrom
marco/feat/beatsync-modulation-3

BuffMcBigHuge commented Mar 17, 2026 •

edited

Loading

Uh oh!

coderabbitai bot commented Mar 17, 2026 •

edited

Loading

Review skipped

Uh oh!

github-actions bot commented Mar 17, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 17, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

BuffMcBigHuge commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Tempo Sync & Modulation

Summary

What's in this PR

Core

Frontend

Modulatable pipelines

Architecture

REST API

How to try it

Testing

Optional dependencies

Breaking changes

Documentation

Summary by CodeRabbit

Uh oh!

coderabbitai bot commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

github-actions bot commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🚀 fal.ai Preview Deployment

Testing

Uh oh!

github-actions bot commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ E2E Tests passed

Test Artifacts

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

BuffMcBigHuge commented Mar 17, 2026 •

edited

Loading

coderabbitai bot commented Mar 17, 2026 •

edited

Loading

github-actions bot commented Mar 17, 2026 •

edited

Loading

github-actions bot commented Mar 17, 2026 •

edited

Loading