Add Interactive Resize Handles to Audio Visualizer Components

Reason for Change

The audio visualizer currently has fixed heights for waveform and spectrogram components, which limits user customization. Users need the ability to dynamically resize these components to optimize their workflow - for example, they might want a larger spectrogram for detailed frequency analysis while keeping a compact waveform for overview.

Problem: Users cannot adjust component heights after initialization, forcing them to compromise on visualization quality or work with suboptimal layouts.

Solution: Implement interactive resize handles that allow users to dynamically adjust waveform and spectrogram heights with real-time visual feedback and optimized performance.

Screenshots

Here is a loom video

https://www.loom.com/share/93867e28a8464e759ff18421eb7688d8

Rollout Strategy

Feature Flag: This feature is controlled by the existing FF_AUDIO_SPECTROGRAMS feature flag for spectrogram functionality. The waveform resize is always available when spectrograms are enabled.

Environment: No additional environment variables required. The feature integrates with existing audio annotation workflows.

Testing

Manual Testing

1. Basic Functionality:

   • Hover over waveform/spectrogram components to verify resize handles appear
   • Drag handles to resize components and verify smooth operation
   • Verify size badge appears during drag showing current dimensions
   • Test minimum (50px) and maximum (500px) height constraints

2. Performance Testing:

   • Verify spectrogram resize doesn't trigger expensive FFT recalculations
   • Confirm waveform resize doesn't cause performance degradation
   • Test rapid resize operations for smoothness

3. Cross-browser Testing:

   • Chrome, Firefox, Safari compatibility
   • Touch device support (if applicable)

4. Edge Cases:

   • Resize during audio playback
   • Resize with multiple audio channels
   • Resize with timeline visible/hidden
   • Mouse leaving component area during drag

Automated Testing

• Unit tests for ResizeRenderer hit testing and cursor management
• Integration tests for InteractionManager event routing
• Visual regression tests for resize handle appearance

Risks

Performance Risks

• Low Risk: Resize operations are optimized to avoid expensive recalculations
  • Spectrogram: Direct config updates without FFT recalculation
  • Waveform: Direct height updates without full redraw
  • Canvas transfer operations are rate-limited

UX Risks

• Low Risk: Resize handles may be initially hard to discover
  • Mitigation: Handles are always visible with subtle styling
  • Hover states provide clear visual feedback
  • Size badge provides immediate feedback during resize

Compatibility Risks

• Low Risk: Touch device support may need refinement
  • Current implementation uses mouse events
  • Touch events can be added in future iteration

Reviewer Notes

Key Implementation Details

1. Architecture:

   • ResizeRenderer implements both Renderer and Interactive interfaces
   • InteractionManager handles event routing for offscreen canvases
   • Layer composition system maintains performance during resize

2. Performance Optimizations:

   • Direct config updates avoid expensive operations
   • Global mouse tracking ensures smooth drag operations
   • Rate-limited canvas transfers prevent performance issues

3. Visual Design:

   • Rounded handles with shadows for modern appearance
   • Dynamic colors with blend modes for visibility
   • Size badge with dark/light mode support

Files to Review

Core Implementation:

• libs/editor/src/lib/AudioUltra/Visual/Renderer/ResizeRenderer.ts - Main resize logic
• libs/editor/src/lib/AudioUltra/Interaction/InteractionManager.ts - Event management
• libs/editor/src/lib/AudioUltra/Interaction/Interactive.ts - Interface definition
• libs/editor/src/lib/AudioUltra/Visual/Visualizer.ts - Integration and height management

Key Changes:

• Added waveform-resize and spectrogram-resize layers
• Implemented handleHeightChange method for dynamic height updates
• Integrated InteractionManager for offscreen canvas event handling
• Optimized renderer config updates to avoid expensive operations

Testing Focus Areas

1. Event Handling: Verify mouse events are properly routed through InteractionManager
2. Performance: Confirm resize operations don't trigger expensive recalculations
3. Visual Feedback: Test resize handle visibility and drag feedback
4. Edge Cases: Test rapid resizing and mouse leaving component area

General Notes

Technical Implementation

The resize functionality uses a layered approach:

1. Visual Layer: ResizeRenderer draws handles and borders on dedicated layers
2. Interaction Layer: InteractionManager routes mouse events to interactive elements
3. Composition Layer: LayerM composes multiple canvases for performance
4. Update Layer: Direct config updates avoid expensive recalculations

User Experience

• Discoverability: Handles are always visible with subtle styling
• Feedback: Size badge shows current dimensions during resize
• Constraints: Min/max heights prevent unusable sizes
• Performance: Smooth resize operations with optimized updates

Future Enhancements

• Touch device support for mobile workflows
• Keyboard shortcuts for precise height adjustments
• Preset height configurations for common use cases
• Animation during resize for smoother visual feedback

This implementation provides a solid foundation for user-customizable audio visualization while maintaining performance and compatibility with existing workflows.