Skip to content

Comments

Refactor video processing pipeline to allow recording of long videos#2164

Merged
rafaellehmkuhl merged 10 commits intobluerobotics:masterfrom
rafaellehmkuhl:refactor-video-processing-pipeline
Oct 8, 2025
Merged

Refactor video processing pipeline to allow recording of long videos#2164
rafaellehmkuhl merged 10 commits intobluerobotics:masterfrom
rafaellehmkuhl:refactor-video-processing-pipeline

Conversation

@rafaellehmkuhl
Copy link
Member

@rafaellehmkuhl rafaellehmkuhl commented Oct 2, 2025 •
edited
Loading

This PR is expected to put an end in the problem of Cockpit failing to process long videos.

This patch completely refactors the video processing pipeline, by doing the following:

  • Moving the concatenation process to happen in the filesystem, and not in the RAM as it used to, effectively ending the problem of long videos failing to process due to memory-usage constraints;
  • Doing the concatenation live with ffmpeg , during the recording, eliminating the need for post-processing;
  • Using a fragmented MP4 (fMP4) as a final container, gaining "immunity" to crashes or other problems that could happens during the recording;
  • Removing the video processing pipeline from Cockpit Lite (the web version), so users can have more peace of mind around that, not seeing video processing fails or long processing freezes. The user can only record the video and download a zip with the chunks to be processed in the standalone version;
  • Adding a dedicated processing tab in the video library to allow users to process ZIPs with video chunks from the web version;
  • Never removing temporary video chunks automatically. There's a dedicated tab for those now, where the user can explicitly remove those that are not needed anymore;
  • Removing the video player, as it was only present in the web version and the web version does not have final playable videos anymore;
  • Refactoring the Video Library UI to account for the processing pipeline refactoring;
Kapture.2025-10-02.at.11.15.46.mp4

I've tested this version with 3 hour-long recordings from both the USB Cam and the RadCam, and they both worked fine (on a Mac Air M3 with 16GB of RAM running Sequoia 15.6).

The size of the Cockpit artifacts will increase as it will now include ffmpeg version 7.1 (platform-specific on each build). This is accomplished through a dedicated script that fetches it from trusted sources (appointed by the ffmpeg team itself).

This PR is already working and can be tested functionality-wise. I'm still working in cleaning up the commits to minimize the changes, as I'm aware this patch changes a lot throughout the codebase.

Fix #1638
Fix #1648
Fix #1765
Fix #1793
Fix #1921
Fix #2004
Fix #2118
Fix #2152

To be merged after #2165

@rafaellehmkuhl rafaellehmkuhl changed the title Refactor video processing pipeline to allow recording long videos Refactor video processing pipeline to allow recording of long videos Oct 2, 2025
@rafaellehmkuhl rafaellehmkuhl force-pushed the refactor-video-processing-pipeline branch 2 times, most recently from 3a2bdba to 31c4ba1 Compare October 2, 2025 14:53
@ES-Alexander ES-Alexander added the docs-needed Change needs to be documented label Oct 2, 2025
@rafaellehmkuhl rafaellehmkuhl force-pushed the refactor-video-processing-pipeline branch 10 times, most recently from 3a801e8 to 0f56c9c Compare October 3, 2025 21:37
@rafaellehmkuhl rafaellehmkuhl mentioned this pull request Oct 3, 2025
Merged
@rafaellehmkuhl rafaellehmkuhl force-pushed the refactor-video-processing-pipeline branch 12 times, most recently from 1a7eded to 993d78f Compare October 4, 2025 21:05
@rafaellehmkuhl
Copy link
Member Author

rafaellehmkuhl commented Oct 6, 2025

@ArturoManzoli I've finished the changes and it's ready for review.

I've let most of the implementation together in the last commit so we can more easily do the changes from the reviews without having to rebase commits, but if you prefer to review commit by commit let me know and I can separate it into the final ones.

@ArturoManzoli
Copy link
Contributor

ArturoManzoli commented Oct 6, 2025 •
edited
Loading

@ArturoManzoli I've finished the changes and it's ready for review.

I've let most of the implementation together in the last commit so we can more easily do the changes from the reviews without having to rebase commits, but if you prefer to review commit by commit let me know and I can separate it into the final ones.

Nice, I'll start the complete review now.
Let's try as it is, but those are very extensive changes (+2,889 −758).

For now, the main commit is still named as WIP even after you force-pushed another content.
Can you check if your last rebase was okay?

@rafaellehmkuhl
Copy link
Member Author

rafaellehmkuhl commented Oct 7, 2025

On the video settings, when no option is marked, or at least 'no live video processing', the recorded video chunks stays available and there is no way to manually process the video.

Will add it.

@rafaellehmkuhl
Copy link
Member Author

rafaellehmkuhl commented Oct 7, 2025

When there is no video on the library, the badge shows that there is one item available:

Will check that.

ArturoManzoli
ArturoManzoli reviewed Oct 7, 2025
View reviewed changes
src/components/VideoLibraryModal.vue Outdated
Comment on lines 191 to 207
<div class="px-4 pt-4 pb-2">
<v-tabs v-model="currentVideoSubTab" color="white" fixed-tabs class="video-sub-tabs">
<v-tab
v-for="tab in videoSubTabs"
:key="tab.name"
:value="tab.name"
:disabled="tab.disabled"
class="text-white"
>
<v-tooltip v-if="tab.tooltip" open-delay="600" activator="parent" location="top">
{{ tab.tooltip }}
</v-tooltip>
<v-icon class="mr-2" size="18">{{ tab.icon }}</v-icon>
{{ tab.label }}
</v-tab>
</v-tabs>
</div>
Copy link
Contributor

@ArturoManzoli ArturoManzoli Oct 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is too much head-space on the tabs menu on the VideoLibraryModal. To better use vertical space and to make the diagramming look nicer, change the top padding on line 191 from pt-4 to pt-1

Before:
before_align_2

After:
after_align

@rafaellehmkuhl rafaellehmkuhl force-pushed the refactor-video-processing-pipeline branch from d7ef271 to 9fbaedd Compare October 7, 2025 16:06
@rafaellehmkuhl
Copy link
Member Author

rafaellehmkuhl commented Oct 7, 2025

@ArturoManzoli all done.

I'm keeping the changes in separate commits but I will squash as necessary when we finish the reviewing.

@ArturoManzoli
Copy link
Contributor

ArturoManzoli commented Oct 7, 2025 •
edited
Loading

Testing on Steam Deck; App image - ubuntu latest

Looks like there is some sort of storage issue that broke the video recording process:

image image

@rafaellehmkuhl
Copy link
Member Author

rafaellehmkuhl commented Oct 7, 2025 •
edited
Loading

Testing on Steam Deck; App image - ubuntu latest

Looks like there is some sort of storage issue that broke the video recording process:

Didn't get it. Is the test on the Steam Deck or using the App image? The Steam Deck should be tested with Flatpak on Desktop mode.

@ArturoManzoli
Copy link
Contributor

ArturoManzoli commented Oct 7, 2025

Testing on Steam Deck; App image - ubuntu latest
Looks like there is some sort of storage issue that broke the video recording process:

Didn't get it. Is the test on the Steam Deck or using the App image? The Steam Deck should be tested with Flatpak on Desktop mode.

Indeed. Let me try again

@rafaellehmkuhl rafaellehmkuhl force-pushed the refactor-video-processing-pipeline branch from 5c1ff0c to 521865f Compare October 7, 2025 22:15
@rafaellehmkuhl
Copy link
Member Author

rafaellehmkuhl commented Oct 8, 2025

Testing on Steam Deck; App image - ubuntu latest
Looks like there is some sort of storage issue that broke the video recording process:

Didn't get it. Is the test on the Steam Deck or using the App image? The Steam Deck should be tested with Flatpak on Desktop mode.

Indeed. Let me try again

After you tested with the Flatpak version and reported it was not working I got to investigate a little more and found out the Flatpak was trying to run ffmpeg in a read-only environment. I've found a workaround for it and it should be working correct now.

ArturoManzoli
ArturoManzoli approved these changes Oct 8, 2025
View reviewed changes
Copy link
Contributor

@ArturoManzoli ArturoManzoli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Working fine now! And super fast

@rafaellehmkuhl
packages: Add yauzl
23e85b6 
It will be used for streaming-based ZIP file operations.
@rafaellehmkuhl
packages: Remove webm-duration-fix
513bc7f 
It was replaced by a `ffmpeg` pipeline, which does not load the entire video into memory.
@rafaellehmkuhl
packages: Add yazl
7474163 
User for creating ZIP files.
@rafaellehmkuhl
yarn: Update lock file
cba3b8a
@rafaellehmkuhl
build: Download platform-specific ffmpeg binaries during install
758f166
@rafaellehmkuhl
electron: storage: Ignore "file not found" errors on file deletion
23aeb1c
@rafaellehmkuhl
electron: storage: Fix return of the getItem method
e3f033e
@rafaellehmkuhl
electron: storage: Enhance file handling with new IPC methods
e2ee6f8 
Added methods for:
- Opening temporary video chunks folder
- Getting file statistics
- Getting path of selected file (through OS selection dialog).
@rafaellehmkuhl rafaellehmkuhl force-pushed the refactor-video-processing-pipeline branch 2 times, most recently from 4a810dc to bae8a8f Compare October 8, 2025 18:53
@rafaellehmkuhl rafaellehmkuhl force-pushed the refactor-video-processing-pipeline branch from bae8a8f to 4000cda Compare October 8, 2025 18:57
@rafaellehmkuhl rafaellehmkuhl force-pushed the refactor-video-processing-pipeline branch from 4000cda to ca5a1b3 Compare October 8, 2025 19:28
@rafaellehmkuhl rafaellehmkuhl merged commit 6537601 into bluerobotics:master Oct 8, 2025
11 of 12 checks passed
@rafaellehmkuhl rafaellehmkuhl deleted the refactor-video-processing-pipeline branch October 8, 2025 20:40
@rafaellehmkuhl rafaellehmkuhl mentioned this pull request Oct 9, 2025
Closed
@rafaellehmkuhl rafaellehmkuhl mentioned this pull request Jan 16, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment