perf(viewer): render video via QML VideoOutput in a QQuickWidget - #2975
Merged
Conversation
- replace the QGraphicsVideoItem-on-raster-QGraphicsView substrate: QVideoFrame::toImage did an RHI offscreen render + GPU->CPU readback per frame, capping presentation at 8.3 fps (Pi 4) / 10-12 fps (Pi 5) with a saturated GUI thread while HW decode ran fine (issue 2967). Validated on both testbeds: Pi 4 30.0 fps presented at 64% total CPU, Pi 5 26.6 fps at 13-35% - VideoOutput keeps frames on the GPU: scene-graph textures with shader YUV->RGB, composited through the same QQuickRenderControl FBO machinery QWebEngineView already uses (eglfs-safe, inherits whole-screen rotation -- re-validated under QT_QPA_EGLFS_ROTATION) - log frames-rendered (QQuickWindow::afterRendering) next to frames-delivered in playback-stats so presentation-side drops are visible -- the sink-only counter is how the 8 fps regression shipped unnoticed; connection is retried from play() so the counter can't silently stay dead - fail hard (qFatal) when the QML scene is unavailable instead of decoding video to nowhere: crash-respawn is supervised and loud, a silent black-screen kiosk is not - video-rotate maps to VideoOutput.orientation (still a defensive no-op; every platform rotates the whole screen) - ship qt6-declarative-dev + qml6-module-qtquick/-qtmultimedia in the Qt6 viewer images; drop the now-unused multimediawidgets - run the C++ tests with QT_QUICK_BACKEND=software so the QML scene loads under the offscreen platform Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
There was a problem hiding this comment.
Pull request overview
This PR improves Qt6 viewer video presentation performance on embedded targets by moving rendering from a raster QGraphicsVideoItem pipeline to a QML VideoOutput scene hosted in QQuickWidget, keeping frames on the GPU via the Qt Quick scene graph and avoiding per-frame QVideoFrame::toImage() readback.
Changes:
- Reworked
VideoViewto render viaQQuickWidget+ QMLVideoOutput, including a newvideoview.qmlscene and.qrcresource wiring. - Extended
playback-stats.loginstrumentation to logframes-rendered(scene-graph renders) alongsideframes-delivered(sink deliveries). - Updated build/runtime dependencies and C++ unit tests to cover QML scene loading and VideoOutput sink wiring.
Reviewed changes
Copilot reviewed 12 out of 13 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tools/image_builder/utils.py | Adds Qt6 runtime deps for QML/Quick VideoOutput-based rendering; drops multimediawidgets. |
| src/anthias_webview/tests/tests.pro | Updates test link modules for Quick/QuickWidgets and includes the QML resource. |
| src/anthias_webview/tests/test_videoview.cpp | Updates unit tests to assert QML scene load, sink wiring, and rotation via orientation. |
| src/anthias_webview/src/view.h | Updates comments to reflect the new VideoOutput-based render path. |
| src/anthias_webview/src/view.cpp | Updates comments to reflect the new VideoOutput-based render path. |
| src/anthias_webview/src/videoview.qrc | New resource file embedding videoview.qml. |
| src/anthias_webview/src/videoview.qml | New QML scene containing VideoOutput on a black Rectangle. |
| src/anthias_webview/src/videoview.h | Updates documentation and adds counters/methods for rendered-frame tracking. |
| src/anthias_webview/src/videoview.cpp | Implements QQuickWidget/QML VideoOutput pipeline, sink wiring, and render counter hookup. |
| src/anthias_webview/AnthiasViewer.pro | Links Qt QuickWidgets and includes the new QRC resource for Qt6 builds. |
| docs/board-enablement.md | Updates board documentation for the new rendering substrate and performance rationale. |
| docker/Dockerfile.viewer.j2 | Adds qt6-declarative-dev to the Qt6 builder stage to support QuickWidgets. |
| bin/test_webview_cpp.sh | Runs tests with QT_QUICK_BACKEND=software to allow offscreen QQuickWidget init. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
This was referenced Jun 2, 2026
vpetersson
added a commit
that referenced
this pull request
Jun 2, 2026
- CalVer (YYYY.0M.MICRO); still June 2026, micro 0 -> 1 - Ships the QML VideoOutput presentation path (#2975), the pi2/pi3 GStreamer HW video pipeline (#2972), and the x86 WLR_DRM_NO_ATOMIC display-freeze fix (#2978) - Also picks up Pi 4 eglfs rotation (#2971, #2973), the armv7 viewer spawn retry (#2969), the Writeback-connector headless guard (#2968), and viewer log cleanups (#2977, #2979) Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issues Fixed
Description
HW decode was never the problem on the Qt6 boards — presentation was. The
QGraphicsVideoItem-on-raster-QGraphicsViewsubstrate ran every frame throughQVideoFrame::toImage()(an RHI offscreen render plus GPU→CPU readback), then a smooth-scaled CPU blit into the backing store, capping presentation at 8.3 fps on Pi 4 / 10–12 fps on Pi 5 while the sink-side stats read "dropped≈0".This PR renders through a QML
VideoOutputhosted in aQQuickWidget: frames stay on the GPU as scene-graph textures with shader YUV→RGB, composited via the sameQQuickRenderControlFBO machineryQWebEngineViewalready uses — so it satisfies eglfs's single-native-window constraint (aQVideoWidgetwould not: it wraps a second nativeQVideoWindow).Measured on the testbeds (1080p30, DRM vblank tracepoints):
Also:
playback-stats.lognow logsframes-rendered(scene-graph renders, the presentation-side rate) next toframes-delivered— the missing counter that let this regression ship unnoticed. Fields are key=value; positional parsers of SAMPLE/END_FILE/STOP lines break on this revision.qFatal(supervised crash-respawn) instead of silently decoding video to nowhere.QT_QPA_EGLFS_ROTATION=90rotates the Quick-composited window (viewport probe 1080x1920) with video still at full rate.qt6-declarative-dev+qml6-module-qtquick+qml6-module-qtmultimedia;multimediawidgetsis dropped.Checklist
🤖 Generated with Claude Code