Skip to content

Fix #568 - decode in hardware on windows#584

Merged
scottdraves merged 7 commits into
masterfrom
fix/568-decode-in-hardware-on-windows
Apr 24, 2026
Merged

Fix #568 - decode in hardware on windows#584
scottdraves merged 7 commits into
masterfrom
fix/568-decode-in-hardware-on-windows

Conversation

@pavlopylypchuk321

Copy link
Copy Markdown
Collaborator

No description provided.

@scottdraves

scottdraves commented Apr 22, 2026

Copy link
Copy Markdown
Contributor

looks good!

there is one annoying bug however, which is when the dreams are seamless, there is a black flash at the end of one dream when the next one starts.
you can see it with this playlist:
https://alpha.infinidream.ai/playlist/13489b20-cc0b-4923-8ea8-3f64015fe389
use the "repeat" feature of the UI.
when the dreams just fade out, then it's not so visible because at the end the transparency is 0 anyway.

we had this bug on the mac and we had to make extra calls to flush the decoder to get every frame
i thought there might be a similar situation here, but actually the flash is really fast even when i turn down the decoding frame rate, so that makes it seam like it's in how the frames blend to the screen.

pavlopylypchuk321 and others added 2 commits April 23, 2026 16:49
…hw decode

Two bugs in the D3D11VA seamless transition path:

1. Black flash (RendererDX11.cpp). The NV12 YUV shader variants were
   getting their own blend uniforms (`weights` for cubic, `delta` for
   linear) created alongside the RGBA variants' uniforms. Those YUV
   uniforms start dirty with zero-initialized data (base CShaderUniform
   ctor sets m_bDirty=true). The first time a freshly-created YUV
   variant's Apply() runs, it uploads zeros to slot b1 — clobbering the
   correct weights the RGBA variant's Apply just bound there — and the
   cubic blend produces (0,0,0) for one frame. After that the uniform
   is clean and never rebinds b1, which is why only the very first
   cubic draw of each new clip rendered black. Invisible at app startup
   because fade-in composites over it; visible at every seamless swap
   where alpha jumps straight to 1.

   Fix: don't create uniforms on the YUV variant. It's only ever bound
   mid-draw via the shader swap in DrawTexturedQuad, immediately after
   the RGBA variant's Apply has already bound the correct CB at b1.

2. Forward jump (Player.cpp). While the next clip is waiting for a
   seamless swap it isn't rendered, but we were still Update()ing it
   every tick — so its decoder queue drained as fast as playback, and
   by swap time the first frame popped was N seconds into the clip
   instead of frame 0. With hw decode this showed up as a visible
   forward jump; sw decode hid it because the decoder couldn't keep up.

   Fix: gate the nextClip Update() on the pending transition type. The
   decoder thread keeps running and fills its queue to the backpressure
   cap regardless, so the queue is fully buffered at swap time ready to
   pop frame 0.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@scottdraves

Copy link
Copy Markdown
Contributor

the key was figuring out it worked if you turned off cubic interpolation. then claude figured it out.

@scottdraves scottdraves merged commit 58ef98a into master Apr 24, 2026
amillionbouncyballs added a commit to amillionbouncyballs/infinidream-client that referenced this pull request May 1, 2026
Conflict resolutions:

ContentDecoder.cpp — HW decode init restructured to accommodate all three
platforms under one unified block. Upstream added Windows D3D11VA support
(e-dream-ai#584/e-dream-ai#598) using #if USE_HW_ACCELERATION + #if WIN32. Our Linux VAAPI
block moved to an #elif LINUX_GNU since USE_HW_ACCELERATION=0 on Linux.
Mac VideoToolbox kept in the #else branch with our improved error handling
(av_buffer_ref, get_format, proper fallback logging).

Clip.cpp (constructor) — upstream adds m_spDecoder->SetD3D11Device() in
a #ifdef WIN32 block for D3D11VA frame sharing; our #ifdef LINUX_GNU frame
generation init retained below it. Both kept.

Clip.cpp (GrabVideoFrame) — upstream inlined the frame upload logic; we
kept our UploadFrameToTexture() call which handles D3D11VA HW frames,
RGB24 (RIFE path), and RGBA. Updated UploadFrameToTexture to also detect
isHWFrame for correct Windows D3D11VA BindFrame routing.

client.h (F1 help text) — added upstream's settings shortcut
(FULLSCREEN_MODIFIER_KEY-comma) using our BK() macro style.

client.h (GPU display) — kept our m_RuntimeDiagnostics.gpuFrameTimeMs
(already sourced from GetGPUFrameTimeMs()) but adopted upstream's
", %i%% app" label for the system GPU utilization string.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants