Skip to content

Conversation

@psyke83
Copy link
Contributor

@psyke83 psyke83 commented Jan 24, 2023

Description

Prior to this change, we relied on the DwmFlush() workaround to avoid
stutter during mouse movement and a visible mouse pointer, but DwmFlush()
was causing the waitable timer to return too late, causing frame drops.

This fixes two separate issues:

  • mouse-induced stutter (that was previously ameliorated by DwmFlush)
  • excessive frame capture delay seen since waitable timer was implemented

Full changes:

  • Remove DwmFlush() workaround
  • Move release_frame() invocation to occur after snapshot() completes.
  • Handle DXGI_ERROR_INVALID_CALL error case in AcquireNextFrame()
    (occurs if snapshot() fully completes but release_frame() doesn't
    release successfully afterwards).
  • Reduce max timeout for both waitable timer and AcquireNextFrame() from 1s
    to frame to frame delay period, which further avoids the waitable timer
    from returning too late.

Comparison of DwmFlush() to PR via the following method:

  • Add waitable timer debug:
diff --git a/src/platform/windows/display_base.cpp b/src/platform/windows/display_base.cpp
index 5c2ea00..8c2e6a2 100644
--- a/src/platform/windows/display_base.cpp
+++ b/src/platform/windows/display_base.cpp
@@ -114,6 +114,7 @@ capture_e display_base_t::capture(snapshot_cb_t &&snapshot_cb, std::shared_ptr<:
       // just capture the frame now and resynchronize the frame interval with
       // the current time.
       next_frame = std::chrono::steady_clock::now() + delay;
+      BOOST_LOG(error) << "wait time: " << wait_time_us;
     }

     auto status = snapshot(img.get(), std::chrono::duration_cast<std::chrono::milliseconds>(delay), *cursor);

  • Leave a game that runs at a locked 60FPS running on host
  • Monitor excess wait_time_us results.

Results via nightly branch (1 minute capture, DwmFlush enabled):

[2023:01:24:05:29:02]: Error: wait time: -474
[2023:01:24:05:29:02]: Error: wait time: -122
[2023:01:24:05:29:02]: Error: wait time: -973
[2023:01:24:05:29:02]: Error: wait time: -14224
[2023:01:24:05:29:02]: Error: wait time: -298
[2023:01:24:05:29:02]: Error: wait time: -11
[2023:01:24:05:29:02]: Error: wait time: -540
[2023:01:24:05:29:02]: Error: wait time: -700
[2023:01:24:05:29:02]: Error: wait time: -867
[2023:01:24:05:29:02]: Error: wait time: -605
[2023:01:24:05:29:03]: Error: wait time: -558
[2023:01:24:05:29:03]: Error: wait time: -336
[2023:01:24:05:29:03]: Error: wait time: -55
[2023:01:24:05:29:05]: Error: wait time: -12650
[2023:01:24:05:29:05]: Error: wait time: -237
[2023:01:24:05:29:07]: Error: wait time: -1009
[2023:01:24:05:29:07]: Error: wait time: -85
[2023:01:24:05:29:07]: Error: wait time: -16752
[2023:01:24:05:29:10]: Error: wait time: -112
[2023:01:24:05:29:10]: Error: wait time: -190
[2023:01:24:05:29:10]: Error: wait time: -15778
[2023:01:24:05:29:10]: Error: wait time: -757
[2023:01:24:05:29:10]: Error: wait time: -507
[2023:01:24:05:29:10]: Error: wait time: -15204
[2023:01:24:05:29:10]: Error: wait time: -32
[2023:01:24:05:29:13]: Error: wait time: -297
[2023:01:24:05:29:13]: Error: wait time: -268
[2023:01:24:05:29:13]: Error: wait time: -34
[2023:01:24:05:29:13]: Error: wait time: -1783
[2023:01:24:05:29:13]: Error: wait time: -14704
[2023:01:24:05:29:15]: Error: wait time: -250
[2023:01:24:05:29:15]: Error: wait time: -153
[2023:01:24:05:29:15]: Error: wait time: -176
[2023:01:24:05:29:15]: Error: wait time: -475
[2023:01:24:05:29:16]: Error: wait time: -14334
[2023:01:24:05:29:16]: Error: wait time: -917
[2023:01:24:05:29:16]: Error: wait time: -540
[2023:01:24:05:29:16]: Error: wait time: -15339
[2023:01:24:05:29:16]: Error: wait time: -81
[2023:01:24:05:29:18]: Error: wait time: -2502
[2023:01:24:05:29:18]: Error: wait time: -16
[2023:01:24:05:29:18]: Error: wait time: -15942
[2023:01:24:05:29:21]: Error: wait time: -773
[2023:01:24:05:29:21]: Error: wait time: -22
[2023:01:24:05:29:21]: Error: wait time: -389
[2023:01:24:05:29:21]: Error: wait time: -965
[2023:01:24:05:29:21]: Error: wait time: -13304
[2023:01:24:05:29:21]: Error: wait time: -358
[2023:01:24:05:29:21]: Error: wait time: -335
[2023:01:24:05:29:24]: Error: wait time: -404
[2023:01:24:05:29:24]: Error: wait time: -15
[2023:01:24:05:29:24]: Error: wait time: -4460
[2023:01:24:05:29:24]: Error: wait time: -1603
[2023:01:24:05:29:27]: Error: wait time: -9380
[2023:01:24:05:29:27]: Error: wait time: -219
[2023:01:24:05:29:27]: Error: wait time: -106
[2023:01:24:05:29:29]: Error: wait time: -631
[2023:01:24:05:29:29]: Error: wait time: -79
[2023:01:24:05:29:29]: Error: wait time: -358
[2023:01:24:05:29:29]: Error: wait time: -391
[2023:01:24:05:29:29]: Error: wait time: -2000
[2023:01:24:05:29:29]: Error: wait time: -837
[2023:01:24:05:29:29]: Error: wait time: -12066
[2023:01:24:05:29:29]: Error: wait time: -209
[2023:01:24:05:29:29]: Error: wait time: -36
[2023:01:24:05:29:32]: Error: wait time: -751
[2023:01:24:05:29:32]: Error: wait time: -14955
[2023:01:24:05:29:32]: Error: wait time: -39
[2023:01:24:05:29:32]: Error: wait time: -53
[2023:01:24:05:29:35]: Error: wait time: -492
[2023:01:24:05:29:35]: Error: wait time: -373
[2023:01:24:05:29:35]: Error: wait time: -385
[2023:01:24:05:29:35]: Error: wait time: -75
[2023:01:24:05:29:35]: Error: wait time: -15984
[2023:01:24:05:29:35]: Error: wait time: -96
[2023:01:24:05:29:35]: Error: wait time: -204
[2023:01:24:05:29:37]: Error: wait time: -386
[2023:01:24:05:29:37]: Error: wait time: -525
[2023:01:24:05:29:37]: Error: wait time: -2119
[2023:01:24:05:29:38]: Error: wait time: -13899
[2023:01:24:05:29:38]: Error: wait time: -18
[2023:01:24:05:29:40]: Error: wait time: -720
[2023:01:24:05:29:40]: Error: wait time: -14190
[2023:01:24:05:29:40]: Error: wait time: -936
[2023:01:24:05:29:40]: Error: wait time: -38
[2023:01:24:05:29:43]: Error: wait time: -16251
[2023:01:24:05:29:43]: Error: wait time: -581
[2023:01:24:05:29:43]: Error: wait time: -1225
[2023:01:24:05:29:43]: Error: wait time: -15366
[2023:01:24:05:29:46]: Error: wait time: -52
[2023:01:24:05:29:46]: Error: wait time: -15127
[2023:01:24:05:29:46]: Error: wait time: -1307
[2023:01:24:05:29:46]: Error: wait time: -146
[2023:01:24:05:29:46]: Error: wait time: -35
[2023:01:24:05:29:46]: Error: wait time: -648
[2023:01:24:05:29:46]: Error: wait time: -1021
[2023:01:24:05:29:46]: Error: wait time: -196
[2023:01:24:05:29:46]: Error: wait time: -276
[2023:01:24:05:29:47]: Error: wait time: -2647
[2023:01:24:05:29:47]: Error: wait time: -790
[2023:01:24:05:29:48]: Error: wait time: -9980
[2023:01:24:05:29:49]: Error: wait time: -609
[2023:01:24:05:29:49]: Error: wait time: -240
[2023:01:24:05:29:49]: Error: wait time: -123
[2023:01:24:05:29:51]: Error: wait time: -141
[2023:01:24:05:29:51]: Error: wait time: -264
[2023:01:24:05:29:51]: Error: wait time: -167
[2023:01:24:05:29:51]: Error: wait time: -146
[2023:01:24:05:29:51]: Error: wait time: -16151
[2023:01:24:05:29:51]: Error: wait time: -5
[2023:01:24:05:29:51]: Error: wait time: -375
[2023:01:24:05:29:51]: Error: wait time: -15697
[2023:01:24:05:29:51]: Error: wait time: -171
[2023:01:24:05:29:54]: Error: wait time: -1336
[2023:01:24:05:29:54]: Error: wait time: -15018
[2023:01:24:05:29:54]: Error: wait time: -806
[2023:01:24:05:29:54]: Error: wait time: -15578
[2023:01:24:05:29:54]: Error: wait time: -342
[2023:01:24:05:29:54]: Error: wait time: -895
[2023:01:24:05:29:54]: Error: wait time: -390
[2023:01:24:05:29:54]: Error: wait time: -15300
[2023:01:24:05:29:57]: Error: wait time: -1260
[2023:01:24:05:29:57]: Error: wait time: -281
[2023:01:24:05:29:57]: Error: wait time: -881
[2023:01:24:05:29:57]: Error: wait time: -14482
[2023:01:24:05:29:59]: Error: wait time: -81
[2023:01:24:05:29:59]: Error: wait time: -831
[2023:01:24:05:29:59]: Error: wait time: -37
[2023:01:24:05:29:59]: Error: wait time: -14915
[2023:01:24:05:29:59]: Error: wait time: -680
[2023:01:24:05:29:59]: Error: wait time: -598

Results via PR (1 minute capture):

[2023:01:24:05:25:00]: Error: wait time: -546
[2023:01:24:05:25:10]: Error: wait time: -328
[2023:01:24:05:25:16]: Error: wait time: -7
[2023:01:24:05:25:33]: Error: wait time: -17
[2023:01:24:05:25:46]: Error: wait time: -150

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Dependency update (updates to dependencies)
  • Documentation update (changes to documentation)
  • Repository update (changes to repository files, e.g. .github/...)

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated the in code docstring/documentation-blocks for new or existing methods/components

Branch Updates

LizardByte requires that branches be up-to-date before merging. This means that after any PR is merged, this branch
must be updated before it can be merged. You must also
Allow edits from maintainers.

  • I want maintainers to keep my branch updated

@psyke83 psyke83 force-pushed the replace_dwmflush branch 3 times, most recently from 6ce3784 to 0f36766 Compare January 24, 2023 06:06
Prior to this change, we relied on the DwmFlush() workaround to avoid
stutter during mouse movement and a visible mouse pointer, but DwmFlush()
was causing the waitable timer to return too late, causing frame drops.

This fixes two separate issues:
* mouse-induced stutter (that was previously ameliorated by DwmFlush)
* excessive frame capture delay seen since waitable timer was implemented

Full changes:
* Move release_frame() invocation to occur after snapshot() completes.
* Handle DXGI_ERROR_INVALID_CALL error case in AcquireNextFrame()
  (occurs if snapshot() fully completes but release_frame() doesn't
  release successfully afterwards).
* Reduce max timeout for waitable timer from 1s to frame to frame delay
* Reduce AcquireNextFrame timeout to 1/2 delay period, and refund 1/2
  delay period on timeout to give an extra chance to capture frame.
@psyke83
Copy link
Contributor Author

psyke83 commented Mar 9, 2023

Closing this. I'm not happy with the current version of the PR, and I noticed that the Desktop Duplication API performance characteristics are different between Windows 10 and 11:

  • On Windows 10 (22H2), setting vsync off and framerate limiting to 60 will result in periodic dropped frames (to ~55fps or so) every 10 seconds or so, which can only be resolved by enabling vsync; not even this PR can solve this stuttering unless vsync is enabled. This is most apparent at 4K, but also happens at sub-4K capture resolutions. Additionally, 4K capture shows some minor FPS drop below the host rate at 4K (even with vsync enabled).
  • On Windows 11 (22H2), capture doesn't exhibit any noticeable stutter or FPS drops with framerate limited to 60, even when vsync is disabled or streaming at 4K.

I'm not sure if this behaviour is specific to AMD cards, but it might be worth checking if any NVIDIA users experience the same issue on Windows 10 and might be planning to upgrade.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant