Skip to content

feat(win32): 添加鼠标锁定跟随模式,支持后台 TPS/FPS 视角控制#1232

Merged
MistEO merged 17 commits intoMaaXYZ:mainfrom
ZeroAd-06:feat/win32-mouse-lock-follow
Apr 2, 2026
Merged

feat(win32): 添加鼠标锁定跟随模式,支持后台 TPS/FPS 视角控制#1232
MistEO merged 17 commits intoMaaXYZ:mainfrom
ZeroAd-06:feat/win32-mouse-lock-follow

Conversation

@ZeroAd-06
Copy link
Copy Markdown
Contributor

@ZeroAd-06 ZeroAd-06 commented Mar 25, 2026

概述

为 Win32 控制器添加鼠标锁定跟随模式(Mouse Lock Follow),专为 TPS/FPS 等在后台将鼠标锁定到窗口内的游戏设计。这是"实现终末地后台移动"计划的一部分,解决后台视角旋转的需求。

动机

终末地等 TPS 游戏在后台运行时会通过 RawInput 锁定鼠标到窗口中心,导致:

  1. 真实鼠标移动被游戏捕获,干扰用户正常操作
  2. 通过常规方式注入的鼠标移动无法正确控制视角

本 PR 通过"窗口跟随光标 + RawInput 对冲 + 合成事件标记"三管齐下,实现后台透明的视角控制。

实现方案

三层机制

1. LL Hook 拦截 + 窗口跟随(60fps 追踪线程)

  • WH_MOUSE_LL 钩子拦截硬件鼠标移动,累加 pending delta
  • 追踪线程以 60fps 消费 delta,通过 SetWindowPos 移动窗口中心对齐光标
  • SuspendProcess/ResumeProcess 防止游戏在窗口移动期间竞争

2. RawInput 对冲

  • 创建 HWND_MESSAGE 隐藏窗口注册 RIDEV_INPUTSINK 鼠标 RawInput
  • 收到硬件鼠标事件时,通过 SendInput 发送反向 delta 抵消
  • 使游戏 RawInput 读取到的硬件净位移为零

3. 合成事件标记(counter_pending_ 原子计数器)

  • relative_move() 调用前 counter_pending_++
  • RawInput 回调中 CAS 检查:计数 > 0 则跳过对冲(识别为自己发的合成事件)
  • 游戏正常感知到注入的 delta,实现视角旋转

DPI 感知

  • 三级回退:Per-Monitor V2 → V1 → System Aware
  • 使用 GetPhysicalCursorPos 确保与 MSLLHOOKSTRUCT::pt 坐标系一致

生命周期

  • 激活时保存窗口原始位置,停用时恢复
  • s_active_instance_ 静态指针 + 安全清理防止 UAF

API 变更

新增 C API:

MaaCtrlId MaaControllerPostMouseLockFollow(
    MaaController* ctrl, MaaBool enabled);

新增 Python / Node.js 绑定:post_mouse_lock_follow(enabled)

行为变更:

  • Win32 平台的 relative_move() 现在要求必须先开启 mouse-lock-follow 模式

变更范围

  • 核心实现集中在 MessageInput.cpp(+518 行)
  • 修改:ControllerAgent、Win32ControlUnitMgr
  • 新增:Agent Client/Server 远程支持
  • 新增:Python / Node.js 语言绑定
  • 更新:中英文文档(22 个文件变更,+743/-12)

测试计划

  • 终末地实际验证:后台视角旋转流畅度
  • 多 DPI 环境下窗口跟随坐标正确性
  • 激活/停用切换后窗口位置正确恢复
  • relative_move() 注入的 delta 不被对冲
  • 长时间运行稳定性(无 UAF、无泄漏)
  • 与后台托管按键守护配合使用验证

此 PR 是"实现终末地后台移动"计划的视角控制部分,与 feat/win32-background-managed-keys(后台托管按键)配合使用。

🤖 Generated with Claude Code

Summary by Sourcery

为后台 TPS/FPS 摄像机控制添加 Win32 鼠标锁定跟随模式,并通过控制器 API 和绑定将其暴露出来。

New Features:

  • 在 Win32 MessageInput 控制器中引入 Mouse Lock Follow 模式,以支持在鼠标锁定类游戏中基于后台参考系的鼠标移动和摄像机控制。
  • 添加控制器动作和 C API,用于切换 Mouse Lock Follow 模式,并在 Win32 relative_move 操作中要求启用该模式。
  • 在 Python 和 Node.js 控制器绑定中,以及通过远程代理客户端/服务器消息机制,暴露鼠标锁定跟随模式的切换功能。

Enhancements:

  • 改进鼠标跟踪线程中的 DPI 感知处理以及定时器/清理的健壮性,包括 RawInput 窗口生命周期管理。
  • 扩展 Win32 控制单元管理器和控制器抽象,以通过新的模式支持相对鼠标移动输入。

Documentation:

  • 在英文和中文的控制方法指南中,记录 Win32 Mouse Lock Follow 模式以及它与 MaaControllerPostRelativeMove 之间的关系。
Original summary in English

Summary by Sourcery

Add a Win32 mouse lock follow mode for background TPS/FPS camera control and expose it through the controller APIs and bindings.

New Features:

  • Introduce Mouse Lock Follow mode in the Win32 MessageInput controller to support background-relative mouse movement and camera control in mouse-locking games.
  • Add a controller action and C API to toggle mouse lock follow mode and require it for Win32 relative_move operations.
  • Expose mouse lock follow toggling in Python and Node.js controller bindings and through remote agent client/server messaging.

Enhancements:

  • Improve DPI-awareness handling and timer/cleanup robustness in the mouse tracking thread, including RawInput window lifecycle management.
  • Extend Win32 control unit manager and controller abstractions to support relative mouse move input via the new mode.

Documentation:

  • Document the Win32 Mouse Lock Follow mode and its relationship to MaaControllerPostRelativeMove in both English and Chinese control method guides.

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - 我这边发现了 2 个问题,并给出了一些整体性的反馈:

  • 有几个跨线程使用的状态标志(例如 mouse_lock_follow_active_tracking_thread_started_for_lock_follow_ 以及 rawinput_ensure_* 这些布尔值)是普通的 bool,但会在不同线程中读写(tracking 线程、action runner、析构函数);建议将它们改为 std::atomic<bool>,或者在所有访问时统一使用 tracking_state_mutex_ 进行保护,以避免数据竞争。
  • tracking 线程现在在每次启动时都会调用 SetProcessDpiAwarenessContext / SetProcessDpiAwareness;这会在进程级别修改 DPI awareness,可能会让宿主应用措手不及——如果可以的话,建议把这部分集中为一次性的初始化(或者放到配置项 / 功能开关后面),而不是在每个控制器的 tracking 线程中都做一次。
  • MessageInputtracking_thread_func 和 RawInput 消息窗口之间的生命周期耦合依赖于 tracking 线程中的 tracking_exit_ 加上 destroy_rawinput_window();请仔细检查所有销毁路径(包括错误场景和 inactive())最终是否都会在 MessageInput 被销毁之前 join tracking 线程,否则静态的 RawInputWndProc 可能会访问已经悬空的 s_active_instance_ 指针。
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- Several state flags used across threads (e.g., `mouse_lock_follow_active_`, `tracking_thread_started_for_lock_follow_`, and the `rawinput_ensure_*` booleans) are plain `bool` and read/written from different threads (tracking thread, action runner, destructor); consider making them `std::atomic<bool>` or consistently guarding all accesses with `tracking_state_mutex_` to avoid data races.
- The tracking thread now calls `SetProcessDpiAwarenessContext` / `SetProcessDpiAwareness` every time it starts; this alters DPI awareness process-wide and may surprise host applications—if possible, centralize this as a one-time initialization (or behind a config/feature flag) rather than doing it in a per-controller tracking thread.
- The lifetime coupling between `MessageInput`, `tracking_thread_func`, and the RawInput message window relies on `tracking_exit_` plus `destroy_rawinput_window()` in the tracking thread; double‑check that all destruction paths (including error cases and `inactive()`) ultimately join the tracking thread before `MessageInput` is destroyed, otherwise the static `RawInputWndProc` may see a dangling `s_active_instance_` target.

## Individual Comments

### Comment 1
<location path="source/MaaWin32ControlUnit/Input/MessageInput.cpp" line_range="491-496" />
<code_context>
+                // 如果失败可能是已经设置过(E_ACCESSDENIED),这是可以接受的
+            }
+        }
+        if (!ok) {
+            HMODULE shcore = LoadLibraryW(L"shcore.dll");
+            if (shcore) {
+                using FnAware = HRESULT(WINAPI*)(int);
+                auto fnAware = reinterpret_cast<FnAware>(GetProcAddress(shcore, "SetProcessDpiAwareness"));
+                if (fnAware) {
+                    fnAware(2 /*PROCESS_PER_MONITOR_DPI_AWARE*/);
+                }
</code_context>
<issue_to_address>
**suggestion (bug_risk):** 考虑在使用完 SetProcessDpiAwareness 之后释放 shcore.dll 的模块句柄。

这里调用了 `LoadLibraryW(L"shcore.dll")`,但返回的 `HMODULE` 从未被释放。由于 `tracking_thread_func` 可能会运行多次,这会导致一个小且重复的资源泄漏问题。请在调用(或尝试调用)`SetProcessDpiAwareness` 之后增加 `FreeLibrary(shcore);`。
</issue_to_address>

### Comment 2
<location path="source/MaaWin32ControlUnit/Input/MessageInput.cpp" line_range="508-517" />
<code_context>
     timeBeginPeriod(1);
-    OnScopeLeave([this]() {
+    bool init_reported = false;
+    OnScopeLeave([&]() {
+        {
+            std::lock_guard lock(tracking_state_mutex_);
+            if (!init_reported) {
+                tracking_thread_init_done_ = true;
+                tracking_thread_init_ok_ = false;
+            }
+            rawinput_ensure_requested_ = false;
+            rawinput_ensure_done_ = true;
+            rawinput_ensure_ok_ = false;
+        }
+        tracking_state_cv_.notify_all();
         close_target_process();
+        destroy_rawinput_window();
         timeEndPeriod(1);
     });

</code_context>
<issue_to_address>
**nitpick:** 在 scope guard 中按引用捕获会带来捕获过多变量的风险。

`tracking_thread_func` 里的 `OnScopeLeave([&] { ... })` 会按引用捕获所有局部变量,但实际上只需要 `this``init_reported`。请改为显式捕获列表(例如 `[this, &init_reported]`),以明确生命周期,并避免将来不小心按引用使用其他局部变量。
</issue_to_address>

Sourcery 对开源项目是免费的——如果你觉得这个评审有帮助,欢迎分享 ✨
帮我变得更有用!请在每条评论上点 👍 或 👎,我会根据反馈改进后续的评审。
Original comment in English

Hey - I've found 2 issues, and left some high level feedback:

  • Several state flags used across threads (e.g., mouse_lock_follow_active_, tracking_thread_started_for_lock_follow_, and the rawinput_ensure_* booleans) are plain bool and read/written from different threads (tracking thread, action runner, destructor); consider making them std::atomic<bool> or consistently guarding all accesses with tracking_state_mutex_ to avoid data races.
  • The tracking thread now calls SetProcessDpiAwarenessContext / SetProcessDpiAwareness every time it starts; this alters DPI awareness process-wide and may surprise host applications—if possible, centralize this as a one-time initialization (or behind a config/feature flag) rather than doing it in a per-controller tracking thread.
  • The lifetime coupling between MessageInput, tracking_thread_func, and the RawInput message window relies on tracking_exit_ plus destroy_rawinput_window() in the tracking thread; double‑check that all destruction paths (including error cases and inactive()) ultimately join the tracking thread before MessageInput is destroyed, otherwise the static RawInputWndProc may see a dangling s_active_instance_ target.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- Several state flags used across threads (e.g., `mouse_lock_follow_active_`, `tracking_thread_started_for_lock_follow_`, and the `rawinput_ensure_*` booleans) are plain `bool` and read/written from different threads (tracking thread, action runner, destructor); consider making them `std::atomic<bool>` or consistently guarding all accesses with `tracking_state_mutex_` to avoid data races.
- The tracking thread now calls `SetProcessDpiAwarenessContext` / `SetProcessDpiAwareness` every time it starts; this alters DPI awareness process-wide and may surprise host applications—if possible, centralize this as a one-time initialization (or behind a config/feature flag) rather than doing it in a per-controller tracking thread.
- The lifetime coupling between `MessageInput`, `tracking_thread_func`, and the RawInput message window relies on `tracking_exit_` plus `destroy_rawinput_window()` in the tracking thread; double‑check that all destruction paths (including error cases and `inactive()`) ultimately join the tracking thread before `MessageInput` is destroyed, otherwise the static `RawInputWndProc` may see a dangling `s_active_instance_` target.

## Individual Comments

### Comment 1
<location path="source/MaaWin32ControlUnit/Input/MessageInput.cpp" line_range="491-496" />
<code_context>
+                // 如果失败可能是已经设置过(E_ACCESSDENIED),这是可以接受的
+            }
+        }
+        if (!ok) {
+            HMODULE shcore = LoadLibraryW(L"shcore.dll");
+            if (shcore) {
+                using FnAware = HRESULT(WINAPI*)(int);
+                auto fnAware = reinterpret_cast<FnAware>(GetProcAddress(shcore, "SetProcessDpiAwareness"));
+                if (fnAware) {
+                    fnAware(2 /*PROCESS_PER_MONITOR_DPI_AWARE*/);
+                }
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Consider freeing the shcore.dll module handle after using SetProcessDpiAwareness.

`LoadLibraryW(L"shcore.dll")` is called here but the returned `HMODULE` is never released. Since `tracking_thread_func` may run multiple times, this can cause a small, repeated resource leak. Please call `FreeLibrary(shcore);` after `SetProcessDpiAwareness` has been invoked (or attempted).
</issue_to_address>

### Comment 2
<location path="source/MaaWin32ControlUnit/Input/MessageInput.cpp" line_range="508-517" />
<code_context>
     timeBeginPeriod(1);
-    OnScopeLeave([this]() {
+    bool init_reported = false;
+    OnScopeLeave([&]() {
+        {
+            std::lock_guard lock(tracking_state_mutex_);
+            if (!init_reported) {
+                tracking_thread_init_done_ = true;
+                tracking_thread_init_ok_ = false;
+            }
+            rawinput_ensure_requested_ = false;
+            rawinput_ensure_done_ = true;
+            rawinput_ensure_ok_ = false;
+        }
+        tracking_state_cv_.notify_all();
         close_target_process();
+        destroy_rawinput_window();
         timeEndPeriod(1);
     });

</code_context>
<issue_to_address>
**nitpick:** Using a by-reference capture in the scope guard risks capturing more than necessary.

The `OnScopeLeave([&] { ... })` in `tracking_thread_func` captures all locals by reference, though only `this` and `init_reported` are needed. Please switch to an explicit capture list (e.g. `[this, &init_reported]`) to clarify lifetimes and avoid accidental future use of other locals by reference in this lambda.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@neko-para
Copy link
Copy Markdown
Contributor

感觉接口过于特化了。如果真的需要做这个配置的话,考虑搞个通用的controller set attribute函数可能好点,或者直接复用现在的set option

@MistEO
Copy link
Copy Markdown
Member

MistEO commented Mar 27, 2026

ControlUnit 重构了下,有冲突了 #1235

Z_06 and others added 16 commits March 28, 2026 03:56
Add a new action API (MaaControllerPostMouseLockFollow) to toggle a mode
designed for TPS/FPS games that lock the mouse to their window in the
background. When enabled, uses a three-layer architecture:

1. WH_MOUSE_LL hook: intercepts hardware mouse movement, accumulates
   processed delta for window/cursor synchronization
2. RawInput (WM_INPUT): receives raw device delta, sends counter-move
   via SendInput to prevent the game from sensing physical movement
3. Tracking thread (60fps): consumes accumulated delta, moves both
   window and cursor using absolute anchor positioning with
   NtSuspendProcess/NtResumeProcess for atomic updates

Supports relative_move for intentional camera rotation (uses counter
pending atomic to bypass the RawInput counter-move). Works with any
MessageInput-based input method (PostMessage/SendMessage variants).

Includes: C API, ControllerAgent action pipeline, Win32ControlUnitAPI
interface, Python/NodeJS bindings, Agent client/server forwarding, and
a standalone PoC (poc_mouse_lock_follow.cpp).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
At display scaling >100%, the window would continuously drift to the
bottom-right corner due to coordinate system mismatch:
- MSLLHOOKSTRUCT::pt always uses physical pixels
- GetCursorPos/SetCursorPos behavior depends on DPI awareness context
- SetThreadDpiAwarenessContext alone is unreliable in DLL contexts

Fix with three approaches applied together:
1. Use GetPhysicalCursorPos in LL hook callback and activate function
   (always returns physical pixels regardless of DPI context)
2. Set process-level DPI awareness with 3-tier fallback in tracking
   thread (matching the validated PoC approach)
3. Temporarily set thread DPI awareness in activate/deactivate for
   SetWindowPos and other DPI-sensitive APIs

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
纯虚函数改为返回 false 的默认实现,避免不需要此功能的子类被迫实现空方法。

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
在析构函数和 deactivate_mouse_lock_follow 中使用 CAS 清除
s_active_instance_,确保钩子回调不会在对象销毁后访问悬空指针。

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
原先的单次 load + CAS 在 relative_move 并发递增 counter_pending_ 时
可能因值变化而失败,导致合成事件被误判为硬件事件并触发多余对冲。
改用 compare_exchange_weak 循环确保正确递减。

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
移除 tracking_thread_func 中的 SetProcessDpiAwarenessContext /
SetProcessDpiAwareness / SetProcessDPIAware 三级回退,仅保留
SetThreadDpiAwarenessContext。进程级设置会影响宿主进程的所有线程,
线程级设置足以确保追踪线程的坐标系一致性。

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
SetWindowPos 后改为轮询检查窗口位置是否到位(最多 100ms),
在窗口快速响应时可提前退出,避免不必要的固定延迟。

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
tracking_exit_ 的重置已在 ensure_tracking_thread() 启动新线程前完成,
deactivate 中 join 后再重置是多余的,且可能与析构函数中的
tracking_exit_ = true 产生竞态。

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
在中英文文档中补充说明 MaaControllerPostRelativeMove 在 Win32 平台
必须先开启 mouse-lock-follow 模式才能使用,否则调用将失败。

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
仅使用线程级 SetThreadDpiAwarenessContext 在部分宿主进程中不可靠,
导致坐标系不匹配窗口飞到右下角。恢复进程级设置作为兜底,失败时静默跳过。

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@ZeroAd-06 ZeroAd-06 force-pushed the feat/win32-mouse-lock-follow branch from b8f1b90 to 2e56336 Compare March 29, 2026 16:52
@ZeroAd-06
Copy link
Copy Markdown
Contributor Author

现在这样怎么样

@ZeroAd-06
Copy link
Copy Markdown
Contributor Author

@neko-para

@neko-para
Copy link
Copy Markdown
Contributor

这几天没空看(

Align MessageInput with existing Win32 patterns so the mouse-lock-follow review fixes do not keep repeating process DPI initialization and remain easier to maintain.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <[email protected]>
@MistEO MistEO merged commit 722e33a into MaaXYZ:main Apr 2, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants