The Generalist Inverse Dynamics Model (G-IDM) learns to predict actions from observation transitions across diverse desktop environments. Trained on our multi-domain corpus collected via the OWA Toolkit, G-IDM achieves strong performance across all in-distribution environments, yielding large gains in Pearson correlation (e.g., +39.5 points on Stardew Valley X) and keyboard accuracy (e.g., +57.6 points on Brotato), demonstrating robust generalization over diverse control dynamics.

A key design choice of the Generalist-IDM is NEP-τ (Next-Event Prediction with Temporal Offset), which shifts the observation window forward by τ milliseconds to incorporate future visual context when predicting the current action. Without any offset (τ = 0), Pearson correlations collapse near zero and keyboard accuracy drops sharply, confirming that future context is essential for resolving the current action. A small offset (τ = 50 ms) recovers mouse prediction but remains suboptimal for keyboard accuracy. Performance stabilizes at τ ≥ 100 ms with only minor variation up to 200 ms, showing that NEP-τ is robust to the exact offset once sufficient future context is provided. We adopt τ = 100 ms as the default in all experiments.

We evaluate G-IDM on two unseen games: Battlefield 6 (3D) and Ogu and the Secret Forest (2D). In Battlefield 6, G-IDM achieves 63% keyboard accuracy, matching or slightly outperforming the Specialist-IDM. When provided with a few-shot prefix, the predicted mouse scale improves significantly, demonstrating in-context adaptation to mouse sensitivity. In Ogu and the Secret Forest, G-IDM more than doubles the Specialist-IDM's performance (from ~12% to nearly 28%), showing substantial gains even under a large domain gap.

To validate the effectiveness of desktop pretraining for embodied AI, we evaluate our approach on three challenging downstream tasks: LIBERO manipulation, CANVAS navigation, and SO101 real-world pick-and-place. These benchmarks represent diverse embodied scenarios requiring different sensorimotor skills - precise object manipulation, spatial navigation, and real-world pick-and-place. Our Vision-Action Pretraining (VAPT) framework transfers desktop-pretrained representations to these physical domains, demonstrating that sensorimotor patterns learned from gaming environments can generalize to real-world robotic tasks.

VAPT without pseudo-labels achieves 96.6% total success and 93.6% on long-horizon tasks, comparable to or surpassing much larger models such as π₀ (3.3B) and OpenVLA (7B). Our 1B-parameter model shows particularly strong advantages on long-horizon tasks that require careful action sequencing.

Adding pseudo-labeled demonstrations increases navigation performance to 83.3%, an 8-point improvement over the baseline. The benefit is especially large under misleading instructions, as in sim_orchard (86.7% vs. 53.3%) and sim_street_sidewalk (73.3% vs. 40.0%), indicating that pseudo-labeling is particularly useful for navigation tasks where success depends on high-level planning rather than precise low-level control.

VAPT consistently outperforms the baseline on Meta-World across all difficulty levels, with gains most pronounced on Hard and Very Hard tasks. We further validate our approach with a real-world pick-and-place experiment using an SO101 robot arm, following the evaluation protocol of SmolVLA (Shukor et al., 2025). The task requires grasping a blue cube and placing it in a white box, with the cube placed at five distinct initial positions. We collect 208 demonstration episodes and evaluate each trained policy over 30 rollouts. The baseline InternVL3-1B achieves a 70% success rate, while both VAPT variants reach 80%, confirming that VAPT transfers effectively to real-world hardware.