FreeOrbit4D: Training-free Arbitrary Camera Redirection for Monocular Videos via Foreground-Complete 4D Reconstruction

ACM SIGGRAPH 2026 Conference Papers

Wei Cao¹, Hao Zhang¹, Fengrui Tian², Yulun Wu¹, Yingying Li¹, Shenlong Wang¹, Ning Yu^3,4, Yaoyao Liu¹

¹University of Illinois Urbana-Champaign, ²University of Pennsylvania, ³Eyeline Labs, ⁴Netflix

TL;DR: FreeOrbit4D is an effective training-free method for large-angle camera redirection via geometry-complete 4D proxy.

Input Video	Interactive 4D (click to explore)	Output Video

Multiple Trajectories from a Single Input

Input Video	Trajectory #1	Trajectory #2

For more results, please visit our project page.

🔥 News

• May 10, 2026: 🤗 We release the code of FreeOrbit4D!
• March 30, 2026: 🎉 FreeOrbit4D is accepted to SIGGRAPH 2026 Conference Papers!

Installation

Validated on Linux with an NVIDIA GPU (≥ 46 GB VRAM recommended, ~130 GB disk for checkpoints and HF cache). The installer pins CPython 3.10 + PyTorch 2.4.0 + xformers 0.0.27.post1 + CUDA 12.1 (cu121 wheels) inside a conda environment, so the host NVIDIA driver must support CUDA 12.1.

Setup

# Clone the repository and its submodules.
git clone --recursive https://github.com/VVeiCao/FreeOrbit4D.git
cd FreeOrbit4D

# Create the conda environment and install dependencies.
bash setup_env.sh
conda activate freeorbit4d

# Download the checkpoints required by the pipeline.
bash download_checkpoints.sh required

Quick Start

Run an end-to-end pipeline on a bundled scene:

python run_pipeline.py full --config configs/scenes/camel.yaml # Other demos: breakdance, car-turn, horsejump-high.

The final video is saved to outputs/rendering/{scene_name}/{trajectory_name}/inference/output_video.mp4.

Visualize the reconstructed 4D point cloud:

python scripts/1_2_visualization.py --config configs/scenes/camel.yaml

For stage-by-stage commands and data structures, see docs/pipeline.md.

Custom Camera Trajectories

Author a new camera path for any reconstructed scene — bundled or your own. Replace <config> with the scene's yaml (e.g. configs/scenes/camel.yaml or configs/user/{scene_name}.yaml).

python scripts/1_3_cam_traj.py --config <config>

1. Set the trajectory parameters you want in the editor.
2. Enter a filename such as my_trajectory.json.
3. Click Generate Trajectory, then click Save.

Re-render with the saved trajectory (skips reconstruction):

python run_pipeline.py full --config <config> --resume_from render --trajectory_json my_trajectory.json

The filename is resolved relative to the scene's prepared directory. For detailed editor controls, see docs/trajectory_editor.md.

Run on Your Own Video

Take an 854x480 video with at least 45 usable frames, annotate the foreground subject, and run the full pipeline on it. The pipeline currently supports a single, fully-visible foreground object per scene — multiple subjects or heavy occlusion will fail downstream.

Step 1 — Annotate the foreground

Download the SAM2 checkpoint (one-time), then launch the Gradio UI to draw a mask and save a scene config to configs/user/{scene_name}.yaml.

bash download_checkpoints.sh sam2
python scripts/prep_interactive_mask.py --port 8890

Step 2 — Run the pipeline

The default trajectory is a yaw -120° orbit. To customize it, edit the generated configs/user/{scene_name}.yaml before running.

python run_pipeline.py full --config configs/user/{scene_name}.yaml

The final video is saved to outputs/user/rendering/{scene_name}/{trajectory_name}/inference/output_video.mp4.

For input requirements, point-prompting tips, and output layout, see docs/process_your_own_video.md.

Citation

@inproceedings{cao2026freeorbit4d,
  title={{FreeOrbit4D}: Training-free Arbitrary Camera Redirection for Monocular Videos via Foreground-Complete {4D} Reconstruction},
  author={Cao, Wei and Zhang, Hao and Tian, Fengrui and Wu, Yulun and Li, Yingying and Wang, Shenlong and Yu, Ning and Liu, Yaoyao},
  booktitle={ACM SIGGRAPH Conference Papers},
  year={2026}
}

Acknowledgements

FreeOrbit4D builds on several excellent open-source projects and pretrained models. We thank the authors of SV4D and Stability AI generative-models, VGGT, PAGE-4D, DiffSynth-Studio, Wan2.2-VACE-Fun-A14B, Qwen3-VL, PyTorch3D, and SAM2.