Awesome-4D-Spatial-Intelligence

This repository collects summaries of over 500 recent studies on methods for reconstructing 4D spatial intelligence, and will be continuously updated.

If you have suggestions for new resources, improvements to methodologies, or corrections for broken links, please don't hesitate to open an issue or submit a pull request. Contributions of all kinds are welcome and greatly appreciated.

Table of Contents

• Level1 -- Low-level 3D cues

  • Depth Estimation
  • Camera pose estimation
  • 3D tracking
  • Unifying depth and camera pose estimation
  • Unifying depth, camera pose, and 3D tracking

• Level2 -- 3D scene components

  • Small-scale 3D object/scene reconstruction
  • Large-scale 3D scene reconstruction

• Level3 -- 4D dynamic scenes

  • General 4D scene reconstruction
  • Human-centric dynamic modeling
    • SMPL-based human mesh recovery and tracking
    • Egocentric motion tracking
    • Appearance-rich dynamic human modeling

• Level 4 -- Interaction among scene components

  • SMPL-based human-centric interaction
    • Human-Object-Interaction (HOI)
    • Human-Scene-Interaction (HSI)
    • Human-Human-Interaction (HHI)
  • Appearance-rich human-centric interaction
  • Egocentric human-centric interaction

• Level 5 -- Incorporation of physical laws and constraints

  • Dynamic 4D human simulation with physics
  • 3D scene reconstruction with physical plausibility

Level 1 -- Low-level 3D cues

Depth Estimation

Year	Venue	Acronym	Paper	Project	GitHub
2018	CVPR	GeoNet	GeoNet: Unsupervised learning of dense depth, optical flow and camera pose	–	GitHub
2019	NeurIPS	SC-SfMLearner	Unsupervised scale-consistent depth and ego-motion learning from monocular video	–	GitHub
2019	ICCV	–	Depth from videos in the wild: Unsupervised monocular depth learning from unknown cameras	–	GitHub
2019	ICCV	–	Exploiting temporal consistency for real-time video depth estimation	–	GitHub
2019	IEEE T‑ITS	FlowGRU	Temporally consistent depth prediction with flow‑guided memory units	Project	GitHub
2019	ICCV	GLNet	Self-supervised learning with geometric constraints in monocular video: Connecting flow, depth, and camera
2020	ICLR	DeepV2D	DeepV2D: Video to Depth with Differentiable Structure from Motion		GitHub
2020	SIGGRAPH		Consistent Video Depth Estimation	Project	GitHub
2020	IEEE RA‑L	–	Don’t forget the past: Recurrent depth estimation from monocular video	Project	–
2020	IROS	FDNet	Video depth estimation by fusing flow‑to‑depth proposals	Project	GitHub
2021	CVPR	ESTDepth	Multi‑view depth estimation using epipolar spatio‑temporal networks	Project	GitHub
2021	CVPR	ManyDepth	The temporal opportunist: Self‑supervised multi‑frame monocular depth	–	GitHub
2021	ECCV	SimpleRecon	Simplerecon: 3D reconstruction without 3D convolutions	Project	GitHub
2021	TOG		Consistent depth of moving objects in video	Project	GitHub
2022	ACMMM	FMNet	Less is More: Skip Connections in Video Depth Estimation		GitHub
2022	CVPR	DepthFormer	Multi‑frame self‑supervised depth with transformers	Project	–
2022	3DV	MonoViT	Monovit: Self‑supervised monocular depth estimation with a vision transformer	–	GitHub
2023	WACV	CODD	Temporally consistent online depth estimation in dynamic scenes	Project	GitHub
2023	ICCV	MAMo	Mamo: Leveraging memory and attention for monocular video depth estimation
2023	ICCV	NVDS	Neural Video Depth Stabilizer	Project	GitHub
2024	T-PAMI	NVDS+	NVDS+: Towards Efficient and Versatile Neural Stabilizer for Video Depth Estimation	Project	GitHub
2024	ECCV	FutureDepth	FutureDepth: Learning to predict the future improves video depth estimation	–	–
2025	ICLR	DepthAnyVideo	Depth Any Video with Scalable Synthetic Data	Project	GitHub
2025	CVPR	DepthCrafter	DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos	Project	GitHub
2025	CVPR	ChronoDepth	Learning Temporally Consistent Video Depth from Video Diffusion Priors	Project	GitHub
2025	CVPR	Video Depth Anything	Video Depth Anything: Consistent Depth Estimation for Super-Long Videos	Project	GitHub
2025		StereoAdapter	StereoAdapter: Adapting Stereo Depth Estimation to Underwater Scenes	Project	GitHub

Camera pose estimation

Year	Venue	Acronym	Paper	Project	GitHub
2014	ECCV	LSD-SLAM	LSD-SLAM: Large-scale direct monocular SLAM	Project	GitHub
2015	TRO	ORB-SLAM	ORB-SLAM: A Versatile and Accurate Monocular SLAM System	Project	GitHub
2017	T-PAMI	DSO	Direct Sparse Odometry	Project	GitHub
2017	TRO	ORB-SLAM2	ORB-SLAM2: An open-source SLAM system for monocular, stereo, and RGB-D cameras		GitHub
2017	ICRA	DeepVO	DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks	Project
2020	CVPR	D3VO	D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry		GitHub
2021	CoRL	TartanVO	TartanVO: A Generalizable Learning-based VO		GitHub
2021	TITS	DDSO	Deep Direct Visual Odometry
2021	T-RO	LF-SLAM	Line Flow Based Simultaneous Localization and Mapping
2022	ICRA	EDPLVO	EDPLVO: Efficient Direct Point-Line Visual Odometry
2022	ECCV	ParticleSfM	ParticleSfM: Exploiting Dense Point Trajectories for Localizing Moving Cameras in the Wild	Project	GitHub
2023	ICCV	XVO	XVO: Generalized Visual Odometry via Cross-Modal Self-Training	Project	GitHub
2023	ICRA	DytanVO	DytanVO: Joint Refinement of Visual Odometry and Motion Segmentation in Dynamic Environments		GitHub
2023	IROS	StereoVO	Stereo VO with Point and Line Matching using Attention GNN
2023	ICRA	Structure PLP-SLAM	Structure PLP-SLAM: Efficient Sparse Mapping using Point, Line and Plane		GitHub
2024	CVPR	LEAP-VO	LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometry	Project	GitHub
2024	CVPR	VGGSfM	VGGSfM: Visual Geometry Grounded Deep Structure From Motion	Project	GitHub
2024	ECCV	DPV-SLAM	Deep Patch Visual SLAM		GitHub
2024	ECCV	RLVO	Reinforcement Learning Meets Visual Odometry		GitHub
2024	RA-L		Efficient Camera Exposure Control for VO via Deep RL		GitHub
2024	T-ASE	UL-SLAM	UL-SLAM: A Universal Monocular Line-Based SLAM via Unifying Structural and Non-Structural Constraints		GitHub
2023	NeurIPS	DPVO	Deep Patch Visual Odometry		GitHub
2025	CVPR	AnyCam	AnyCam: Learning to Recover Camera Poses and Intrinsics from Casual Videos	Project	GitHub
2025	CVPR	DynPose	Dynamic Camera Poses and Where to Find Them	Project
2025	T-RO	AirSLAM	AirSLAM: An Efficient and Illumination-Robust Point-Line SLAM System	Project	GitHub
2025	arXiv	Puffin	Thinking with Camera: A Unified Multimodal Model for Camera-Centric Understanding and Generation	Project	GitHub

---

3D tracking

Year	Venue	Acronym	Paper	Project	GitHub
2023	ICCV	OmniMotion	Tracking Everything Everywhere All at Once	Project	GitHub
2024	ECCV	OmniTrackFast	Track Everything Everywhere Fast and Robustly	Project	GitHub
2024	CVPR	SpatialTracker	SpatialTracker: Tracking Any 2D Pixels in 3D Space	Project	GitHub
2025	WACV	EgoPoints	EgoPoints: Advancing Point Tracking for Egocentric Videos	Project	GitHub
2025	T-PAMI	SceneTracker	SceneTracker: Long-term Scene Flow Estimation Network		GitHub
2025	ICLR	DELTA	DELTA: Dense Efficient Long-range 3D Tracking for any video	Project	GitHub
2025	CVPR	Seurat	Seurat: From Moving Points to Depth	Project	GitHub
2025	arXiv	TAPIP3D	TAPIP3D: Tracking Any Point in Persistent 3D Geometry	Project	GitHub
2025		SyncTrack4D	SyncTrack4D: Cross-Video Motion Alignment and Video Synchronization for Multi-Video 4D Gaussian Splatting
2025	NeurIPS	TrackingWorld	TrackingWorld: World-centric Monocular 3D Tracking of Almost All Pixels	Project	GitHub
2025		HybridSplat	HybridSplat: Fast Reflection-baked Gaussian Tracing using Hybrid Splatting	Project

---

Unifying depth and camera pose estimation

Year	Venue	Acronym	Paper	Project	GitHub
2021	CVPR	Robust-CVD	Robust consistent video depth estimation	Project	GitHub
2022	ECCV	CasualSAM	Structure and motion from casual videos
2025	CVPR	MegaSam	Megasam: Accurate, fast, and robust structure and motion from casual dynamic videos	Project	GitHub
2025	3DV	Spann3R	3D reconstruction with spatial memory	Project	GitHub
2025	ICLR	MonST3R	MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion	Project	GitHub
2025	CVPR	Align3R	Align3R: Aligned Monocular Depth Estimation for Dynamic Videos	Project	GitHub
2025	CVPR	CUT3R	Continuous 3D Perception Model with Persistent State	Project	GitHub
2025	ICCV	Easi3R	Easi3R: Estimating Disentangled Motion from DUSt3R Without Training	Project	GitHub
2025	ICCV	Geometrycrafter	Geometrycrafter: Consistent geometry estimation for open-world videos with diffusion priors	Project	GitHub
2025	ICCV	Aether	Aether: Geometric-aware unified world modeling	Project	GitHub
2025	ICCV	Geo4D	Geo4d: Leveraging video generators for geometric 4D scene reconstruction	Project	GitHub
2025	arXiv	UniGeo	UniGeo: Taming Video Diffusion for Unified Consistent Geometry Estimation	Project	GitHub
2025	arXiv	Point3R	Point3R: Streaming 3D Reconstruction with Explicit Spatial Pointer Memory	Project	GitHub
2025	arXiv	StreamVGGT	Streaming 4D Visual Geometry Transformer	Project	GitHub
2025	arXiv	π$^3$	π$^3$: Scalable Permutation-Equivariant Visual Geometry Learning	Project	GitHub
2025	arXiv	STream3R	STream3R: Scalable Sequential 3D Reconstruction with Causal Transformer	Project	GitHub
2025	arXiv	ViPE	ViPE: Video Pose Engine for 3D Geometric Perception	Project	GitHub
2025	arXiv	TUN3D	TUN3D: Towards Real-World Scene Understanding from Unposed Images		GitHub
2025	arXiv	MASt3R-Fusion	MASt3R-Fusion: Integrating Feed-Forward Visual Model with IMU, GNSS for High-Functionality SLAM		GitHub
2025		Lyra	Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation	Project	GitHub
2025		WinT3R	WinT3R: Window-Based Streaming Reconstruction with Camera Token Pool	Project	GitHub
2025		Dental3R	Dental3R: Geometry-Aware Pairing for Intraoral 3D Reconstruction from Sparse-View Photographs
2025		OmniVGGT	OmniVGGT: Omni-Modality Driven Visual Geometry Grounded Transformer	Project	GitHub
2025		Depth Anything 3	Depth Anything 3: Recovering the Visual Space from Any Views	Project	GitHub
2025		LiteVGGT	LiteVGGT: Boosting Vanilla VGGT via Geometry-aware Cached Token Merging	Project	GitHub
2025		MUT3R	MUT3R: Motion-aware Updating Transformer for Dynamic 3D Reconstruction
2025		AVGGT	AVGGT: Rethinking Global Attention for Accelerating VGGT

Unifying depth, camera pose, and 3D tracking

Year	Venue	Acronym	Paper	Project	GitHub
2024	NeurIPS	TracksTo4D	Fast Encoder-Based 3D from Casual Videos via Point Track Processing	Project	GitHub
2025	CVPR	Uni4D	Uni4D: Unifying Visual Foundation Models for 4D Modeling from a Single Video	Project	GitHub
2025	arXiv	BA-Track	Back on Track: Bundle Adjustment for Dynamic Scene Reconstruction	Project
2025	arXiv	TrackingWorld	TrackingWorld: World-centric Monocular 3D Tracking of Almost All Pixels	Project	GitHub
2025	CVPR	Stereo4D	Stereo4D: Learning How Things Move in 3D from Internet Stereo Videos	Project	GitHub
2025	arXiv	DPM	Dynamic Point Maps: A Versatile Representation for Dynamic 3D Reconstruction	Project
2025	ICCV	St4RTrack	St4RTrack: Simultaneous 4D Reconstruction and Tracking in the World	Project
2025	arXiv	POMATO	POMATO: Marrying Pointmap Matching with Temporal Motion for Dynamic 3D Reconstruction		GitHub
2025	arXiv	D$^2$USt3R	D$^2$USt3R: Enhancing 3D Reconstruction with 4D Pointmaps for Dynamic Scenes	Project
2025	CVPR	VGGT	VGGT: Visual Geometry Grounded Transformer	Project	GitHub
2025	CVPR	Zero-MSF	Zero-Shot Monocular Scene Flow Estimation in the Wild	Project	GitHub
2025	ICCV	SpatialTrackerV2	SpatialTrackerV2: 3D Point Tracking Made Easy	Project	GitHub
2025	ICCV	MVTracker	Multi-View 3D Point Tracking	Project	GitHub
2025		Trace Anything	Trace Anything: Representing Any Video in 4D via Trajectory Fields	Project	GitHub
2025	WACV	PointSt3R	PointSt3R: Point Tracking through 3D Grounded Correspondence	Project	GitHub
2025		Dens3R	Dens3R: A Foundation Model for 3D Geometry Prediction	Project	GitHub
2025		FlashVGGT	FlashVGGT: Efficient and Scalable Visual Geometry Transformers with Compressed Descriptor Attention	Project	GitHub
2025	NeurIPS	Fin3R	Fin3R: Fine-tuning Feed-forward 3D Reconstruction Models via Monocular Knowledge Distillation	Project	GitHub
2025		HTTM	HTTM: Head-wise Temporal Token Merging for Faster VGGT
2025		AMB3R	AMB3R: Accurate Feed-forward Metric-scale 3D Reconstruction with Backend	Project	GitHub
2025		Selfi	Selfi: Self Improving Reconstruction Engine via 3D Geometric Feature Alignment	Project
2025		UniPR-3D	UniPR-3D: Towards Universal Visual Place Recognition with Visual Geometry Grounded Transformer		GitHub

Level 2 -- 3D scene components

Small-scale 3D object/scene reconstruction

Year	Venue	Acronym	Paper	Project	GitHub
2006	Photogrammetric Computer Vision		Bundle Adjustment Rules
2010	ECCV		Bundle adjustment in the large	Project
2016	CVPR	COLMAP	Structure-from-motion Revisited		GitHub
2016	ECCV		Pixelwise View Selection for Unstructured Multi-View Stereo		GitHub
2019	ECCV	MVSNet	MVSNet: Depth Inference for Unstructured Multi-view Stereo		GitHub
2020	BMVC		Visibility-aware Multi-view Stereo Network		GitHub
2020	CVPR		Cost Volume Pyramid Based Depth Inference for Multi-View Stereo		GitHub
2020	CVPR		Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching		GitHub
2021	ICCV		Pixel-Perfect Structure-from-Motion with Featuremetric Refinement		GitHub
2021	NeurIPS	NeuS	NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction	Project	GitHub
2021	NeurIPS	VolSDF	Volume Rendering of Neural Implicit Surfaces	Project	GitHub
2021	CVPR	PatchMatchNet	PatchmatchNet: Learned Multi-View Patchmatch Stereo		GitHub
2022	ECCV	SparseNeuS	SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse Views	Project	GitHub
2023	ICCV	C2F2NeUS	C2F2NeUS: Cascade Cost Frustum Fusion for High Fidelity and Generalizable Neural Surface Reconstruction
2023	NeurIPS	GenS	GenS: Generalizable Neural Surface Reconstruction from Multi-View Images		GitHub
2023	CVPR	NeAT	NeAT: Learning Neural Implicit Surfaces with Arbitrary Topologies from Multi-view Images	Project	GitHub
2023	CVPR	Neuralangelo	Neuralangelo: High-Fidelity Neural Surface Reconstruction	Project	GitHub
2024	Siggraph	2DGS	2D Gaussian Splatting for Geometrically Accurate Radiance Fields	Project	GitHub
2024	TVCG	PGSR	PGSR: Planar-based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction	Project	GitHub
2025		SolidGS	SolidGS: Consolidating Gaussian Surfel Splatting for Sparse-View Surface Reconstruction	Project
2024	CVPR	UFORecon	UFORecon: Generalizable Sparse-View Surface Reconstruction from Arbitrary and UnFavOrable Sets	Project	GitHub
2024	CVPR	SuGaR	SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering	Project	GitHub
2024	ECCV	SuRF	Surface-Centric Modeling for High-Fidelity Generalizable Neural Surface Reconstruction		GitHub
2024	ECCV	LaRa	LaRa: Efficient Large-Baseline Radiance Fields	Project	GitHub
2024	ECCV	SceneScript	SceneScript: Reconstructing Scenes With An Autoregressive Structured Language Model	Project	GitHub
2024	ECCV	EgoLifter	EgoLifter: Open-world 3D Segmentation for Egocentric Perception	Project	GitHub
2025	TMM	Tri2Plane	Tri2Plane: Advancing Neural Implicit Surface Reconstruction for Indoor Scenes
2025	CVPR	GaussianUDF	GaussianUDF: Inferring Unsigned Distance Functions through 3D Gaussian Splatting	Project	GitHub
2025		SOF	SOF: Sorted Opacity Fields for Fast Unbounded Surface Reconstruction
2025		RNb-NeuS2	RNb-NeuS2: Multi-View Surface Reconstruction Using Normal and Reflectance Cues		GitHub
2025		QuickSplat	QuickSplat: Fast 3D Surface Reconstruction via Learned Gaussian Initialization	Project
2025	CVPR		Geometry Field Splatting with Gaussian Surfels
2025	NeurIPS	ReTR	ReTR: Modeling Rendering Via Transformer for Generalizable Neural Surface Reconstruction	Project	GitHub
2025			3D Gaussian Flats: Hybrid 2D/3D Photometric Scene Reconstruction
2025		VolSplat	VolSplat: Rethinking Feed-Forward 3D Gaussian Splatting with Voxel-Aligned Prediction	Project	GitHub
2025	NeurIPS	HyRF	HyRF: Hybrid Radiance Fields for Memory-efficient and High-quality Novel View Synthesis	Project	GitHub
2025	NeurIPS	GeoSVR	GeoSVR: Taming Sparse Voxels for Geometrically Accurate Surface Reconstruction	Project	GitHub
2025		Distilled-3DGS	Distilled-3DGS:Distilled 3D Gaussian Splatting	Project	GitHub
2025		UP2You	UP2You: Fast Reconstruction of Yourself from Unconstrained Photo Collections	Project	GitHub
2025	NeurIPS	HoloScene	HoloScene: Simulation-Ready Interactive 3D Worlds from a Single Video	Project	GitHub
2025		OMG4	Optimized Minimal 4D Gaussian Splatting	Project	GitHub
2025		Triangle Splatting+	Triangle Splatting+: Differentiable Rendering with Opaque Triangles	Project	GitHub
2025		VGGT-X	VGGT-X: When VGGT Meets Dense Novel View Synthesis	Project	GitHub
2025	NeurIPS	NExF	Learning Neural Exposure Fields for View Synthesis	Project
2025		IGGT	IGGT: Instance-Grounded Geometry Transformer for Semantic 3D Reconstruction		GitHub
2025			Geometry Meets Vision: Revisiting Pretrained Semantics in Distilled Fields	Project	GitHub
2025	TMLR	SCas4D	SCas4D: Structural Cascaded Optimization for Boosting Persistent 4D Novel View Synthesis	Project
2025	NeurIPS	MaterialRefGS	MaterialRefGS: Reflective Gaussian Splatting with Multi-view Consistent Material Inference	Project
2025	ICCV	Liberated-GS	Liberated-GS: 3D Gaussian Splatting Independent from SfM Point Clouds
2025		InstantSfM	InstantSfM: Fully Sparse and Parallel Structure-from-Motion	Project	GitHub
2025		SaLon3R	SaLon3R: Structure-aware Long-term Generalizable 3D Reconstruction from Unposed Images	Project	GitHub
2025		2DGS-R	2DGS-R: Revisiting the Normal Consistency Regularization in 2D Gaussian Splatting
2025		ReAct-GS	Re-Activating Frozen Primitives for 3D Gaussian Splatting	Project	GitHub
2025		EV3DGS	Extreme Views: 3DGS Filter for Novel View Synthesis from Out-of-Distribution Camera Poses	Project
2025	NeurIPS	PLANA3R	PLANA3R: Zero-shot Metric Planar 3D Reconstruction via Feed-Forward Planar Splatting	Project	GitHub
2025	NeurIPS	PlanarGS	PlanarGS: High-Fidelity Indoor 3D Gaussian Splatting Guided by Vision-Language Planar Priors	Project	GitHub
2025		ZeroGS	ZeroGS: Training 3D Gaussian Splatting from Unposed Images	Project	GitHub
2025			Optimizing 3D Gaussian Splattering for Mobile GPUs
2025	AAAI		Gaussian Blending: Rethinking Alpha Blending in 3D Gaussian Splatting	Project	GitHub
2025			Reconstructing 3D Scenes in Native High Dynamic Range
2025	3DV	ConeGS	ConeGS: Error-Guided Densification Using Pixel Cones for Improved Reconstruction with Fewer Primitives	Project	GitHub
2025		YoNoSplat	YoNoSplat: You Only Need One Model for Feedforward 3D Gaussian Splatting	Project
2025			Improving Multi-View Reconstruction via Texture-Guided Gaussian-Mesh Joint Optimization
2025	NeurIPS	IBGS	IBGS: Image-Based Gaussian Splatting
2025		CuriGS	CuriGS: Curriculum-Guided Gaussian Splatting for Sparse View Synthesis	Project
2025	Eurographics	OUGS	OUGS: Active View Selection via Object-aware Uncertainty Estimation in 3DGS
2025		JOGS	JOGS: Joint Optimization of Pose Estimation and 3D Gaussian Splatting
2025	NeurIPS	AtlasGS	AtlasGS: Atlanta-world Guided Surface Reconstruction with Implicit Structured Gaussians	Project	GitHub
2025		ItG-GS	Initialize to Generalize: A Stronger Initialization Pipeline for Sparse-View 3DGS		GitHub
2025		MuSASplat	MuSASplat: Efficient Sparse-View 3D Gaussian Splats via Lightweight Multi-Scale Adaptation
2025	ICCV	RobustSplat	RobustSplat: Decoupling Densification and Dynamics for Transient-Free 3DGS	Project	GitHub
2025		RobustSplat++	RobustSplat: Decoupling Densification and Dynamics for Transient-Free 3DGS
2025		UTrice	UTrice: Unifying Primitives in Differentiable Ray Tracing and Rasterization via Triangles for Particle-Based 3D Scenes
2025			Radiance Meshes for Volumetric Reconstruction	Project	GitHub
2025		EGGS	EGGS: Exchangeable 2D/3D Gaussian Splatting for Geometry-Appearance Balanced Novel View Synthesis	Project	GitHub
2025	NeurIPS	SV2CGS	Novel View Synthesis from A Few Glimpses via Test-Time Natural Video Completion	Project
2025	Siggraph Asia	DeMapGS	DeMapGS: Simultaneous Mesh Deformation and Surface Attribute Mapping via Gaussian Splatting	Project
2025		GaMO	GaMO: Geometry-aware Multi-view Diffusion Outpainting for Sparse-View 3D Reconstruction	Project	GitHub
2025		EcoSplat	EcoSplat: Efficiency-controllable Feed-forward 3D Gaussian Splatting from Multi-view Images	Project	GitHub

---

Large-scale 3D scene reconstruction

Year	Venue	Acronym	Paper	Project	GitHub
2021	CVPR	NeRF++	NeRF++: Analyzing and Improving Neural Radiance Fields		GitHub
2021	CVPR	NeuralRecon	NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video	Project	GitHub
2021	NeurIPS	TransformerFusion	TransformerFusion: Monocular RGB Scene Reconstruction using Transformers		GitHub
2022	CVPR	Block-NeRF	Block-NeRF: Scalable Large Scene Neural View Synthesis	Project
2022	CVPR	Mega-NeRF	Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughs	Project	GitHub
2022	ECCV	BungeeNeRF	BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Rendering	Project	GitHub
2022	NeurIPS	MonoSDF	MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction	Project	GitHub
2023	ICCV	FineRecon	FineRecon: Depth-aware Feed-forward Network for Detailed 3D Reconstruction		GitHub
2023	ICCV	CVRecon	CVRecon: Rethinking 3D Geometric Feature Learning For Neural Reconstruction	Project	GitHub
2023	ICCV	Zip-NeRF	Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields	Project	GitHub
2023	CVPR	F2-NeRF	F2-NeRF: Fast Neural Radiance Field Training with Free Camera Trajectories	Project	GitHub
2023	CVPR	DG-Recon	DG-Recon: Depth-Guided Neural 3D Scene Reconstruction
2023	CVPR	VisFusion	VisFusion: Visibility-aware Online 3D Scene Reconstruction from Videos	Project	GitHub
2023	AAAI	Flora	Flora: Dual-Frequency LOss-Compensated ReAl-Time Monocular 3D Video Reconstruction
2023	ICLR	Switch-NeRF	Switch-nerf: Learning scene decomposition with mixture of experts for large-scale neural radiance fields	Project	GitHub
2024	ECCV	CityGaussian	CityGaussian: Real-time High-quality Large-Scale Scene Rendering with Gaussians	Project	GitHub
2024		Octree-GS	Octree-GS: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussians	Project	GitHub
2024		SCALAR-NeRF	SCALAR-NeRF: SCAlable LARge-scale Neural Radiance Fields for Scene Reconstruction
2024	Siggraph Asia		Gaussian Opacity Fields: Efficient Adaptive Surface Reconstruction in Unbounded Scenes	Project	GitHub
2024	CVPR	Scaffold-GS	Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering	Project	GitHub
2024	CVPR	MonoSelfRecon	MonoSelfRecon: Purely Self-Supervised Explicit Generalizable 3D Reconstruction of Indoor Scenes from Monocular RGB Views		GitHub
2025		CityGS-X	CityGS-X: A Scalable Architecture for Efficient and Geometrically Accurate Large-Scale Scene Reconstruction	Project	GitHub
2025		LODGE	LODGE: Level-of-Detail Large-Scale Gaussian Splatting with Efficient Rendering	Project
2025	ICLR	CityGaussianV2	CityGaussianV2: Efficient and Geometrically Accurate Reconstruction for Large-Scale Scenes	Project	GitHub
2025	TMM	DetailRecon	Focusing on Detailed Regions for Online Monocular 3D Reconstruction
2025	IROS		3D Gaussian Splatting for Fine-Detailed Surface Reconstruction in Large-Scale Scene
2025	EGSR		Multiview Geometric Regularization of Gaussian Splatting for Accurate Radiance Fields
2025	Siggraph		Photoreal Scene Reconstruction from an Egocentric Device	Project	GitHub
2025		ARTDECO	ARTDECO: Towards Efficient and High-Fidelity On-the-Fly 3D Reconstruction with Structured Scene Representation	Project	GitHub
2025		ReSplat	ReSplat: Learning Recurrent Gaussian Splats	Project
2025		NVSim	NVSim: Novel View Synthesis Simulator for Large Scale Indoor Navigation	Project
2025		FastGS	FastGS: Training 3D Gaussian Splatting in 100 Seconds	Project	GitHub
2025		SwiftVGGT	SwiftVGGT: A Scalable Visual Geometry Grounded Transformer for Large-Scale Scenes	Project	GitHub
2025		MetroGS	MetroGS: Efficient and Stable Reconstruction of Geometrically Accurate High-Fidelity Large-Scale Scenes	Project	GitHub
2025		G3Splat	G3Splat: Geometrically Consistent Generalizable Gaussian Splatting	Project
2025		WorldWarp	WorldWarp: Propagating 3D Geometry with Asynchronous Video Diffusion	Project	GitHub
2025		Nexels	Nexels: Neurally-Textured Surfels for Real-Time Novel View Synthesis with Sparse Geometries	Project	GitHub
2025			On-the-fly Large-scale 3D Reconstruction from Multi-Camera Rigs
2025	OpenSource Studio	LightField Studio	LightField Studio: A high-performance C++ and CUDA implementation of 3D Gaussian Splatting	Project

Level 3 -- 4D dynamic scenes

General 4D scene reconstruction

Year	Venue	Acronym	Paper	Project	GitHub
2005	Siggraph		Video-based rendering
2017	CVPR		3D Menagerie: Modeling the 3D shape and pose of animals
2020	NeurIPS		Online adaptation for consistent mesh reconstruction in the wild
2020	CVPR		Novel View Synthesis of Dynamic Scenes with Globally Coherent Depths from a Monocular Camera	Project
2021		DCT-NeRF	Neural Trajectory Fields for Dynamic Novel View Synthesis
2021	IJCV		The Isowarp: The Template-Based Visual Geometry of Isometric Surfaces
2021	CVPR		Space-time Neural Irradiance Fields for Free-Viewpoint Video	Project
2021	CVPR	LASR	LASR: Learning Articulated Shape Reconstruction from a Monocular Video	Project	GitHub
2021	CVPR		Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes	Project	GitHub
2021	ICCV		Neural radiance flow for 4d view synthesis and video processing	Project	GitHub
2021	ICCV	Nerfies	Nerfies: Deformable Neural Radiance Fields	Project	GitHub
2021	ICCV		Dynamic View Synthesis from Dynamic Monocular Video	Project	GitHub
2021	ICCV		Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video	Project	GitHub
2021	Siggraph Asia	HyperNeRF	HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields	Project	GitHub
2021	CVPR	D-NeRF	D-NeRF: Neural Radiance Fields for Dynamic Scenes	Project	GitHub
2022	CVPR		ϕ-SfT: Shape-from-Template with a Physics-Based Deformation Model	Project	GitHub
2022	CVPR	BANMo	BANMo: Building Animatable 3D Neural Models from Many Casual Videos	Project	GitHub
2022	CVPR		Revealing Occlusions with 4D Neural Fields	Project	GitHub
2023		EmerNeRF	EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervision	Project	GitHub
2023		DeformGS	DeformGS: Scene Flow in Highly Deformable Scenes for Deformable Object Manipulation	Project	GitHub
2023			Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis	Project	GitHub
2023			An Efficient 3D Gaussian Representation for Monocular/Multi-view Dynamic Scenes		GitHub
2023	CVPR	Tensor4D	Tensor4D : Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Rendering	Project	GitHub
2023	CVPR		Unbiased 4D: Monocular 4D Reconstruction with a Neural Deformation Model	Project	GitHub
2023	CVPR	DyLiN	DyLiN: Making Light Field Networks Dynamic	Project	GitHub
2023	CVPR	HexPlane	HexPlane: A Fast Representation for Dynamic Scenes		GitHub
2023	CVPR	K-Planes	K-Planes: Explicit Radiance Fields in Space, Time, and Appearance	Project	GitHub
2023	CVPR		Flow supervision for Deformable NeRF	Project	GitHub
2023	CVPR		Neural Scene Chronology	Project	GitHub
2023	CVPR		Spacetime Surface Regularization for Neural Dynamic Scene Reconstruction
2023	CVPR		Robust Dynamic Radiance Fields	Project	GitHub
2023	ICCV	PPR	PPR: Physically Plausible Reconstruction from Monocular Videos	Project	GitHub
2023	ICCV	MonoNeRF	MonoNeRF: Learning a Generalizable Dynamic Radiance Field from Monocular Videos		GitHub
2023	TVCG	NeRFPlayer	NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fields	Project	GitHub
2024	CVPR	REACTO	REACTO: Reconstructing Articulated Objects from a Single Video	Project	GitHub
2024	CVPR		Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis	Project	GitHub
2024	CVPR	3DGStream	3DGStream: On-the-Fly Training of 3D Gaussians for Efficient Streaming of Photo-Realistic Free-Viewpoint Videos	Project	GitHub
2024	CVPR		4D Gaussian Splatting for Real-Time Dynamic Scene Rendering	Project	GitHub
2024	CVPR		Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction	Project	GitHub
2024	CVPR	DaReNeRF	DaReNeRF: Direction-aware Representation for Dynamic Scenes
2024	CVPR		Neural Parametric Gaussians for Monocular Non-Rigid Object Reconstruction	Project	GitHub
2024	CVPR		3D Geometry-aware Deformable Gaussian Splatting for Dynamic View Synthesis	Project	GitHub
2024	CVPR	SC-GS	SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes	Project	GitHub
2024	CVPRW	FlowIBR	FlowIBR: Leveraging Pre-Training for Efficient Neural Image-Based Rendering of Dynamic Scenes	Project
2024	ICLR		Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting	Project	GitHub
2024	Siggraph		4D-Rotor Gaussian Splatting: Towards Efficient Novel View Synthesis for Dynamic Scenes	Project	GitHub
2024	Siggraph Asia		Dynamic Gaussian Marbles for Novel View Synthesis of Casual Monocular Videos	Project	GitHub
2024		MoSca	MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffolds	Project	GitHub
2024		Das3R	DAS3R: Dynamics-Aware Gaussian Splatting for Static Scene Reconstruction		GitHub
2024			Feed-Forward Bullet-Time Reconstruction of Dynamic Scenes from Monocular Videos	Project
2024			Shape of Motion: 4D Reconstruction from a Single Video	Project	GitHub
2024	TVCG		Decoupling Dynamic Monocular Videos for Dynamic View Synthesis
2024	TMLR	GaussianFlow	GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creation	Project	GitHub
2024	ECCV		Per-Gaussian Embedding-Based Deformation for Deformable 3D Gaussian Splatting	Project	GitHub
2024	NeurIPS	DN-4DGS	DN-4DGS: Denoised Deformable Network with Temporal-Spatial Aggregation for Dynamic Scene Rendering		GitHub
2024	NeurIPS	MotionGS	MotionGS: Exploring Explicit Motion Guidance for Deformable 3D Gaussian Splatting	Project	GitHub
2024	ICLR		Neural SDF Flow for 3D Reconstruction of Dynamic Scenes		GitHub
2024	ICLR		Pseudo-Generalized Dynamic View Synthesis from a Video	Project	GitHub
2024		DynaSurfGS	DynaSurfGS: Dynamic Surface Reconstruction with Planar-based Gaussian Splatting	Project	GitHub
2024		st-2dgs	Space-time 2D Gaussian Splatting for Accurate Surface Reconstruction under Complex Dynamic Scenes	Project	GitHub
2024		DGNS	DGNS: Deformable Gaussian Splatting and Dynamic Neural Surface for Monocular Dynamic 3D Reconstruction
2024		Trans4D	Trans4D: Realistic Geometry-Aware Transition for Compositional Text-to-4D Synthesis		GitHub
2025	ICLR	DG-Mesh	Dynamic Gaussians Mesh: Consistent Mesh Reconstruction from Dynamic Scenes	Project	GitHub
2025	ICLR	MoDGS	MoDGS: Dynamic Gaussian Splatting from Casually-captured Monocular Videos with Depth Priors	Project	GitHub
2025	WACV	AT-GS	Adaptive and Temporally Consistent Gaussian Surfels for Multi-view Dynamic Reconstruction	Project	GitHub
2025	CVPR	SpectroMotion	SpectroMotion: Dynamic 3D Reconstruction of Specular Scenes	Project	GitHub
2025			Light of Normals: Unified Feature Representation for Universal Photometric Stereo	Project	GitHub
2025		WideRange4D	WideRange4D: Enabling High-Quality 4D Reconstruction with Wide-Range Movements and Scenes		GitHub
2025		MoVieS	MoVieS: Motion-Aware 4D Dynamic View Synthesis in One Second	Project	GitHub
2025	CVPR	Instruct-4DGS	Instruct-4DGS: Efficient Dynamic Scene Editing via 4D Gaussian-based Static-Dynamic Separation	Project	GitHub
2025		Pixie	Pixie: Fast and Generalizable Supervised Learning of 3D Physics from Pixels	Project	GitHub
2025		MeshSplat	MeshSplat: Generalizable Sparse-View Surface Reconstruction via Gaussian Splatting	Project	GitHub
2025	ICCV	LongSplat	LongSplat: Robust Unposed 3D Gaussian Splatting for Casual Long Videos	Project	GitHub
2025	NeurIPS	Instant4D	Instant4D: 4D Gaussian Splatting in Minutes	Project	GitHub
2025		Human3R	Human3R: Everyone Everywhere All at Once	Project	GitHub
2025		TTT3R	TTT3R: 3D Reconstruction as Test-Time Training	Project	GitHub
2025		Mono4DGS-HDR	Mono4DGS-HDR: High Dynamic Range 4D Gaussian Splatting from Alternating-exposure Monocular Videos	Project	GitHub
2025		MoE-GS	MoE-GS: Mixture of Experts for Dynamic Gaussian Splatting	Project
2025		TrackerSplat	TrackerSplat: Exploiting Point Tracking for Fast and Robust Dynamic 3D Gaussians Reconstruction		GitHub
2025		Diff4Splat	Diff4Splat: Controllable 4D Scene Generation with Latent Dynamic Reconstruction Models	Project
2025		CogNVS	Reconstruct, Inpaint, Finetune: Dynamic Novel-view Synthesis from Monocular Videos	Project	GitHub
2025	NeurIPS	4D3R	4D3R: Motion-Aware Neural Reconstruction and Rendering of Dynamic Scenes from Monocular Videos
2025		PAGE-4D	PAGE-4D: Disentangled Pose and Geometry Estimation for 4D Perception
2025			Dynamic Visual SLAM using a General 3D Prior
2025	NeurIPS	Flux4D	Flux4D: Flow-based Unsupervised 4D Reconstruction	Project
2025		VGGT4D	VGGT4D: Mining Motion Cues in Visual Geometry Transformers for 4D Scene Reconstruction	Project	GitHub
2025		4D-VGGT	4D-VGGT: A General Foundation Model with SpatioTemporal Awareness for Dynamic Scene Geometry Estimation
2025		Any4D	Any4D: Unified Feed-Forward Metric 4D Reconstruction	Project	GitHub
2025		D2GSLAM	D2GSLAM: 4D Dynamic Gaussian Splatting SLAM
2025		Play4D	Play4D: Accelerated and Interactive Free-viewpoint Video Streaming for Virtual Reality and Light Field Displays	Project
2025	Siggraph Asia		Prior-Enhanced Gaussian Splatting for Dynamic Scene Reconstruction from Casual Video	Project
2025		MoRel	MoRel: Long-Range Flicker-Free 4D Motion Modeling via Anchor Relay-based Bidirectional Blending with Hierarchical Densification	Project	GitHub

---

Human-centric dynamic modeling - SMPL

Year	Venue	Acronym	Paper	Project	GitHub
2015	TOG	SMPL	Smpl: A skinned multi-person linear model	Project
2016	ECCV	SMPLify	Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image	Project	GitHub
2018	CVPR	HMR	End-to-end Recovery of Human Shape and Pose	Project	GitHub
2018	CVPR		Learning to Estimate 3D Human Pose and Shape from a Single Color Image
2019	CVPR	GraphCMR	Convolutional Mesh Regression for Single-Image Human Shape Reconstruction	Project	GitHub
2019	CVPR	SMPL-X	Expressive Body Capture: 3D Hands, Face, and Body from a Single Image	Project	GitHub
2019	CVPR	HoloPose	HoloPose: Holistic 3D Human Reconstruction In-The-Wild
2019	CVPR		Learning 3D Human Dynamics from Video	Project	GitHub
2019	ICCV	SPIN	Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop	Project	GitHub
2019	ICCV	DenseRaC	DenseRaC: Joint 3D Pose and Shape Estimation by Dense Render-and-Compare
2019	MM	DaNet	DaNet: Decompose-and-aggregate Network for 3D Human Shape and Pose Estimation
2019	NeurIPS		Sim2real transfer learning for 3D human pose estimation: motion to the rescue
2020	CVPR	DecoMR	3D Human Mesh Regression with Dense Correspondence		GitHub
2020	CVPR	VIBE	VIBE: Video Inference for Human Body Pose and Shape Estimation		GitHub
2020	TOG	PhysCap	PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Time	Project	GitHub
2020	ECCV		Human Body Model Fitting by Learned Gradient Descent		GitHub
2020	ECCV	HKMR	Hierarchical Kinematic Human Mesh Recovery
2020	ECCV	I2L-MeshNet	I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Image		GitHub
2021	CVPR	METRO	End-to-End Human Pose and Mesh Reconstruction with Transformers		GitHub
2021	CVPR	HybrIK	HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation	Project	GitHub
2021	ICCV	MAED	Encoder-decoder with Multi-level Attention for 3D Human Shape and Pose Estimation		GitHub
2021	ICCV	SPEC	SPEC: Seeing People in the Wild with an Estimated Camera	Project	GitHub
2021	ICCV	PyMAF	PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop	Project	GitHub
2021	ICCV	HuMoR	HuMoR: 3D Human Motion Model for Robust Pose Estimation	Project	GitHub
2021	ICCV	PARE	PARE: Part Attention Regressor for 3D Human Body Estimation	Project	GitHub
2022	CVPR	GLAMR	GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras	Project	GitHub
2022	ECCV	CLIFF	CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimation		GitHub
2022	ECCV	D&D	D&D: Learning Human Dynamics from Dynamic Camera		GitHub
2023	TPAMI	PyMAF-X	PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images	Project	GitHub
2023	CVPR	TRACE	TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D Environments	Project	GitHub
2023	CVPR	SLAHMR	Decoupling Human and Camera Motion from Videos in the Wild	Project	GitHub
2023	CVPR	IPMAN	3D Human Pose Estimation via Intuitive Physics	Project	GitHub
2023	CVPR	NIKI	NIKI: Neural Inverse Kinematics with Invertible Neural Networks for 3D Human Pose and Shape Estimation		GitHub
2023	ICCV	HMR2.0	Humans in 4D: Reconstructing and Tracking Humans with Transformers	Project	GitHub
2023	NeurIPS	SMPLer-X	SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimation	Project	GitHub
2024	CVPR	TokenHMR	TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representation	Project	GitHub
2024	CVPR	WHAM	WHAM: Reconstructing World-grounded Humans with Accurate 3D Motion	Project	GitHub
2024	ECCV	TRAM	TRAM: Global Trajectory and Motion of 3D Humans from in-the-wild Videos	Project	GitHub
2024	ECCV	COIN	COIN: Control-Inpainting Diffusion Prior for Human and Camera Motion Estimatio	Project
2024	NeurIPS	NLF	Neural Localizer Fields for Continuous 3D Human Pose and Shape Estimation	Project	GitHub
2024	Siggraph Asia	GVHMR	World-Grounded Human Motion Recovery via Gravity-View Coordinates	Project	GitHub
2025	3DV	CameraHMR	CameraHMR: Aligning People with Perspective	Project	GitHub
2025	AAAI	GenHMR	GenHMR: Generative Human Mesh Recovery	Project
2025	ICLR	CoMotion	CoMotion: Concurrent Multi-person 3D Motion		GitHub
2025	CVPR	BLADE	BLADE: Single-view Body Mesh Learning through Accurate Depth Estimation	Project
2025	CVPR	HSMR	Reconstructing Humans with a Biomechanically Accurate Skeleton	Project	GitHub
2025	ICCV	GENMO	GENMO: A GENarlist Model for Human MOtion	Project
2025		FastHMR	FastHMR: Accelerating Human Mesh Recovery via Token and Layer Merging with Diffusion Decoding	Project	GitHub
2025	WACV	SkelSplat	SkelSplat: Robust Multi-view 3D Human Pose Estimation with Differentiable Gaussian Rendering	Project	GitHub
2025		SAM 3D	SAM 3D: 3Dfy Anything in Images	Project	GitHub

---

Human-centric dynamic modeling - Egocentric

Year	Venue	Acronym	Paper	Project	GitHub
2022	ICCV	Ego-Fusion	EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognition	Project	GitHub
2022	ECCV	AvatarPoser	Avatarposer: Articulated full-body pose tracking from sparse motion sensing	Project	GitHub
2022	NeurIPS	Epic-Kitchens	EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations	Project
2023	CVPR	EgoEgo	Ego-Body Pose Estimation via Ego-Head Pose Estimation	Project	GitHub
2023	CVPR	BoDiffusion	BoDiffusion: Diffusing Sparse Observations for Full-Body Human Motion Synthesis		GitHub
2023	CVPR	SeceneEgo	Scene-aware egocentric 3d human pose estimation		GitHub
2023	CVPR	AGRoL	Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Model	Project	GitHub
2023	NeurIPS	Epic Fields	EPIC Fields: Marrying 3D Geometry and Video Understanding	Project	GitHub
2024	ECCV	EgoPoser	EgoPoser: Robust Real-Time Egocentric Pose Estimation from Sparse and Intermittent Observations Everywhere	Project	GitHub
2024	ECCV	MANIKIN	MANIKIN: Biomechanically Accurate Neural Inverse Kinematics for Human Motion Estimation	Project
2024	ECCV	AMEGO	AMEGO: Active Memory from long EGOcentric videos	Project	GitHub
2024	CVPR	EgoWholeBody	Egocentric Whole-Body Motion Capture with FisheyeViT and Diffusion-Based Motion Refinement		GitHub
2024	CVPR	EventEgo3D	EventEgo3D: 3D Human Motion Capture from Egocentric Event Streams	Project	GitHub
2024	CVPR	HMD-Poser	HMD-Poser: On-Device Real-time Human Motion Tracking from Scalable Sparse Observations	Project	GitHub
2024	arXiv	HOI-Ref	HOI-Ref: Hand-Object Interaction Referral in Egocentric Vision	Project	GitHub
2025	3DV	HMD$^2$	HMD$^2$: Environment-aware Motion Generation from Single Egocentric Head-Mounted Device	Project
2025	3DV	OSNOM	Spatial Cognition from Egocentric Video: Out of Sight, Not Out of Mind	Project	GitHub
2025	IJCV	EventEgo3D++	EventEgo3D++: 3D Human Motion Capture from a Head Mounted Event Camera	Project	GitHub
2025	CVPR	EgoLM	Egolm: Multi-modal language model of egocentric motions	Project
2025	CVPR	EgoAllo	Estimating Body and Hand Motion in an Ego-sensed World	Project	GitHub
2025	CVPR	Ego4o	Ego4o: Egocentric Human Motion Capture and Understanding from Multi-Modal Input	Project
2025	CVPR	FRAME	FRAME: Floor-aligned Representation for Avatar Motion from Egocentric Video	Project	GitHub
2025	arXiv	EgoH4	The Invisible EgoHand: 3D Hand Forecasting through EgoBody Pose Estimation	Project	GitHub
2025	ICCV	UniEgoMotion	UniEgoMotion: A Unified Model for Egocentric Motion Reconstruction, Forecasting, and Generation	Project	GitHub
2025	ICCVW	EgoInstruct	EgoInstruct: An Egocentric Video Dataset of Face-to-face Instructional Interactions with Multi-modal LLM Benchmarking
2025		EgoTwin	EgoTwin: Dreaming Body and View in First Person	Project

---

Human-centric dynamic modeling - Appearance-rich

Year	Venue	Acronym	Paper	Project	GitHub
2018	TOG	MonoPerfCap	MonoPerfCap: Human Performance Capture from Monocular Video
2018	CVPR	VideoAvatars	Video Based Reconstruction of 3D People Models		GitHub
2018	TOG	LiveCap	LiveCap: Real-time Human Performance Capture from Monocular Video
2021	3DV		Human Performance Capture from Monocular Video in the Wild	Project	GitHub
2021	NeurIPS	A-NeRF	A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose	Project	GitHub
2021	NeurIPS	ViSER	ViSER: Video-Specific Surface Embeddings for Articulated 3D Shape Reconstruction	Project	GitHub
2022	CVPR	SelfRecon	SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Video	Project	GitHub
2022	ECCV	DANBO	DANBO: Disentangled Articulated Neural Body Representations via Graph Neural Networks	Project	GitHub
2022	ECCV	AvatarPoser	Avatarposer: Articulated full-body pose tracking from sparse motion sensing		GitHub
2022	NeurIPS	FOF	FOF: Learning Fourier Occupancy Field for Monocular Real-time Human Reconstruction	Project	GitHub
2023	CVPR	Vid2Avatar	Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decomposition	Project	GitHub
2024	CVPR	HUGS	HUGS: Human Gaussian Splats	Project	GitHub
2024	CVPR	GaussianAvatar	GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians	Project	GitHub
2024	CVPR	Animatable Gaussians	Animatable Gaussians: Learning Pose-dependent Gaussian Maps for High-fidelity Human Avatar Modeling	Project	GitHub
2024	CVPR	GPS-Gaussian	GPS-Gaussian: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesis	Project	GitHub
2024	CVPR	3DGS-Avatar	3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting	Project	GitHub
2025	CVPR	GauSTAR	GauSTAR: Gaussian Surface Tracking and Reconstruction	Project	GitHub
2025	Siggraph Asia		Detail Enhanced Gaussian Splatting for Large-Scale Volumetric Capture	Project	GitHub
2025		AHA	AHA! Animating Human Avatars in Diverse Scenes with Gaussian Splatting
2025	IROS	STG-Avatar	STG-Avatar: Animatable Human Avatars via Spacetime Gaussian		GitHub

Level 4 -- Interaction among scene components

SMPL-based human-centric interaction - HOI

Year	Venue	Acronym	Paper	Project	GitHub
2016	TOG	PiGraphs	PiGraphs: learning interaction snapshots from observations	Project	GitHub
2020	ECCV	PHOSA	Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wild	Project	GitHub
2021	arXiv	D3D-HOI	3d-hoi: Dynamic 3d human-object interactions from videos		GitHub
2021	CVPR	GraviCap	Gravity-Aware Monocular 3D Human-Object Reconstruction	Project	GitHub
2022	CVPR	BEHAVE	BEHAVE: Dataset and Method for Tracking Human Object Interactions	Project	GitHub
2022	ECCV	CHORE	CHORE: Contact, Human and Object REconstruction from a single RGB image	Project	GitHub
2023	ICCV	CHAIRS	Full-Body Articulated Human-Object Interaction	Project	GitHub
2023	IJCAI	StackFLOW	StackFLOW: Monocular Human-Object Reconstruction by Stacked Normalizing Flow with Offset	Project	GitHub
2023	CVPR	VisTracker	Visibility Aware Human-Object Interaction Tracking from Single RGB Camera	Project	GitHub
2024	IJCV	InterCap	InterCap: Joint Markerless 3D Tracking of Humans and Objects in Interaction	Project	GitHub
2024	MM	WildHOI	Monocular Human-Object Reconstruction in the Wild	Project	GitHub
2024	CVPR	I'M HOI	I'M HOI: Inertia-aware Monocular Capture of 3D Human-Object Interactions	Project	GitHub
2024	CVPR	HDM	Template Free Reconstruction of Human-object Interaction with Procedural Interaction Generation	Project	GitHub
2024	NeurIPS	InterDreamer	InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interaction	Project
2025	3DV	InterTrack	InterTrack: Tracking Human Object Interaction without Object Templates	Project	GitHub
2025	CVPR	InteractVLM	InteractVLM: 3D Interaction Reasoning from 2D Foundational Models	Project	GitHub

SMPL-based human-centric interaction - HSI

Year	Venue	Acronym	Paper	Project	GitHub
2019	ICCV	PROX	Resolving 3D Human Pose Ambiguities with 3D Scene Constraints	Project	GitHub
2020	ECCV	HMP	Long-term Human Motion Prediction with Scene Context	Project	GitHub
2022	CVPR	RICH	Capturing and Inferring Dense Full-Body Human-Scene Contact	Project	GitHub
2022	ECCV	SitComs3D	The One Where They Reconstructed 3D Humans and Environments in TV Shows	Project	GitHub
2023	CVPR	CIRCLE	CIRCLE: Capture In Rich Contextual Environments		GitHub
2024	CVPR	TRUMANS	Scaling Up Dynamic Human-Scene Interaction Modeling	Project	GitHub
2025	arXiv	JOSH	Joint Optimization for 4D Human-Scene Reconstruction in the Wild	Project	GitHub
2025	CVPR	ODHSR	ODHSR: Online Dense 3D Reconstruction of Humans and Scenes from Monocular Videos	Project

SMPL-based human-centric interaction - HHI

Year	Venue	Acronym	Paper	Project	GitHub
2021	ICCV	ROMP	Monocular, One-stage, Regression of Multiple 3D People		GitHub
2022	CVPR	BEV	Putting People in their Place: Monocular Regression of 3D People in Depth	Project	GitHub
2023	Siggraph Asia	CloseMoCap	Reconstructing Close Human Interactions from Multiple Views		GitHub
2023	CVPR	Hi4D	Hi4D: 4D Instance Segmentation of Close Human Interaction	Project	GitHub
2024	CVPR	BUDDI	Generative Proxemics: A Prior for 3D Social Interaction from Images	Project	GitHub
2024	CVPR	CloseInt	Closely Interactive Human Reconstruction with Proxemics and Physics-Guided Adaption		GitHub
2024	CVPR	MultiPhys	MultiPhys: Multi-Person Physics-aware 3D Motion Estimation	Project	GitHub
2024	ECCV	AvatarPose	AvatarPose: Avatar-guided 3D Pose Estimation of Close Human Interaction from Sparse Multi-view Videos	Project	GitHub
2024	NeurIPS	Harmony4D	Harmony4D: A Video Dataset for In-The-Wild Close Human Interactions	Project	GitHub

---

Appearance-rich human-centric interaction

Year	Venue	Acronym	Paper	Project	GitHub
2022	ECCV	NeuMan	NeuMan: Neural Human Radiance Field from a Single Video	Project	GitHub
2023	ICCV	HOSNeRF	HOSNeRF: Dynamic Human-Object-Scene Neural Radiance Fields from a Single Video	Project	GitHub

---

Egocentric human-centric interaction

Year	Venue	Acronym	Paper	Project	GitHub
2021	ICCV	H2O	H2O: Two Hands Manipulating Objects for First Person Interaction Recognition
2022	CVPR	HOI4D	HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interaction
2023		Aria	Project Aria: A New Tool for Egocentric Multi-Modal AI Research
2023	MICCAI	POV-Surgery	POV-Surgery: A Dataset for Egocentric Hand and Tool Pose Estimation During Surgical Activities	Project	GitHub
2024	CVPR	Ego-Exo4D	Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives	Project
2024		Nymeria	Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wild	Project
2025	CVPR	HOT3D	Introducing HOT3D: An Egocentric Dataset for 3D Hand and Object Tracking	Project

Level 5 -- Incorporation of physical laws and constraints

Dynamic 4D human simulation with physics

Year	Venue	Acronym	Paper	Project	GitHub
1995	Siggraph		Animating human athletics
2002	TOG		Interactive control of avatars animated with human motion data
2007	TOG	SIMBICON	Simbicon: Simple biped locomotion control
2007	Siggraph		Construction and optimal search of interpolated motion graphs
2007	Siggraph		Near-optimal Character Animation with Continuous Control
2010	Siggraph Asia		Motion fields for interactive character locomotion
2010	TOG		Generalized biped walking control
2010	TOG		Spatial relationship preserving character motion adaptation
2012	TOG		Continuous character control with low-dimensional embeddings
2014	TOG		Learning bicycle stunts
2016	NeurIPS	GAIL	Generative Adversarial Imitation Learning
2017	TOG	PFNN	Phase-functioned neural networks for character control		GitHub
2017	TOG		Learning to schedule control fragments for physics-based characters using deep q-learning
2018	TOG	MANN	Mode-adaptive neural networks for quadruped motion control		GitHub
2018	TOG		Learning basketball dribbling skills using trajectory optimization and deep reinforcement learning
2018	TOG	DeepMimic	DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills		GitHub
2020		UniCon	UniCon: Universal Neural Controller For Physics-based Character Motion	Project
2020	TOG	ScaDiver	A scalable approach to control diverse behaviors for physically simulated characters	Project	GitHub
2020	Siggraph	MotionVAEs	Character Controllers using Motion VAEs		GitHub
2021	TOG	AMP	AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control
2022	NeurIPS	MoCapAct	MoCapAct: A Multi-Task Dataset for Simulated Humanoid Control	Project	GitHub
2022	Siggraph Asia	PADL	PADL: Language-Directed Physics-Based Character Control		GitHub
2022	TOG	ASE	ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters	Project	GitHub
2022	TOG	ControlVAE	ControlVAE: Model-Based Learning of Generative Controllers for Physics-Based Characters	Project	GitHub
2023	Siggraph	CALM	CALM: Conditional Adversarial Latent Models for Directable Virtual Characters	Project	GitHub
2023	Siggraph		Simulation and Retargeting of Complex Multi-Character Interactions		GitHub
2023	Siggraph	PMP	PMP: Learning to Physically Interact with Environments using Part-wise Motion Priors		GitHub
2023	CVPR		Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusion	Project	GitHub
2023	TOG		Learning Physically Simulated Tennis Skills from Broadcast Videos	Project	GitHub
2024	3DV		Physically Plausible Full-Body Hand-Object Interaction Synthesis	Project
2024	NeurIPS	Omnigrasp	Omnigrasp: Grasping Diverse Objects with Simulated Humanoids		GitHub
2024	Siggraph	SuperPADL	SuperPADL: Scaling Language-Directed Physics-Based Control with Progressive Supervised Distillation	Project
2024	Siggraph Asia	PDP	PDP: Physics-Based Character Animation via Diffusion Policy
2024	TOG	MaskedMimic	MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpainting	Project	GitHub
2024	CVPR	PACER++	PACER+: On-Demand Pedestrian Animation Controller in Driving Scenarios		GitHub
2025	ICLR	CLoSD	CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character control	Project	GitHub
2025	ICLR		Hierarchical World Models as Visual Whole-Body Humanoid Controllers	Project	GitHub
2025	CVPR	SkillMimic	SkillMimic: Learning Basketball Interaction Skills from Demonstrations	Project	GitHub
2025	ICRA	HOVER	HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots	Project	GitHub
2025	RSS	ASAP	ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills	Project	GitHub
2025		UniPhys	UniPhys: Unified Planner and Controller with Diffusion for Flexible Physics-Based Character Control	Project
2025		VisualMimic	VisualMimic: Visual Humanoid Loco-Manipulation via Motion Tracking and Generation	Project	GitHub
2025	NeurIPS	EgoBridge	EgoBridge: Domain Adaptation for Generalizable Imitation from Egocentric Human Data	Project

---

3D scene reconstruction with physical plausibility

Year	Venue	Acronym	Paper	Project	GitHub
2022	CVPR	AugNeRF	Aug-NeRF: Training Stronger Neural Radiance Fields with Triple-Level Physically-Grounded Augmentations		GitHub
2024	ECCV	PhysDreamer	PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generation	Project	GitHub
2024	Siggraph Asia		Planar Reflection-Aware Neural Radiance Fields
2024	NeurIPS	PhyRecon	Phyrecon: Physically plausible neural scene reconstruction	Project	GitHub
2025	ICML	PhysicsNeRF	PhysicsNeRF: Physics-Guided 3D Reconstruction from Sparse Views		GitHub
2025	CVPR	PBR-NeRF	PBR-NeRF: Inverse Rendering with Physics-Based Neural Fields		GitHub
2025		CAST	CAST: Component-Aligned 3D Scene Reconstruction from an RGB Image	Project