Jiajun Wu

Google Scholar / DBLP / Stanford Profiles

Research Communities

I am an Assistant Professor of Computer Science and, by courtesy, of Psychology at Stanford University. Before joining Stanford, I was a Visiting Faculty Researcher at Google Research, working with Noah Snavely. I finished my PhD at MIT, advised by Bill Freeman and Josh Tenenbaum, and my undergraduate degrees at Tsinghua University, working with Zhuowen Tu.

Research

My group studies physical scene understanding---building machines that see, reason about, and interact with the physical world. Besides learning algorithms, what are the levels of abstraction needed by AI systems in their representations, and where do they come from? Our research aims to answer these fundamental questions, drawing inspiration from nature, i.e., the physical world itself, and from human cognition. Representative projects include Galileo, MarrNet, 4D Roses, the Neuro-Symbolic Concept Learner, and the Scene Language.

In addition to developing the representations themselves, we simultaneously explore their applications in various domains:

multi-modal perception from visual, acoustic, and tactile signals, as in ObjectFolder and RealImpact;
visual generation of the 4D, physical world, as in 3D-GAN, pi-GAN, Point-Voxel Diffusion, SDEdit, and WonderWorld;
visual reasoning via physical concept grounding, often in a neuro-symbolic way, as in NS-VQA, Shape Programs, CLEVRER, and LEFT;
robotics and embodied AI using the learned physical scene representations, as in RoboCook and BEHAVIOR.

Here is some information for prospective students and visitors.

Thank you for your interest in joining my group! Due to the large number of emails I receive, I cannot respond to every email individually. Please review the information below before contacting me. Thanks.

Current Stanford students and prospective visiting students: please fill out this form. For Stanford MS students and undergraduates, the minimum time commitment is 15 hours per week for six months. For visiting graduate students, the minimum length of a visit is six months.

Prospective postdocs: please email me directly with your CV.

Prospective graduate students: please apply through the system and list me as a potential advisor in your application. There is no need to contact me, unless you have a particular research question/idea that you would like to discuss further.

Group

Postdocs

Raven Huang (with Fei-Fei Li)
R. Kenny Jones (with Maneesh Agrawala)
Ruohan Zhang (with Fei-Fei Li and Silvio Savarese)

PhD Students

Ziyu Chen
Cristobal Eyzaguirre (with Juan Carlos Niebles)
Yue Gao (with Juan Carlos Niebles)
Chen Geng
Joy Hsu
Zizhang Li
Kyle Sargent (with Fei-Fei Li)
Jeff Tan
Stephen Tian
Koven Yu
Yanjie Ze (with Karen Liu)
Yunzhi Zhang

Affiliates

Alumni

Jiaman Li (PhD 2025, with Karen Liu), Applied Scientist, Amazon FAR
Fan-Yun Sun (PhD 2025, with Nick Haber), Co-Founder & CEO, Moonlake AI
Samuel Clarke (PhD 2025), Senior ML Engineer, Tesla
Kyle Hsu (PhD 2025, with Chelsea Finn), Senior ML Engineer, Tesla
Michelle Guo (PhD 2025, with Karen Liu), Research Scientist, Meta Superintelligence Labs
Weiyu Liu (postdoc 2025), Assistant Professor, University of Utah (starting 2026)
Mengdi Xu (postdoc 2025, with Fei-Fei Li), Assistant Professor, Tsinghua University
Stefan Stojanov (postdoc 2025, with Dan Yamins), Research Scientist, Google
Eric Chan (2024, with Gordon Wetzstein), Co-Founder & Chief Scientist, Rhoda AI
Elliott Wu (postdoc 2024), Assistant Professor, University of Cambridge
Manling Li (postdoc 2024), Assistant Professor, Northwestern University
Ryosuke Sawata (visiting PhD 2023), Research Scientist, Sony AI
Yunhao Ge (visiting PhD 2023, with Laurent Itti), Research Scientist, Nvidia
Yunzhu Li (postdoc 2023, with Fei-Fei Li), Assistant Professor, Columbia University
Ruohan Gao (postdoc 2023, with Fei-Fei Li and Silvio Savarese), Assistant Professor, University of Maryland, College Park
Sumith Kulal (PhD 2023, with Alex Aiken), Co-Founder, Black Forest Labs
Michael Lingelbach (2023, with Fei-Fei Li), Founder & CEO, Hedra
Huazhe Xu (postdoc 2022), Assistant Professor, Tsinghua University

Teaching

SYMSYS1/SYMSYS200/CS24/LINGUIST35/PHIL99/PSYCH35: Minds and Machines, Spr 2026, Wtr 2025, 2024, Fall 2022 (with Thomas Icard)
CS348I: Computer Graphics in the Era of AI, Wtr 2024, Fall 2021, 2020 (all with Karen Liu)
CS231N: Deep Learning for Computer Vision, Spr 2022 (with Fei-Fei Li and Ruohan Gao)
PSYCH225/CS322: Triangulating Intelligence: Melding Neuroscience, Psychology, and AI, Wtr 2022 (with Hyo Gweon and Dan Yamins)
CS221: Artificial Intelligence: Principles and Techniques, Wtr 2021 (with Tatsu Hashimoto)
CS131: Computer Vision: Foundations and Applications, Fall 2020 (with Juan Carlos Niebles)

Talks

Research Overview (Jan 2023)

Scene Understanding via Intrinsics (July 2024)

Scene Understanding via Intrinsics thumbnail

Reasoning and Concept Learning (Nov 2024)

Reasoning and Concept Learning thumbnail

Publications (show selected / show all by date / show all by topic)

Year: 2026 / 2025 / 2024 / 2023 / 2022 / 2021 / 2020 / 2019 / 2018 / 2017 / 2016 / 2015 / 2014 and before

Current Research Topics: Physical Scene Understanding / Multi-Modal Perception / Visual Generation / Visual Reasoning / Robotics and Embodied AI; Past Research Topic: Weakly-Supervised Learning

(* and † indicate equal contribution or alphabetical order)

Building Intelligent Agents with Neuro-Symbolic Concepts

Jiayuan Mao, Joshua B. Tenenbaum, Jiajun Wu

CACM 2026 / Paper

Choreographing a World of Dynamic Objects

Yanzhe Lyu*, Chen Geng*, Karthik Dharmarajan, Yunzhi Zhang, Hadi Alzayer, Shangzhe Wu, Jiajun Wu

CVPR 2026 / Paper / Project Page

Discovering Hybrid World Representations with Co-Evolving Foundation Models

Jiajun Wu, Yunzhi Zhang, Hong-Xing Yu, Joy Hsu, Jiayuan Mao

AAAI 2026 Emerging Trends in AI / Paper

A Review of Learning-Based Dynamics Models for Robotic Manipulation

Bo Ai, Stephen Tian, Haochen Shi, Yixuan Wang, Tobias Pfaff, Cheston Tan, Henrik I. Christensen, Hao Su, Jiajun Wu, Yunzhu Li

Science Robotics 2025 / Paper

TWIST: Teleoperated Whole-Body Imitation System

Yanjie Ze*, Zixuan Chen*, Joao Pedro Araujo, Zi-ang Cao, Xue Bin Peng, Jiajun Wu†, C. Karen Liu†

CoRL 2025 / Paper / Project Page

Birth and Death of a Rose

Chen Geng, Yunzhi Zhang, Shangzhe Wu, Jiajun Wu

CVPR 2025 / Paper / Project Page

Oral Presentation

The Scene Language: Representing Scenes with Programs, Words, and Embeddings

Yunzhi Zhang, Zizhang Li, Matt Zhou, Shangzhe Wu, Jiajun Wu

CVPR 2025 / Paper / Project Page

Highlight

WonderWorld: Interactive 3D Scene Generation from a Single Image

Hong-Xing Yu*, Haoyi Duan*, Charles Herrmann, William T. Freeman, Jiajun Wu

CVPR 2025 / Paper / Project Page

Highlight

Physical Scene Understanding

Jiajun Wu

AI Magazine 2024 / Paper

Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making

Manling Li*, Shiyu Zhao*, Qineng Wang*, Kangrui Wang*, Yu Zhou*, Sanjana Srivastava, Cem Gokmen, Tony Lee, Li Erran Li, Ruohan Zhang, Weiyu Liu, Percy Liang, Li Fei-Fei, Jiayuan Mao, Jiajun Wu

NeurIPS 2024 Datasets and Benchmarks / Paper / Project Page

Oral Presentation

Learning the 3D Fauna of the Web

Zizhang Li*, Dor Litvak*, Ruining Li, Yunzhi Zhang, Tomas Jakab, Christian Rupprecht, Shangzhe Wu†, Andrea Vedaldi†, Jiajun Wu†

CVPR 2024 / Paper / Project Page

What's Left? Concept Grounding with Logic-Enhanced Foundation Models

Joy Hsu*, Jiayuan Mao*, Joshua B. Tenenbaum, Jiajun Wu

NeurIPS 2023 / Paper / Project Page

RoboCook: Long-Horizon Elasto-Plastic Object Manipulation with Diverse Tools

Haochen Shi*, Huazhe Xu*, Samuel Clarke, Yunzhu Li, Jiajun Wu

CoRL 2023 / Paper / Project Page

Best Systems Paper Award

Fluid Simulation on Neural Flow Maps

Yitong Deng, Hong-Xing Yu, Diyang Zhang, Jiajun Wu, Bo Zhu

TOG (SIGGRAPH Asia) 2023 / Paper / Project Page

Best Paper Award

Seeing a Rose in Five Thousand Ways

Yunzhi Zhang, Shangzhe Wu, Noah Snavely, Jiajun Wu

CVPR 2023 / Paper / Project Page

Ego-Body Pose Estimation via Ego-Head Pose Estimation

Jiaman Li, C. Karen Liu†, Jiajun Wu†

CVPR 2023 / Paper / Project Page

Best Paper Award Candidate

RealImpact: A Dataset of Impact Sound Fields for Real Objects

Samuel Clarke, Ruohan Gao, Mason Wang, Mark Rau, Julia Xu, Jui-Hsien Wang, Doug L. James, Jiajun Wu

CVPR 2023 / Paper / Project Page

Highlight

BEHAVIOR-1K: A Benchmark for Embodied AI with 1,000 Everyday Activities and Realistic Simulation

Chengshu Li*, Ruohan Zhang*, Josiah Wong*, Cem Gokmen*, Sanjana Srivastava*, Roberto Martin-Martin*, Chen Wang*, Gabrael Levine*, Michael Lingelbach, Jiankai Sun, Mona Anvari, Minjune Hwang, Manasi Sharma, Arman Aydin, Dhruva Bansal, Samuel Hunter, Kyu-Young Kim, Alan Lou, Caleb R. Matthews, Ivan Villa-Renteria, Jerry Huayang Tang, Claire Tang, Fei Xia, Silvio Savarese, Hyowon Gweon, C. Karen Liu, Jiajun Wu, Li Fei-Fei

CoRL 2022 / Paper / Project Page

Best Paper Award Nominee

ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer

Ruohan Gao*, Zilin Si*, Yen-Yu Chang*, Samuel Clarke, Jeannette Bohg, Li Fei-Fei, Wenzhen Yuan, Jiajun Wu

CVPR 2022 / Paper / Project Page

3D Shape Generation and Completion Through Point-Voxel Diffusion

Linqi Zhou, Yilun Du, Jiajun Wu

ICCV 2021 / Paper / Project Page

Oral Presentation

Learning to See the Physical World

PhD Dissertation 2019

ACM Doctoral Dissertation Award Honorable Mention
Joint AAAI/ACM SIGAI Doctoral Dissertation Award
MIT George M. Sprowls PhD Thesis Award in Artificial Intelligence and Decision-Making

The Neuro-Symbolic Concept Learner:
Interpreting Scenes, Words, and Sentences from Natural Supervision

Jiayuan Mao, Chuang Gan, Pushmeet Kohli, Joshua B. Tenenbaum, Jiajun Wu

ICLR 2019 / Paper / Project Page / MIT News / MIT Technology Review

Oral Presentation

Learning to Infer and Execute 3D Shape Programs

Yonglong Tian, Andrew Luo, Xingyuan Sun, Kevin Ellis, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu

ICLR 2019 / Paper / Project Page

Visual Dynamics: Stochastic Future Generation via Layered Cross Convolutional Networks

Tianfan Xue*, Jiajun Wu*, Katherine L. Bouman, William T. Freeman

TPAMI 2019 / Paper / Project Page

NeurIPS 2016 / Paper

Oral Presentation

Learning to Reconstruct Shapes from Unseen Classes

Xiuming Zhang*, Zhoutong Zhang*, Chengkai Zhang, Joshua B. Tenenbaum, William T. Freeman, Jiajun Wu

NeurIPS 2018 / Paper / Project Page

Oral Presentation

Augmenting Physical Simulators with Stochastic Neural Networks:
Case Study of Planar Pushing and Bouncing

Anurag Ajay, Jiajun Wu, Nima Fazeli, Maria Bauza, Leslie P. Kaelbling, Joshua B. Tenenbaum, Alberto Rodriguez

IROS 2018 / Paper / Project Page

Best Paper Award on Cognitive Robotics

3D Interpreter Networks for Viewer-Centered Wireframe Modeling

Jiajun Wu*, Tianfan Xue*, Joseph J. Lim, Yuandong Tian, Joshua B. Tenenbaum, Antonio Torralba, William T. Freeman

IJCV 2018 / Paper / Project Page

ECCV 2016 / Paper

Oral Presentation

Learning to See Physics via Visual De-animation

Jiajun Wu, Erika Lu, Pushmeet Kohli, William T. Freeman, Joshua B. Tenenbaum

NeurIPS 2017 / Paper / Project Page / MIT News

Spotlight Presentation

MarrNet: 3D Shape Reconstruction via 2.5D Sketches

Jiajun Wu*, Yifan Wang*, Tianfan Xue, Xingyuan Sun, William T. Freeman, Joshua B. Tenenbaum

NeurIPS 2017 / Paper / Project Page

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Jiajun Wu*, Chengkai Zhang*, Tianfan Xue, William T. Freeman, Joshua B. Tenenbaum

NeurIPS 2016 / Paper / Project Page / MIT Technology Review

Galileo: Perceiving Physical Object Properties by Integrating a Physics Engine with Deep Learning

Jiajun Wu*, Ilker Yildirim*, Joseph J. Lim, William T. Freeman, Joshua B. Tenenbaum

NeurIPS 2015 / Paper / Project Page / Demo / MIT News / VICE / WIRED / Engadget / Popular Science / The Boston Globe / PCWorld / TechRadar

2026

Building Intelligent Agents with Neuro-Symbolic Concepts

Jiayuan Mao, Joshua B. Tenenbaum, Jiajun Wu

CACM 2026 / Paper

Choreographing a World of Dynamic Objects

Yanzhe Lyu*, Chen Geng*, Karthik Dharmarajan, Yunzhi Zhang, Hadi Alzayer, Shangzhe Wu, Jiajun Wu

CVPR 2026 / Paper / Project Page

NEUROK: Generative 4D Neural Object Kinematics

Chen Geng*, Guangzhao He*, Yue Gao*, Yunzhi Zhang, Shangzhe Wu, Jiajun Wu

CVPR 2026 / Paper

WonderZoom: Multi-Scale 3D World Generation

Jin Cao*, Hong-Xing Yu*, Jiajun Wu

CVPR 2026 / Paper / Project Page

AnyLift: Scaling Motion Reconstruction from Internet Videos via 2D Diffusion

Hongjie Li*, Heng Yu*, Jiaman Li, Hong-Xing Yu, Ehsan Adeli, C. Karen Liu, Jiajun Wu

CVPR 2026 / Paper

PerpetualWonder: Long-Horizon Action-Conditioned 4D Scene Generation

Jiahao Zhan*, Zizhang Li*, Hong-Xing Yu, Jiajun Wu

CVPR 2026 / Paper / Project Page

Highlight

Coupled Diffusion Sampling for Training-Free Multi-View Image Editing

Hadi Alzayer, Yunzhi Zhang, Chen Geng, Jia-Bin Huang, Jiajun Wu

CVPR 2026 / Paper / Project Page

VLIC: Vision-Language Models As Perceptual Judges for Human-Aligned Image Compression

Kyle Sargent, Ruiqi Gao, Philipp Henzler, Charles Herrmann, Aleksander Holynski, Li Fei-Fei, Jiajun Wu, Jason Y. Zhang

CVPR 2026 / Paper / Project Page

Self-Consistency for LLM-Based Motion Trajectory Generation and Verification

Jiaju Ma, R. Kenny Jones, Jiajun Wu, Maneesh Agrawala

CVPR 2026 / Paper / Project Page

ART: Articulated Reconstruction Transformer

Zizhang Li, Cheng Zhang, Zhengqin Li, Henry Howard-Jenkins, Zhaoyang Lv, Chen Geng, Jiajun Wu, Richard Newcombe, Jakob Engel, Zhao Dong

CVPR 2026 / Paper / Project Page

Talk2Move: Reinforcement Learning for Text-Instructed Object-Level Geometric Transformation in Scenes

Jing Tan, Zhaoyang Zhang, Yantao Shen, Jiarui Cai, Shuo Yang, Jiajun Wu, Wei Xia, Zhuowen Tu, Stefano Soatto

CVPR 2026 / Paper / Project Page

Highlight

Product of Experts for Visual Generation

Yunzhi Zhang, Carson Murtuza-Lanier, Zizhang Li, Yilun Du, Jiajun Wu

ICLR 2026 / Paper / Project Page

Neuro-Symbolic Decoding of Neural Activity

Yanchen Wang*, Joy Hsu*, Ehsan Adeli†, Jiajun Wu†

ICLR 2026 / Paper / Project Page

Spatial Mental Modeling from Limited Views

Qineng Wang*, Baiqiao Yin*, Pingyue Zhang, Jianshu Zhang, Kangrui Wang, Zihan Wang, Jieyu Zhang, Keshigeyan Chandrasegaran, Han Liu, Ranjay Krishna, Saining Xie, Jiajun Wu†, Li Fei-Fei†, Manling Li†

ICLR 2026 / Paper / Project Page

ENACT: Evaluating Embodied Cognition with World Modeling of Egocentric Interaction

Qineng Wang*, Wenlong Huang*, Yu Zhou, Hang Yin, Tianwei Bao, Jianwen Lyu, Weiyu Liu, Ruohan Zhang†, Jiajun Wu†, Li Fei-Fei†, Manling Li†

ICLR 2026 / Paper / Project Page

Theory of Space: Can Foundation Models Construct Spatial Beliefs Through Active Exploration?

Pingyue Zhang*, Zihan Huang*, Yue Wang*, Jieyu Zhang*, Letian Xue, Zihan Wang, Qineng Wang, Keshigeyan Chandrasegaran, Ruohan Zhang, Yejin Choi, Ranjay Krishna, Jiajun Wu, Li Fei-Fei, Manling Li

ICLR 2026 / Paper / Project Page

GenFusion: Feed-forward Human Performance Capture via Progressive Canonical Space Updates

Youngjoong Kwon, Yao He*, Heejung Choi*, Chen Geng, Zhengmao Liu, Jiajun Wu, Ehsan Adeli

ICLR 2026 / Paper / Project Page

ROSETTA: Constructing Code-Based Reward from Unconstrained Language Preference

Sanjana Srivastava*, Kangrui Wang*, Yung-Chieh Chan*, Tianyuan Dai, Manling Li, Ruohan Zhang, Mengdi Xu, Jiajun Wu, Li Fei-Fei

ICLR 2026 / Paper / Project Page

MoMaGen: Generating Demonstrations Under Soft and Hard Constraints for Multi-Step Bimanual Mobile Manipulation

Chengshu Li*, Mengdi Xu*, Arpit Bahety*, Hang Yin*, Yunfan Jiang, Huang Huang, Josiah Wong, Sujay Garlanka, Cem Gokmen, Ruohan Zhang, Weiyu Liu, Jiajun Wu, Roberto Martin-Martin, Li Fei-Fei

ICLR 2026 / Paper / Project Page

Learning Composable Skills by Discovering Spatial and Temporal Structure with Foundation Models

Neil Nie, Wenlong Huang, Jiayuan Mao, Li Fei-Fei, Weiyu Liu†, Jiajun Wu†

ICRA 2026 / Paper / Project Page

TWIST2: Scalable, Portable, and Holistic Humanoid Data Collection System

Yanjie Ze, Siheng Zhao, Weizhuo Wang, Angjoo Kanazawa†, Rocky Duan†, Pieter Abbeel†, Guanya Shi†, Jiajun Wu†, C. Karen Liu†

ICRA 2026 / Paper / Project Page

IMPASTO: Integrating Model-Based Planning with Learned Dynamics Models for Robotic Oil Painting Reproduction

Yingke Wang, Hao Li, Yifeng Zhu, Hong-Xing Yu, Ken Goldberg, Li Fei-Fei, Jiajun Wu, Yunzhu Li, Ruohan Zhang

ICRA 2026 / Paper / Project Page

Dream2Flow: Bridging Video Generation and Open-World Manipulation with 3D Object Flow

Karthik Dharmarajan, Wenlong Huang, Jiajun Wu, Li Fei-Fei†, Ruohan Zhang†

ICRA 2026 / Paper / Project Page

Retargeting Matters: General Motion Retargeting for Humanoid Motion Tracking

Joao Pedro Araujo*, Yanjie Ze*, Pei Xu*, Jiajun Wu†, C. Karen Liu†

ICRA 2026 / Paper / Project Page

Phys2Real: Fusing VLM Priors with Interactive Online Adaptation for Uncertainty-Aware Sim-to-Real Manipulation

Maggie Wang, Stephen Tian, Aiden Swann, Ola Shorinwa, Jiajun Wu, Mac Schwager

ICRA 2026 / Paper / Project Page

Deep Sketch-Based 3D Modeling: A Survey

Alberto Tono, Jiajun Wu, Gordon Wetzstein, Iro Armeni, Martin Fischer, Hariharan Subramonyam, James Landay

CGF 2026 / Paper / Project Page

ZeroHSI: Zero-Shot 4D Human-Scene Interaction by Video Generation

Hongjie Li*, Hong-Xing Yu*, Jiaman Li, Jiajun Wu

3DV 2026 / Paper / Project Page

3D-Generalist: Vision-Language-Action Models for Crafting 3D Worlds

Fan-Yun Sun, Shengguang Wu, Christian Jacobsen, Thomas Yim, Haoming Zou, Alex Zook, Shangru Li, Yu-Hsin Chou, Ethem Can, Xunlei Wu, Clemens Eppner, Valts Blukis, Jonathan Tremblay, Jiajun Wu, Stan Birchfield†, Nick Haber†

3DV 2026 / Paper / Project Page

Discovering Hybrid World Representations with Co-Evolving Foundation Models

Jiajun Wu, Yunzhi Zhang, Hong-Xing Yu, Joy Hsu, Jiayuan Mao

AAAI 2026 Emerging Trends in AI / Paper

10 Open Challenges Steering the Future of Vision-Language-Action Models

Soujanya Poria, Navonil Majumder, Chia-Yu Hung, Amir Ali Bagherzadeh, Chuan Li, Kenneth Kwok, Ziwei Wang, Cheston Tan, Jiajun Wu, David Hsu

AAAI 2026 Senior Member Presentation / Paper

A Tool Bottleneck Framework for Clinically-Informed and Interpretable Medical Image Understanding

Christina Liu*, Alan Wang*, Joy Hsu, Jiajun Wu, Ehsan Adeli

MIDL 2026 / Paper / Project Page

2025

A Review of Learning-Based Dynamics Models for Robotic Manipulation

Bo Ai, Stephen Tian, Haochen Shi, Yixuan Wang, Tobias Pfaff, Cheston Tan, Henrik I. Christensen, Hao Su, Jiajun Wu, Yunzhu Li

Science Robotics 2025 / Paper

Self-Supervised Learning of Motion Concepts by Optimizing Counterfactuals

Stefan Stojanov*, David Wendt*, Seungwoo Kim*, Rahul Venkatesh*, Kevin Feigelis, Jiajun Wu, Daniel L. K. Yamins

NeurIPS 2025 / Paper / Project Page

Spotlight Presentation

From Programs to Poses: Factored Real-World Scene Generation via Learned Program Libraries

Joy Hsu, Emily Jin, Jiajun Wu, Niloy J. Mitra

NeurIPS 2025 / Paper / Project Page

Seeing the Wind from a Falling Leaf

Zhiyuan Gao*, Jiageng Mao*, Hong-Xing Yu, Haozhe Lou, Emily Yue-ting Jia, Jernej Barbic, Jiajun Wu, Yue Wang

NeurIPS 2025 / Paper / Project Page

Taming Generative Video Models for Zero-Shot Optical Flow Extraction

Seungwoo Kim*, Khai Loong Aw*, Klemen Kotar*, Cristobal Eyzaguirre, Wanhee Lee, Yunong Liu, Jared Watrous, Stefan Stojanov, Juan Carlos Niebles, Jiajun Wu, Daniel L. K. Yamins

NeurIPS 2025 / Paper / Project Page

Model-Based Policy Adaptation for Closed-Loop End-to-End Autonomous Driving

Haohong Lin, Yunzhi Zhang, Wenhao Ding, Jiajun Wu, Ding Zhao

NeurIPS 2025 / Paper

VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents

Kangrui Wang*, Pingyue Zhang*, Zihan Wang*, Yaning Gao*, Linjie Li*, Qineng Wang, Hanyang Chen, Chi Wan, Yiping Lu, Zhengyuan Yang, Lijuan Wang, Ranjay Krishna, Jiajun Wu, Li Fei-Fei, Yejin Choi, Manling Li

NeurIPS 2025 / Paper / Project Page

Web-Scale Collection of Video Data for 4D Animal Reconstruction

Brian Nlong Zhao, Jiajun Wu†, Shangzhe Wu†

NeurIPS 2025 Datasets and Benchmarks / Paper

DexSkin: High-Coverage Conformable Robotic Skin for Learning Contact-Rich Manipulation

Suzannah Wistreich*, Baiyu Shi*, Stephen Tian*, Samuel Clarke, Michael Nath, Chengyi Xu, Zhenan Bao, Jiajun Wu

CoRL 2025 / Paper / Project Page

Oral Presentation

TWIST: Teleoperated Whole-Body Imitation System

Yanjie Ze*, Zixuan Chen*, Joao Pedro Araujo, Zi-ang Cao, Xue Bin Peng, Jiajun Wu†, C. Karen Liu†

CoRL 2025 / Paper / Project Page

BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activities

Yunfan Jiang, Ruohan Zhang, Josiah Wong, Chen Wang, Yanjie Ze, Hang Yin, Cem Gokmen, Shuran Song, Jiajun Wu, Li Fei-Fei

CoRL 2025 / Paper / Project Page

WonderPlay: Dynamic 3D Scene Generation from a Single Image and Actions

Zizhang Li*, Hong-Xing Yu*, Wei Liu, Yin Yang, Charles Herrmann, Gordon Wetzstein, Jiajun Wu

ICCV 2025 / Paper / Project Page

Highlight

WorldScore: Unified Evaluation Benchmark for World Generation

Haoyi Duan*, Hong-Xing Yu*, Sirui Chen, Li Fei-Fei, Jiajun Wu

ICCV 2025 / Paper / Project Page

X-Capture: An Open-Source Portable Device for Multi-Sensory Learning

Samuel Clarke, Suzannah Wistreich, Yanjie Ze, Jiajun Wu

ICCV 2025 / Paper / Project Page

Flow to the Mode: Mode-Seeking Diffusion Autoencoders for State-of-the-Art Image Tokenization

Kyle Sargent, Kyle Hsu, Li Fei-Fei, Justin Johnson, Jiajun Wu

ICCV 2025 / Paper / Project Page

Weakly-Supervised Learning of Dense Functional Correspondences

Stefan Stojanov*, Linan Zhao*, Yunzhi Zhang, Daniel L. K. Yamins, Jiajun Wu

ICCV 2025 / Paper / Project Page

Generalizable Humanoid Manipulation with 3D Diffusion Policies

Yanjie Ze, Zixuan Chen, Wenhao Wang, Tianyi Chen, Xialin He, Ying Yuan, Xue Bin Peng, Jiajun Wu

IROS 2025 / Paper / Project Page

Learning Smooth Humanoid Locomotion Through Lipschitz-Constrained Policies

Zixuan Chen*, Xialin He*, Yen-Jen Wang*, Qiayuan Liao, Yanjie Ze, Zhongyu Li, S. Shankar Sastry, Jiajun Wu, Koushil Sreenath, Saurabh Gupta, Xue Bin Peng

IROS 2025 / Paper / Project Page

Birth and Death of a Rose

Chen Geng, Yunzhi Zhang, Shangzhe Wu, Jiajun Wu

CVPR 2025 / Paper / Project Page

Oral Presentation

The Scene Language: Representing Scenes with Programs, Words, and Embeddings

Yunzhi Zhang, Zizhang Li, Matt Zhou, Shangzhe Wu, Jiajun Wu

CVPR 2025 / Paper / Project Page

Highlight

WonderWorld: Interactive 3D Scene Generation from a Single Image

Hong-Xing Yu*, Haoyi Duan*, Charles Herrmann, William T. Freeman, Jiajun Wu

CVPR 2025 / Paper / Project Page

Highlight

FluidNexus: 3D Fluid Reconstruction and Prediction from a Single Video

Yue Gao*, Hong-Xing Yu*, Bo Zhu, Jiajun Wu

CVPR 2025 / Paper / Project Page

Oral Presentation

Lifting Motion to the 3D World via 2D Diffusion

Jiaman Li, C. Karen Liu†, Jiajun Wu†

CVPR 2025 / Paper / Project Page

Highlight

Category-Agnostic Neural Object Rigging

Guangzhao He*, Chen Geng*, Shangzhe Wu, Jiajun Wu

CVPR 2025 / Paper / Project Page

LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Models

Fan-Yun Sun*, Weiyu Liu*, Siyi Gu, Dylan Lim, Goutam Bhat, Federico Tombari, Manling Li, Nick Haber, Jiajun Wu

CVPR 2025 / Paper / Project Page

Digital Twin Catalog: A Large-Scale Photorealistic 3D Object Digital Twin Dataset

Zhao Dong, Ka Chen*, Zhaoyang Lv*, Hong-Xing Yu*, Yunzhi Zhang*, Cheng Zhang*, Yufeng Zhu*, Stephen Tian, Zhengqin Li, Geordie Moffatt, Sean Christofferson, James Fort, Xiaqing Pan, Mingfei Yan, Jiajun Wu, Yufeng Ren, Richard Newcombe

CVPR 2025 / Paper / Project Page

Highlight

Diffusion Self-Distillation for Zero-Shot Customized Image Generation

Shengqu Cai, Eric Ryan Chan, Yunzhi Zhang, Leonidas J. Guibas, Jiajun Wu, Gordon Wetzstein

CVPR 2025 / Paper / Project Page

PGC: Physics-Based Gaussian Cloth from a Single Pose

Michelle Guo, Matt Jen-Yuan Chiang, Igor Santesteban, Nikolaos Sarafianos, Hsiao-yu Chen, Oshri Halimi, Aljaz Bozic, Shunsuke Saito, Jiajun Wu, C. Karen Liu, Tuur Stuyck, Egor Larionov

CVPR 2025 / Paper / Project Page

Highlight

Re-thinking Temporal Search for Long-Form Video Understanding

Jinhui Ye*, Zihan Wang*, Haosen Sun, Keshigeyan Chandrasegaran, Zane Durante, Cristobal Eyzaguirre, Yonatan Bisk, Juan Carlos Niebles, Ehsan Adeli, Li Fei-Fei, Jiajun Wu, Manling Li

CVPR 2025 / Paper / Project Page

Anymate: A Dataset and Baselines for Learning 3D Object Rigging

Yufan Deng*, Yuhao Zhang*, Chen Geng, Shangzhe Wu†, Jiajun Wu†

SIGGRAPH 2025 / Paper / Project Page

Understanding Complexity in VideoQA via Visual Program Generation

Cristobal Eyzaguirre, Igor Vasiljevic, Achal Dave, Jiajun Wu, Rares Ambrus, Thomas Kollar, Juan Carlos Niebles, Pavel Tokmakov

ICML 2025 / Paper / Project Page

Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas

Shiqi Chen, Tongyao Zhu, Ruochen Zhou, Jinghan Zhang, Siyang Gao, Juan Carlos Niebles, Mor Geva, Junxian He, Jiajun Wu, Manling Li

ICML 2025 / Paper / Project Page

What Makes a Maze Look Like a Maze?

Joy Hsu, Jiayuan Mao, Joshua B. Tenenbaum, Noah D. Goodman, Jiajun Wu

ICLR 2025 / Paper / Project Page

Predicate Hierarchies Improve Few-Shot State Classification

Emily Jin*, Joy Hsu*, Jiajun Wu

ICLR 2025 / Paper / Project Page

Range, not Independence, Drives Modularity in Biologically Inspired Representations

Will Dorrell*, Kyle Hsu*, Luke Hollingsworth†, Jin Hwa Lee†, Jiajun Wu, Chelsea Finn, Peter E. Latham, Tim Behrens, James C. R. Whittington

ICLR 2025 / Paper / Project Page

UAD: Unsupervised Affordance Distillation for Generalization in Robotic Manipulation

Yihe Tang, Wenlong Huang, Yingke Wang, Chengshu Li, Roy Yuan, Ruohan Zhang, Jiajun Wu, Li Fei-Fei

ICRA 2025 / Paper / Project Page

Best Paper Award Finalist

Visually Descriptive Language Model for Vector Graphics Reasoning

Zhenhailong Wang, Joy Hsu, Xingyao Wang, Kuan-Hao Huang, Manling Li, Jiajun Wu, Heng Ji

TMLR 2025 / Paper / Project Page

Unsupervised Discovery of Object-Centric Neural Fields

Rundong Luo*, Hong-Xing Yu*, Jiajun Wu

TMLR 2025 / Paper / Project Page

CRAFT: Designing Creative and Functional 3D Objects

Michelle Guo*, Mia Tang*, Hannah Cha, Ruohan Zhang, C. Karen Liu, Jiajun Wu

WACV 2025 / Paper / Project Page

2024

Physical Scene Understanding

Jiajun Wu

AI Magazine 2024 / Paper

FactorSim: Generative Simulation via Factorized Representation

Fan-Yun Sun, S. I. Harini, Angela Yi, Yihan Zhou, Alex Zook, Jonathan Tremblay, Logan Cross, Jiajun Wu, Nick Haber

NeurIPS 2024 / Paper / Project Page

Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making

Manling Li*, Shiyu Zhao*, Qineng Wang*, Kangrui Wang*, Yu Zhou*, Sanjana Srivastava, Cem Gokmen, Tony Lee, Li Erran Li, Ruohan Zhang, Weiyu Liu, Percy Liang, Li Fei-Fei, Jiayuan Mao, Jiajun Wu

NeurIPS 2024 Datasets and Benchmarks / Paper / Project Page

Oral Presentation

IKEA Manuals at Work: 4D Grounding of Assembly Instructions on Internet Videos

Yunong Liu, Cristobal Eyzaguirre, Manling Li, Shubh Khanna, Juan Carlos Niebles, Vineeth Ravi, Saumitra Mishra, Weiyu Liu†, Jiajun Wu†

NeurIPS 2024 Datasets and Benchmarks / Paper / Project Page

HourVideo: 1-Hour Video-Language Understanding

Keshigeyan Chandrasegaran, Agrim Gupta, Lea M. Hadzic, Taran Kota, Jimming He, Cristobal Eyzaguirre, Zane Durante, Manling Li, Jiajun Wu, Li Fei-Fei

NeurIPS 2024 Datasets and Benchmarks / Paper / Project Page

MARPLE: A Benchmark for Long-Horizon Inference

Emily Jin*, Zhuoyi Huang*, Jan-Philipp Fränken, Weiyu Liu, Hannah Cha, Erik Brockbank, Sarah A. Wu, Ruohan Zhang, Jiajun Wu, Tobias Gerstenberg

NeurIPS 2024 Datasets and Benchmarks / Paper / Project Page

Streaming Detection of Queried Event Start

Cristobal Eyzaguirre, Eric Tang, Shyamal Buch, Adrien Gaidon, Jiajun Wu, Juan Carlos Niebles

NeurIPS 2024 Datasets and Benchmarks / Paper / Project Page

Learning Compositional Behaviors from Demonstration and Language

Weiyu Liu*, Neil Nie*, Ruohan Zhang, Jiayuan Mao†, Jiajun Wu†

CoRL 2024 / Paper / Project Page

View-Invariant Policy Learning via Zero-Shot Novel View Synthesis

Stephen Tian, Blake Wulfe, Kyle Sargent, Katherine Liu, Sergey Zakharov, Vitor Guizilini, Jiajun Wu

CoRL 2024 / Paper / Project Page

Automated Creation of Digital Cousins for Robust Policy Learning

Tianyuan Dai*, Josiah Wong*, Yunfan Jiang, Chen Wang, Cem Gokmen, Ruohan Zhang, Jiajun Wu, Li Fei-Fei

CoRL 2024 / Paper / Project Page

D3Fields: Dynamic 3D Descriptor Fields for Zero-Shot Generalizable Rearrangement

Yixuan Wang*, Mingtong Zhang*, Zhuoran Li*, Tarik Kelestemur, Katherine Driggs-Campbell, Jiajun Wu, Li Fei-Fei, Yunzhu Li

CoRL 2024 / Paper / Project Page

Oral Presentation

TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction

Yunfan Jiang, Chen Wang, Ruohan Zhang, Jiajun Wu, Li Fei-Fei

CoRL 2024 / Paper / Project Page

Evaluating Real-World Robot Manipulation Policies in Simulation

Xuanlin Li*, Kyle Hsu*, Jiayuan Gu*, Karl Pertsch, Oier Mees, Homer Rich Walke, Chuyuan Fu, Ishikaa Lunawat, Isabel Sieh, Sean Kirmani, Sergey Levine, Jiajun Wu, Chelsea Finn, Hao Su†, Quan Vuong†, Ted Xiao†

CoRL 2024 / Paper / Project Page

Foundation Models in Robotics: Applications, Challenges, and the Future

Roya Fizoori, Johnathan Tucker, Stephen Tian, Anirudha Majumdar, Jiankai Sun, Weiyu Liu, Yuke Zhu, Shuran Song, Ashish Kapoor, Karol Hausman, Brian Ichter, Danny Driess, Jiajun Wu, Cewu Lu, Mac Schwager

IJRR 2024 / Paper / Project Page

An Eulerian Vortex Method on Flow Maps

Sinan Wang, Yitong Deng, Molin Deng, Hong-Xing Yu, Junwei Zhou, Duowen Chen, Taku Komura, Jiajun Wu, Bo Zhu

TOG (SIGGRAPH Asia) 2024 / Paper / Project Page

PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generation

Tianyuan Zhang, Hong-Xing Yu, Rundi Wu, Brandon Y. Feng, Changxi Zheng, Noah Snavely, Jiajun Wu, William T. Freeman

ECCV 2024 / Paper / Project Page

Oral Presentation

Ponymation: Learning Articulated 3D Animal Motions from Unlabeled Online Videos

Keqiang Sun*, Dor Litvak*, Yunzhi Zhang, Hongsheng Li, Jiajun Wu†, Shangzhe Wu†

ECCV 2024 / Paper / Project Page

3D Congealing: 3D-Aware Image Alignment in the Wild

Yunzhi Zhang, Zizhang Li, Amit Raj, Andreas Engelhardt, Yuanzhen Li, Tingbo Hou, Jiajun Wu, Varun Jampani

ECCV 2024 / Paper / Project Page

Reconstruction and Simulation of Elastic Objects with Spring-Mass 3D Gaussians

Licheng Zhong, Hong-Xing Yu, Jiajun Wu, Yunzhu Li

ECCV 2024 / Paper / Project Page

Controllable Human-Object Interaction Synthesis

Jiaman Li, Alexander Clegg, Roozbeh Mottaghi, Jiajun Wu, Xaiver Puig†, C. Karen Liu†

ECCV 2024 / Paper / Project Page

Oral Presentation

Learning the 3D Fauna of the Web

Zizhang Li*, Dor Litvak*, Ruining Li, Yunzhi Zhang, Tomas Jakab, Christian Rupprecht, Shangzhe Wu†, Andrea Vedaldi†, Jiajun Wu†

CVPR 2024 / Paper / Project Page

WonderJourney: Going from Anywhere to Everywhere

Hong-Xing Yu, Haoyi Duan, Junhwa Hur, Kyle Sargent, Michael Rubinstein, William T. Freeman, Forrester Cole, Deqing Sun, Noah Snavely, Jiajun Wu, Charles Herrmann

CVPR 2024 / Paper / Project Page

ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Real Image

Kyle Sargent, Zizhang Li, Tanmay Shah, Charles Herrmann, Hong-Xing Yu, Yunzhi Zhang, Eric Ryan Chan, Dmitry Lagun, Li Fei-Fei, Deqing Sun, Jiajun Wu

CVPR 2024 / Paper / Project Page

Hearing Anything Anywhere

Mason Wang*, Ryosuke Sawata*, Samuel Clarke, Ruohan Gao, Shangzhe Wu, Jiajun Wu

CVPR 2024 / Paper / Project Page

Naturally Supervised 3D Visual Grounding with Language-Regularized Concept Learners

Chun Feng*, Joy Hsu*, Weiyu Liu, Jiajun Wu

CVPR 2024 / Paper / Project Page

BEHAVIOR Vision Suite: Customizable Dataset Generation via Simulation

Yunhao Ge*, Yihe Tang*, Jiashu Xu*, Cem Gokmen*, Chengshu Li, Wensi Ai, Benjamin Jose Martinez, Arman Aydin, Mona Anvari, Ayush K Chakravarthy, Hong-Xing Yu, Josiah Wong, Sanjana Srivastava, Sharon Lee, Shengxin Zha, Laurent Itti, Yunzhu Li, Roberto Martin-Martin, Miao Liu, Pengchuan Zhang, Ruohan Zhang, Li Fei-Fei, Jiajun Wu

CVPR 2024 / Paper / Project Page

Highlight

ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understanding

Le Xue, Ning Yu, Shu Zhang, Junnan Li, Roberto Martin-Martin, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, Silvio Savarese

CVPR 2024 / Paper / Project Page

Holodeck: Language Guided Generation of 3D Embodied AI Environments

Yue Yang*, Fan-Yun Sun*, Luca Weihs*, Eli VanderBilt, Alvaro Herrasti, Winson Han, Jiajun Wu, Nick Haber, Ranjay Krishna, Lingjie Liu, Chris Callison-Burch, Mark Yatskar, Aniruddha Kembhavi, Christopher Clark

CVPR 2024 / Paper / Project Page

DiffSound: Differentiable Modal Sound Rendering and Inverse Rendering for Diverse Inference Tasks

Xutong Jin*, Chenxi Xu*, Ruohan Gao, Jiajun Wu, Guoping Wang, Sheng Li

SIGGRAPH 2024 / Paper / Project Page

Tripod: Three Complementary Inductive Biases for Disentangled Representation Learning

Kyle Hsu*, Jubayer Ibn Hamid*, Kaylee Burns, Chelsea Finn, Jiajun Wu

ICML 2024 / Paper / Project Page

RoboPack: Learning Tactile-Informed Dynamics Models for Dense Packing

Bo Ai*, Stephen Tian*, Haochen Shi, Yixuan Wang, Cheston Tan, Yunzhu Li, Jiajun Wu

RSS 2024 / Paper / Project Page

DROID: A Large-Scale In-the-Wild Robot Manipulation Dataset

DROID Dataset Team

RSS 2024 / Paper / Project Page

Whodunnit? Inferring What Happened from Multimodal Evidence

Sarah A. Wu*, Erik Brockbank*, Hannah Cha, Jan-Philipp Fränken, Emily Jin, Zhuoyi Huang, Weiyu Liu, Ruohan Zhang, Jiajun Wu, Tobias Gerstenberg

CogSci 2024 / Paper

D5RL: Diverse Datasets for Data-Driven Deep Reinforcement Learning

Rafael Rafailov*, Kyle Hatch*, Anikait Singh, Laura Smith, Aviral Kumar, Ilya Kostrikov, Philippe Hansen-Estruch, Victor Kolev, Philip J. Ball, Jiajun Wu, Chelsea Finn, Sergey Levine

RLC 2024 / Paper

Language-Informed Visual Concept Learning

Sharon Lee*, Yunzhi Zhang*, Shangzhe Wu, Jiajun Wu

ICLR 2024 / Paper / Project Page

Neural Polynomial Gabor Fields for Macro Motion Analysis

Chen Geng*, Hong-Xing Yu*, Sida Peng, Xiaowei Zhou, Jiajun Wu

ICLR 2024 / Paper / Project Page

Learning Planning Abstractions from Language

Weiyu Liu*, Geng Chen*, Joy Hsu, Jiayuan Mao†, Jiajun Wu†

ICLR 2024 / Paper / Project Page

Patched Denoising Diffusion Models For High-Resolution Image Synthesis

Zheng Ding*, Mengqi Zhang*, Jiajun Wu, Zhuowen Tu

ICLR 2024 / Paper / Project Page

RoboCraft: Learning to See, Simulate, and Shape Elasto-Plastic Objects in 3D with Graph Networks

Haochen Shi*, Huazhe Xu*, Zhiao Huang, Yunzhu Li, Jiajun Wu

IJRR 2024 / Paper / Project Page

Learning to Design 3D Printable Adaptations on Everyday Objects for Robot Manipulation

Michelle Guo, Ziang Liu, Stephen Tian, Zhaoming Xie, Jiajun Wu, C. Karen Liu

ICRA 2024 / Paper / Project Page

Physically Grounded Vision-Language Models for Robotic Manipulation

Jensen Gao, Bidipta Sarkar, Fei Xia, Ted Xiao, Jiajun Wu, Brian Ichter, Anirudha Majumdar, Dorsa Sadigh

ICRA 2024 / Paper / Project Page

Open X-Embodiment: Robotic Learning Datasets and RT-X Models

Open X-Embodiment Collaboration

ICRA 2024 / Paper / Project Page

Best Paper Award

Efficient Imitation Learning with Conservative World Models

Victor Kolev*, Rafael Rafailov*, Kyle Hatch, Jiajun Wu, Chelsea Finn

L4DC 2024 / Paper

Oral Presentation

SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensing

Zhecheng Wang, Rajanie Prabha*, Tianyuan Huang*, Jiajun Wu, Ram Rajagopal

AAAI 2024 / Paper

CityPulse: Fine-Grained Assessment of Urban Change with Street View Time Series

Tianyuan Huang, Zejia Wu, Jiajun Wu, Jackelyn Hwang, Ram Rajagopal

AAAI 2024 AI for Social Impact / Paper

Unsupervised 3D Scene Representation Learning via Movable Object Inference

Honglin Chen*, Wanhee Lee*, Hong-Xing Yu, Rahul Venkatesh, Joshua B. Tenenbaum, Daniel M. Bear, Jiajun Wu, Daniel L. K. Yamins

TMLR 2024 / Paper

Partial-View Object View Synthesis via Filtering Inversion

Fan-Yun Sun*, Jonathan Tremblay, Valts Blukis, Kevin Lin, Danfei Xu, Boris Ivanovic, Peter Karkus, Stan Birchfield, Dieter Fox, Ruohan Zhang, Yunzhu Li, Jiajun Wu, Marco Pavone, Nick Haber

3DV 2024 / Paper / Project Page

Spotlight Presentation

2023

What's Left? Concept Grounding with Logic-Enhanced Foundation Models

Joy Hsu*, Jiayuan Mao*, Joshua B. Tenenbaum, Jiajun Wu

NeurIPS 2023 / Paper / Project Page

Inferring Hybrid Neural Fluid Fields from Videos

Hong-Xing Yu*, Yang Zheng*, Yuan Gao, Yitong Deng, Bo Zhu, Jiajun Wu

NeurIPS 2023 / Paper / Project Page

Siamese Masked Autoencoders

Agrim Gupta, Jiajun Wu, Jia Deng, Li Fei-Fei

NeurIPS 2023 / Paper / Project Page

Oral Presentation

3D Copy-Paste: Physically Plausible Object Insertion for Monocular 3D Detection

Yunhao Ge, Hong-Xing Yu, Cheng Zhao, Yuliang Guo, Xinyu Huang, Liu Ren, Laurent Itti, Jiajun Wu

NeurIPS 2023 / Paper / Project Page

Disentanglement via Latent Quantization

Kyle Hsu, Will Dorrell, James C. R. Whittington, Jiajun Wu, Chelsea Finn

NeurIPS 2023 / Paper / Project Page

Model-Based Control with Sparse Neural Dynamics

Ziang Liu*, Jeff He*, Genggeng Zhou, Tobia Marcucci, Li Fei-Fei, Jiajun Wu, Yunzhu Li

NeurIPS 2023 / Paper / Project Page

Stanford-ORB: A Real-World 3D Object Inverse Rendering Benchmark

Zhengfei Kuang*, Yunzhi Zhang*, Hong-Xing Yu, Samir Agarwala, Shangzhe Wu, Jiajun Wu

NeurIPS 2023 Datasets and Benchmarks / Paper / Project Page

Are These the Same Apple? Comparing Images Based on Object Intrinsics

Klemen Kotar*, Stephen Tian*, Hong-Xing Yu, Daniel L. K. Yamins, Jiajun Wu

NeurIPS 2023 Datasets and Benchmarks / Paper / Project Page

SoundCam: A Dataset for Finding Humans Using Room Acoustics

Mason Wang, Samuel Clarke, Jui-Hsien Wang, Ruohan Gao, Jiajun Wu

NeurIPS 2023 Datasets and Benchmarks / Paper / Project Page

Holistic Evaluation of Text-to-Image Models

Tony Lee*, Michihiro Yasunaga*, Chenlin Meng*, Yifan Mai, Joon Sung Park, Agrim Gupta, Yunzhi Zhang, Deepak Narayanan, Hannah Benita Teufel, Marco Bellagente, Minguk Kang, Taesung Park, Jure Leskovec, Jun-Yan Zhu, Li Fei-Fei, Jiajun Wu, Stefano Ermon, Percy Liang

NeurIPS 2023 Datasets and Benchmarks / Paper / Project Page

Spotlight Presentation

RoboCook: Long-Horizon Elasto-Plastic Object Manipulation with Diverse Tools

Haochen Shi*, Huazhe Xu*, Samuel Clarke, Yunzhu Li, Jiajun Wu

CoRL 2023 / Paper / Project Page

Best Systems Paper Award

VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models

Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Jiajun Wu, Li Fei-Fei

CoRL 2023 / Paper / Project Page

Oral Presentation

Learning to Design and Use Tools for Robotic Manipulation

Ziang Liu*, Stephen Tian*, Michelle Guo, C. Karen Liu, Jiajun Wu

CoRL 2023 / Paper / Project Page

NOIR: Neural Signal Operated Intelligent Robots for Everyday Activities

Ruohan Zhang*, Sharon Lee*, Minjune Hwang*, Ayano Hiranaka*, Chen Wang, Wensi Ai, Jin Jie Ryan Tan, Shreya Gupta, Yilun Hao, Gabrael Levine, Ruohan Gao, Anthony Norcia, Li Fei-Fei, Jiajun Wu

CoRL 2023 / Paper / Project Page

Composable Part-Based Manipulation

Weiyu Liu, Jiayuan Mao, Joy Hsu, Tucker Hermans, Animesh Garg, Jiajun Wu

CoRL 2023 / Paper / Project Page

Learning Sequential Acquisition Policies for Robot-Assisted Feeding

Priya Sundaresan, Jiajun Wu, Dorsa Sadigh

CoRL 2023 / Paper / Project Page

Compositional Diffusion-Based Continuous Constraint Solvers

Zhutian Yang, Jiayuan Mao, Yilun Du, Jiajun Wu, Joshua B. Tenenbaum, Tomas Lozano-Perez, Leslie P. Kaelbling

CoRL 2023 / Paper / Project Page

Object Motion Guided Human Motion Synthesis

Jiaman Li, Jiajun Wu†, C. Karen Liu†

TOG (SIGGRAPH Asia) 2023 / Paper / Project Page

Fluid Simulation on Neural Flow Maps

Yitong Deng, Hong-Xing Yu, Diyang Zhang, Jiajun Wu, Bo Zhu

TOG (SIGGRAPH Asia) 2023 / Paper / Project Page

Best Paper Award

Editing Motion Graphics Videos via Program Induction and Transformation

Sharon Zhang, Jiaju Ma, Jiajun Wu, Daniel Ritchie, Maneesh Agrawala

TOG (SIGGRAPH Asia) 2023 / Paper / Project Page

Tree-Structured Shading Decomposition

Chen Geng*, Hong-Xing Yu*, Sharon Zhang, Maneesh Agrawala, Jiajun Wu

ICCV 2023 / Paper / Project Page

VQ3D: Learning a 3D-Aware Generative Model on ImageNet

Kyle Sargent, Jing Yu Koh, Han Zhang, Huiwen Chang, Charles Herrmann, Pratul Srinivasan, Jiajun Wu, Deqing Sun

ICCV 2023 / Paper / Project Page

Best Paper Award Finalist

Rendering Humans from Object-Occluded Monocular Videos

Tiange Xiang, Adam Sun, Jiajun Wu, Ehsan Adeli, Li Fei-Fei

ICCV 2023 / Paper / Project Page

Primitive Skill-Based Robot Learning from Human Evaluative Feedback

Ayano Hiranaka*, Minjune Hwang*, Sharon Lee, Chen Wang, Li Fei-Fei, Jiajun Wu, Ruohan Zhang

IROS 2023 / Paper / Project Page

Seeing a Rose in Five Thousand Ways

Yunzhi Zhang, Shangzhe Wu, Noah Snavely, Jiajun Wu

CVPR 2023 / Paper / Project Page

Ego-Body Pose Estimation via Ego-Head Pose Estimation

Jiaman Li, C. Karen Liu†, Jiajun Wu†

CVPR 2023 / Paper / Project Page

Best Paper Award Candidate

RealImpact: A Dataset of Impact Sound Fields for Real Objects

Samuel Clarke, Ruohan Gao, Mason Wang, Mark Rau, Julia Xu, Jui-Hsien Wang, Doug L. James, Jiajun Wu

CVPR 2023 / Paper / Project Page

Highlight

The ObjectFolder Benchmark: Multisensory Learning with Neural and Real Objects

Ruohan Gao*, Yiming Dou*, Hao Li*, Tanmay Agarwal, Jeannette Bohg, Yunzhu Li, Li Fei-Fei, Jiajun Wu

CVPR 2023 / Paper / Project Page

NS3D: Neuro-Symbolic Grounding of 3D Objects and Relations

Joy Hsu, Jiayuan Mao, Jiajun Wu

CVPR 2023 / Paper / Project Page

Multi-Object Manipulation via Object-Centric Neural Scattering Functions

Stephen Tian*, Yancheng Cai*, Hong-Xing Yu, Sergey Zakharov, Katherine Liu, Adrien Gaidon, Yunzhu Li, Jiajun Wu

CVPR 2023 / Paper / Project Page

Accidental Light Probes

Hong-Xing Yu, Samir Agarwala, Charles Herrmann, Richard Szeliski, Noah Snavely, Jiajun Wu, Deqing Sun

CVPR 2023 / Paper / Project Page

3D Neural Field Generation using Triplane Diffusion

J. Ryan Shue*, Eric Ryan Chan*, Ryan Po*, Zachary Ankner*, Jiajun Wu, Gordon Wetzstein

CVPR 2023 / Paper / Project Page

Putting People in Their Place: Affordance-Aware Human Insertion into Scenes

Sumith Kulal, Tim Brooks, Alex Aiken, Jiajun Wu, Jimei Yang, Jingwan Lu, Alexei A. Efros, Krishna Kumar Singh

CVPR 2023 / Paper / Project Page

CIRCLE: Capture In Rich Contextual Environments

Joao Pedro Araujo, Jiaman Li, Karthik Vetrivel, Rishi Agarwal, Deepak Gopinath, Jiajun Wu, Alexander Clegg, C. Karen Liu

CVPR 2023 / Paper / Project Page

ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understanding

Le Xue, Mingfei Gao, Chen Xing, Roberto Martin-Martin, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, Silvio Savarese

CVPR 2023 / Paper / Project Page

PyPose: A Library for Robot Learning with Physics-based Optimization

PyPose Contributors

CVPR 2023 / Paper / Project Page

Motion Question Answering via Modular Motion Programs

Mark Endo*, Joy Hsu*, Jiaman Li, Jiajun Wu

ICML 2023 / Paper / Project Page

Modeling Dynamic Environments with Scene Graph Memory

Andrey Kurenkov, Michael Lingelbach, Tanmay Agarwal, Emily Jin, Chengshu Li, Ruohan Zhang, Li Fei-Fei, Jiajun Wu, Silvio Savarese, Roberto Martin-Martin

ICML 2023 / Paper / Project Page

Dynamic-Resolution Model Learning for Object Pile Manipulation

Yixuan Wang*, Yunzhu Li*, Katherine Driggs-Campbell, Li Fei-Fei, Jiajun Wu

RSS 2023 / Paper / Project Page

Best Paper Award at the IROS Workshop on Learning Meets Model-Based Methods

Quantifying the Effect of Visual Impairments on Daily Activities in Virtual, Interactive Environments

Wensi Ai, Sharon Lee, Li Fei-Fei, Jiajun Wu, Ruohan Zhang

CogSci 2023 / Paper

A Control-Centric Benchmark for Video Prediction

Stephen Tian, Chelsea Finn, Jiajun Wu

ICLR 2023 / Paper / Project Page

Programmatically Grounded, Compositionally Generalizable Robotic Manipulation

Renhao Wang, Jiayuan Mao, Joy Hsu, Hang Zhao, Jiajun Wu, Yang Gao

ICLR 2023 / Paper / Project Page

Notable Top 25%

Learning Vortex Dynamics for Fluid Inference and Prediction

Yitong Deng, Hong-Xing Yu, Jiajun Wu, Bo Zhu

ICLR 2023 / Paper / Project Page

MaskViT: Masked Visual Pre-Training for Video Prediction

Agrim Gupta, Stephen Tian, Yunzhi Zhang, Jiajun Wu, Roberto Martin-Martin, Li Fei-Fei

ICLR 2023 / Paper / Project Page

An Extensible Multi-Modal Multi-Task Object Dataset with Materials

Trevor Standley, Ruohan Gao, Dawn Chen, Jiajun Wu, Silvio Savarese

ICLR 2023 / Paper / Project Page

Neurosymbolic Models for Computer Graphics

Daniel Ritchie, Paul Guerrero, R. Kenny Jones, Niloy J. Mitra, Adriana Schulz, Karl D. D. Willis, Jiajun Wu

Eurographics STAR 2023 / Paper

Sonicverse: A Multisensory Simulation Platform for Embodied Household Agents that See and Hear

Ruohan Gao*, Hao Li*, Gokul Dharan, Zhuzhu Wang, Chengshu Li, Fei Xia, Silvio Savarese, Li Fei-Fei, Jiajun Wu

ICRA 2023 / Paper / Project Page

Task-Driven Graph Attention for Hierarchical Relational Object Navigation

Michael Lingelbach, Chengshu Li, Minjune Hwang, Andrey Kurenkov, Alan Lou, Roberto Martin-Martin, Ruohan Zhang, Li Fei-Fei, Jiajun Wu

ICRA 2023 / Paper

STAP: Sequencing Task-Agnostic Policies

Christopher Agia*, Toki Migimatsu*, Jiajun Wu, Jeannette Bohg

ICRA 2023 / Paper / Project Page

Differentiable Physics Simulation of Dynamics-Augmented Neural Objects

Simon Le Cleac'h, Hong-Xing Yu, Michelle Guo, Taylor Howell, Ruohan Gao, Jiajun Wu, Zachary Manchester, Mac Schwager

RA-L 2023 / Paper / Video

Learning Object-Centric Neural Scattering Functions for Free-viewpoint Relighting and Scene Composition

Hong-Xing Yu*, Michelle Guo*, Alireza Fathi, Yen-Yu Chang, Eric Ryan Chan, Ruohan Gao, Thomas Funkhouser, Jiajun Wu

TMLR 2023 / Paper / Project Page

Unsupervised Discovery and Composition of Object Light Fields

Cameron Smith, Hong-Xing Yu, Sergey Zakharov, Fredo Durand, Joshua B. Tenenbaum, Jiajun Wu, Vincent Sitzmann

TMLR 2023 / Paper / Project Page

DisCo: Improving Compositional Generalization in Visual Reasoning through Distribution Coverage

Joy Hsu, Jiayuan Mao, Jiajun Wu

TMLR 2023 / Paper / Project Page

Benchmarking Rigid Body Contact Models

Michelle Guo, Yifeng Jiang, Andrew Spielberg, Jiajun Wu, C. Karen Liu

L4DC 2023 / Paper / Project Page

Learning Rational Subgoals from Demonstrations and Instructions

Zhezheng Luo*, Jiayuan Mao*, Jiajun Wu, Tomas Lozano-Perez, Joshua B. Tenenbaum, Leslie P. Kaelbling

AAAI 2023 / Paper / Project Page

Physically Plausible Animation of Human Upper Body from a Single Image

Ziyuan Huang*, Zhengping Zhou*, Yung-Yu Chuang, Jiajun Wu, C. Karen Liu

WACV 2023 / Paper

2022

BEHAVIOR-1K: A Benchmark for Embodied AI with 1,000 Everyday Activities and Realistic Simulation

CoRL 2022 / Paper / Project Page

Best Paper Award Nominee

See, Hear, and Feel: Smart Sensory Fusion for Robotic Manipulation

Hao Li*, Yizhi Zhang*, Junzhe Zhu, Shaoxiong Wang, Michelle A. Lee, Huazhe Xu, Edward H. Adelson, Li Fei-Fei, Ruohan Gao†, Jiajun Wu†

CoRL 2022 / Paper / Project Page

A Dual Representation Framework for Robot Learning with Human Guidance

Ruohan Zhang*, Dhruva Bansal*, Yilun Hao, Ayano Hiranaka, Jialu Gao, Chen Wang, Roberto Martin-Martin, Li Fei-Fei, Jiajun Wu

CoRL 2022 / Paper / Project Page

Scene Synthesis from Human Motion

Sifan Ye*, Yixing Wang*, Jiaman Li, Dennis Park, C. Karen Liu, Huazhe Xu†, Jiajun Wu†

SIGGRAPH Asia 2022 / Paper / Project Page

Unsupervised Learning of Shape Programs with Repeatable Implicit Parts

Boyang Deng*, Sumith Kulal*, Zhengyang Dong, Congyue Deng, Yonglong Tian, Jiajun Wu

NeurIPS 2022 / Paper / Project Page

Interaction Modeling with Multiplex Attention

Fan-Yun Sun, Isaac Kauvar, Mykel Kochenderfer, Ruohan Zhang, Jiajun Wu, Jiachen Li, Nick Haber

NeurIPS 2022 / Paper / Project Page

E-MAPP: Efficient Multi-Agent Reinforcement Learning with Parallel Program Guidance

Can Chang, Ni Mu, Jiajun Wu, Ling Pan, Huazhe Xu

NeurIPS 2022 / Paper / Project Page

Spotlight Presentation

IKEA-Manual: Seeing Shape Assembly Step by Step

Ruocheng Wang, Yunzhi Zhang, Jiayuan Mao, Ran Zhang, Chin-Yi Cheng, Jiajun Wu

NeurIPS 2022 Datasets and Benchmarks / Paper / Project Page

CLEVRER-Humans: Describing Physical and Causal Events the Human Way

Jiayuan Mao*, Xuelin Yang*, Xikun Zhang, Noah D. Goodman, Jiajun Wu

NeurIPS 2022 Datasets and Benchmarks / Paper / Project Page

Geoclidean: Few-Shot Generalization in Euclidean Geometry

Joy Hsu, Jiajun Wu, Noah D. Goodman

NeurIPS 2022 Datasets and Benchmarks / Paper / Project Page

MOMA-LRG: Language-Refined Graphs for Multi-Object Multi-Actor Activity Parsing

Zelun Luo, Zane Durante*, Linden Li*, Wanze Xie, Ruochen Liu, Emily Jin, Zhuoyi Huang, Lun Yu Li, Jiajun Wu, Juan Carlos Niebles, Ehsan Adeli, Li Fei-Fei

NeurIPS 2022 Datasets and Benchmarks / Paper / Project Page

Sparse and Local Networks for Hypergraph Reasoning

Guangxuan Xiao, Leslie P. Kaelbling, Jiajun Wu, Jiayuan Mao

LoG 2022 / Paper / Project Page

Retrospectives on the Embodied AI Workshop

CVPR Embodied AI Workshop Organizers

arXiv 2022 / Paper / Project Page

Video Extrapolation in Space and Time

Yunzhi Zhang, Jiajun Wu

ECCV 2022 / Paper / Project Page

Translating a Visual LEGO Manual to a Machine-Executable Plan

Ruocheng Wang, Yunzhi Zhang, Jiayuan Mao, Chin-Yi Cheng, Jiajun Wu

ECCV 2022 / Paper / Project Page

Unsupervised Segmentation in Real-World Images via Spelke Object Inference

Honglin Chen, Rahul Venkatesh, Yoni Friedman, Jiajun Wu, Joshua B. Tenenbaum, Daniel L. K. Yamins†, Daniel M. Bear†

ECCV 2022 / Paper / Project Page

Oral Presentation

ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer

Ruohan Gao*, Zilin Si*, Yen-Yu Chang*, Samuel Clarke, Jeannette Bohg, Li Fei-Fei, Wenzhen Yuan, Jiajun Wu

CVPR 2022 / Paper / Project Page

Programmatic Concept Learning for Human Motion Description and Synthesis

Sumith Kulal*, Jiayuan Mao*, Alex Aiken†, Jiajun Wu†

CVPR 2022 / Paper / Project Page

Rotationally Equivariant 3D Object Detection

Hong-Xing Yu, Jiajun Wu, Li Yi

CVPR 2022 / Paper / Project Page

Revisiting the "Video" in Video-Language Understanding

Shyamal Buch, Cristobal Eyzaguirre, Adrien Gaidon, Jiajun Wu, Li Fei-Fei, Juan Carlos Niebles

CVPR 2022 / Paper / Project Page

Oral Presentation

RoboCraft: Learning to See, Simulate, and Shape Elasto-Plastic Objects with Graph Networks

Haochen Shi*, Huazhe Xu*, Zhiao Huang, Yunzhu Li, Jiajun Wu

RSS 2022 / Paper / Project Page

Unsupervised Discovery of Object Radiance Fields

Hong-Xing Yu, Leonidas J. Guibas, Jiajun Wu

ICLR 2022 / Paper / Project Page

Vision-Based Manipulators Need to Also See from Their Hands

Kyle Hsu*, Moo Jin Kim*, Rafael Rafailov, Jiajun Wu, Chelsea Finn

ICLR 2022 / Paper / Project Page

Oral Presentation

SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations

Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, Stefano Ermon

ICLR 2022 / Paper / Project Page

2021

ObjectFolder: A Dataset of Objects with Implicit Visual, Auditory, and Tactile Representations

Ruohan Gao, Yen-Yu Chang*, Shivani Mall*, Li Fei-Fei, Jiajun Wu

CoRL 2021 / Paper / Project Page

BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments

Sanjana Srivastava*, Chengshu Li*, Michael Lingelbach*, Roberto Martin-Martin*, Fei Xia, Kent Vainio, Zheng Lian, Cem Gokmen, Shyamal Buch, C. Karen Liu, Silvio Savarese, Hyowon Gweon, Jiajun Wu, Li Fei-Fei

CoRL 2021 / Paper / Project Page

DiffImpact: Differentiable Rendering and Identification of Impact Sounds

Samuel Clarke, Negin Heravi, Mark Rau, Ruohan Gao, Jiajun Wu, Doug L. James, Jeannette Bohg

CoRL 2021 / Paper / Project Page

Oral Presentation

iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks

Chengshu Li*, Fei Xia*, Roberto Martin-Martin*, Michael Lingelbach, Sanjana Srivastava, Bokui Shen, Kent Vainio, Cem Gokmen, Gokul Dharan, Tanish Jain, Andrey Kurenkov, C. Karen Liu, Hyowon Gweon, Jiajun Wu, Li Fei-Fei, Silvio Savarese

CoRL 2021 / Paper / Project Page

Single-Shot Scene Reconstruction

Sergey Zakharov, Rares Ambrus, Dennis Park, Vitor Guizilini, Wadim Kehl, Fredo Durand, Joshua B. Tenenbaum, Vincent Sitzmann, Jiajun Wu, Adrien Gaidon

CoRL 2021 / Paper

Grammar-Based Grounded Lexicon Learning

Jiayuan Mao, Haoyue Shi, Jiajun Wu, Roger P. Levy, Joshua B. Tenenbaum

NeurIPS 2021 / Paper / Project Page

3D Shape Generation and Completion Through Point-Voxel Diffusion

Linqi Zhou, Yilun Du, Jiajun Wu

ICCV 2021 / Paper / Project Page

Oral Presentation

Neural Radiance Flow for 4D View Synthesis and Video Processing

Yilun Du, Yinan Zhang, Hong-Xing Yu, Joshua B. Tenenbaum, Jiajun Wu

ICCV 2021 / Paper / Project Page

Learning Temporal Dynamics from Cycles in Narrated Video

Dave Epstein, Jiajun Wu, Cordelia Schmid, Chen Sun

ICCV 2021 / Paper / Project Page

Language-Mediated, Object-Centric Representation Learning

Ruocheng Wang*, Jiayuan Mao*, Samuel J. Gershman†, Jiajun Wu†

ACL 2021 Findings / Paper / Project Page

On the Opportunities and Risks of Foundation Models

Center for Research on Foundation Models

arXiv 2021 / Paper / Project Page

DASH: Modularized Human Manipulation Simulation with Vision and Language for Embodied AI

Yifeng Jiang, Michelle Guo, Jiangshan Li, Ioannis Exarchos, Jiajun Wu, C. Karen Liu

SCA 2021 / Paper / Video

Temporal and Object Quantification Networks

Jiayuan Mao*, Zhezheng Luo*, Chuang Gan, Joshua B. Tenenbaum, Jiajun Wu, Leslie P. Kaelbling, Tomer D. Ullman

IJCAI 2021 / Paper / Project Page

Hierarchical Motion Understanding via Motion Programs

Sumith Kulal*, Jiayuan Mao*, Alex Aiken, Jiajun Wu

CVPR 2021 / Paper / Project Page

pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis

Eric Ryan Chan*, Marco Monteiro*, Petr Kellnhofer, Jiajun Wu, Gordon Wetzstein

CVPR 2021 / Paper / Project Page

Oral Presentation

KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control

Tomas Jakab, Richard Tucker, Ameesh Makadia, Jiajun Wu, Noah Snavely, Angjoo Kanazawa

CVPR 2021 / Paper / Project Page

Oral Presentation

De-rendering the World's Revolutionary Artefacts

Shangzhe Wu, Ameesh Makadia, Jiajun Wu, Noah Snavely, Richard Tucker, Angjoo Kanazawa

CVPR 2021 / Paper / Project Page

Repopulating Street Scenes

Yifan Wang, Andrew Liu, Richard Tucker, Jiajun Wu, Brian Curless, Steven M. Seitz, Noah Snavely

CVPR 2021 / Paper / Project Page

When is Particle Filtering Efficient for Planning in Partially Observed Linear Dynamical Systems?

Simon S. Du*, Wei Hu*, Zhiyuan Li*, Ruoqi Shen*, Zhao Song*, Jiajun Wu*

UAI 2021 / Paper / Supplementary Material

Unsupervised Discovery of 3D Physical Objects from Video

Yilun Du, Kevin Smith, Tomer D. Ullman, Joshua B. Tenenbaum, Jiajun Wu

ICLR 2021 / Paper / Project Page

Grounding Physical Concepts of Objects and Events Through Dynamic Visual Reasoning

Zhenfang Chen, Jiayuan Mao, Jiajun Wu, Kwan-Yee K. Wong, Joshua B. Tenenbaum, Chuang Gan

ICLR 2021 / Paper / Project Page

Augmenting Policy Learning with Routines Discovered from a Single Demonstration

Zelin Zhao, Chuang Gan, Jiajun Wu, Xiaoxiao Guo, Joshua B. Tenenbaum

AAAI 2021 / Paper / Project Page

2020

Multi-Plane Program Induction with 3D Box Priors

Yikai Li*, Jiayuan Mao*, Xiuming Zhang, William T. Freeman, Joshua B. Tenenbaum, Noah Snavely, Jiajun Wu

NeurIPS 2020 / Paper / Project Page

Learning Physical Graph Representations from Visual Scenes

Daniel M. Bear, Chaofei Fan, Damian Mrowca, Yunzhu Li, Seth Alter, Aran Nayebi, Jeremy Schwartz, Li Fei-Fei, Jiajun Wu, Joshua B. Tenenbaum, Daniel L. K. Yamins

NeurIPS 2020 / Paper / Project Page

Oral Presentation

Learning 3D Dynamic Scene Representations for Robot Manipulation

Zhenjia Xu, Zhanpeng He, Jiajun Wu, Shuran Song

CoRL 2020 / Paper / Project Page

Multi-Frame to Single-Frame: Knowledge Distillation for 3D Object Detection

Yue Wang, Alireza Fathi, Jiajun Wu, Thomas Funkhouser, Justin Solomon

ECCV Workshop 2020 / Paper

Visual Grounding of Learned Physical Models

Yunzhu Li, Toru Lin*, Kexin Yi*, Daniel M. Bear, Daniel L. K. Yamins, Jiajun Wu, Joshua B. Tenenbaum, Antonio Torralba

ICML 2020 / Paper / Project Page

Object-Centric Diagnosis of Visual Reasoning

Jianwei Yang, Jiayuan Mao, Jiajun Wu, Devi Parikh, David D. Cox, Joshua B. Tenenbaum, Chuang Gan

arXiv 2020 / Paper

Object-Centric Neural Scene Rendering

Michelle Guo, Alireza Fathi, Jiajun Wu, Thomas Funkhouser

arXiv 2020 / Paper / Project Page

Perspective Plane Program Induction from a Single Image

Yikai Li*, Jiayuan Mao*, Xiuming Zhang, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu

CVPR 2020 / Paper / Project Page

Probabilistic Video Prediction from Noisy Data with a Posterior Confidence

Yunbo Wang, Jiajun Wu, Mingsheng Long, Joshua B. Tenenbaum

CVPR 2020 / Paper / Project Page

End-to-End Optimization of Scene Layout

Andrew Luo, Zhoutong Zhang, Jiajun Wu, Joshua B. Tenenbaum

CVPR 2020 / Paper / Project Page

Oral Presentation

Learning Compositional Koopman Operators for Model-Based Control

Yunzhu Li*, Hao He*, Jiajun Wu, Dina Katabi, Antonio Torralba

ICLR 2020 / Paper / Project Page

Spotlight Presentation

CLEVRER: Collision Events for Video Representation and Reasoning

Kexin Yi*, Chuang Gan*, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, Joshua B. Tenenbaum

ICLR 2020 / Paper / Project Page / MIT Technology Review

Spotlight Presentation

Deep Audio Priors Emerge from Harmonic Convolutional Networks

Zhoutong Zhang, Yunyun Wang, Chuang Gan, Jiajun Wu, Joshua B. Tenenbaum, Antonio Torralba, William T. Freeman

ICLR 2020 / Paper / Project Page

DualSMC: Tunneling Differentiable Filtering and Planning Under Continuous POMDPs

Yunbo Wang*, Bo Liu*, Jiajun Wu, Yuke Zhu, Simon S. Du, Li Fei-Fei, Joshua B. Tenenbaum

IJCAI 2020 / Paper / Project Page

Accurate Vision-based Manipulation through Contact Reasoning

Alina Kloss, Maria Bauza, Jiajun Wu, Joshua B. Tenenbaum, Alberto Rodriguez, Jeannette Bohg

ICRA 2020 / Paper / Video

Look, Listen, and Act: Towards Audio-Visual Embodied Navigation

Chuang Gan*, Yiwei Zhang*, Jiajun Wu, Boqing Gong, Joshua B. Tenenbaum

ICRA 2020 / Paper / Project Page

Learning Generative Models of 3D Structures

Siddhartha Chaudhuri, Daniel Ritchie, Jiajun Wu, Kai Xu, Hao Zhang

Eurographics STAR 2020 / Paper

The Fine Structure of Surprise in Intuitive Physics: When, Why, and How Much?

Kevin Smith, Lingjie Mei, Shunyu Yao, Jiajun Wu, Elizabeth S. Spelke, Joshua B. Tenenbaum, Tomer D. Ullman

CogSci 2020 / Paper

2019

Learning to See the Physical World

PhD Dissertation 2019

ACM Doctoral Dissertation Award Honorable Mention
Joint AAAI/ACM SIGAI Doctoral Dissertation Award
MIT George M. Sprowls PhD Thesis Award in Artificial Intelligence and Decision-Making

Visual Concept-Metaconcept Learning

Chi Han*, Jiayuan Mao*, Chuang Gan, Joshua B. Tenenbaum, Jiajun Wu

NeurIPS 2019 / Paper / Project Page

Modeling Expectation Violation in Intuitive Physics with Coarse Probabilistic Object Representations

Kevin Smith*, Lingjie Mei*, Shunyu Yao*, Jiajun Wu, Elizabeth S. Spelke, Joshua B. Tenenbaum, Tomer D. Ullman

NeurIPS 2019 / Paper / Project Page / MIT News

Program-Guided Image Manipulators

Jiayuan Mao*, Xiuming Zhang*, Yikai Li, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu

ICCV 2019 / Paper / Project Page

Entity Abstraction in Visual Model-Based Reinforcement Learning

Rishi Veerapaneni*, John D. Co-Reyes*, Michael Chang*, Michael Janner, Chelsea Finn, Jiajun Wu, Joshua B. Tenenbaum, Sergey Levine

CoRL 2019 / Paper / Project Page

A Computational Model for Combinatorial Generalization in Physical Perception from Sound

Yunyun Wang, Chuang Gan, Max H. Siegel, Zhoutong Zhang, Jiajun Wu, Joshua B. Tenenbaum

CCN 2019 / Paper

DensePhysNet: Learning Dense Physical Object Representations via Multi-Step Dynamic Interactions

Zhenjia Xu, Jiajun Wu, Andy Zeng, Joshua B. Tenenbaum, Shuran Song

RSS 2019 / Paper / Project Page

Neurally-Guided Structure Inference

Sidi Lu*, Jiayuan Mao*, Joshua B. Tenenbaum, Jiajun Wu

ICML 2019 / Paper / Project Page

The Neuro-Symbolic Concept Learner:
Interpreting Scenes, Words, and Sentences from Natural Supervision

Jiayuan Mao, Chuang Gan, Pushmeet Kohli, Joshua B. Tenenbaum, Jiajun Wu

ICLR 2019 / Paper / Project Page / MIT News / MIT Technology Review

Oral Presentation

Reasoning About Physical Interactions with Object-Oriented Prediction and Planning

Michael Janner, Sergey Levine, William T. Freeman, Joshua B. Tenenbaum, Chelsea Finn, Jiajun Wu

ICLR 2019 / Paper / Project Page

Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids

Yunzhu Li, Jiajun Wu, Russ Tedrake, Joshua B. Tenenbaum, Antonio Torralba

ICLR 2019 / Paper / Project Page / MIT News

Learning to Infer and Execute 3D Shape Programs

Yonglong Tian, Andrew Luo, Xingyuan Sun, Kevin Ellis, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu

ICLR 2019 / Paper / Project Page

Unsupervised Discovery of Parts, Structure, and Dynamics

Zhenjia Xu*, Zhijian Liu*, Chen Sun, Kevin Murphy, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu

ICLR 2019 / Paper / Project Page

Stochastic Prediction of Multi-Agent Interactions from Partial Observations

Chen Sun, Per Karlsson, Jiajun Wu, Joshua B. Tenenbaum, Kevin Murphy

ICLR 2019 / Paper

Learning to Describe Scenes with Programs

Yunchao Liu, Zheng Wu, Daniel Ritchie, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu

ICLR 2019 / Paper

See, Feel, Act: Hierarchical Learning for Complex Manipulation Skills with Multisensory Fusion

Nima Fazeli, Miquel Oller, Jiajun Wu, Zheng Wu, Joshua B. Tenenbaum, Alberto Rodriguez

Science Robotics 2019 / Paper / Video / MIT News / BBC / CBS / CNN / TechCrunch / Gizmodo / Popular Science / The Guardian / WIRED

An Integrative Computational Architecture for Object-Driven Cortex

Ilker Yildirim, Jiajun Wu, Nancy Kanwisher, Joshua B. Tenenbaum

CONEUR 2019 / Paper

ChainQueen: A Real-Time Differentiable Physical Simulator for Soft Robotics

Yuanming Hu, Jiancheng Liu*, Andrew Spielberg*, Joshua B. Tenenbaum, William T. Freeman, Jiajun Wu, Daniela Rus, Wojciech Matusik

ICRA 2019 / Paper / Project Page / Video

Propagation Networks for Model-Based Control Under Partial Observation

Yunzhu Li, Jiajun Wu, Jun-Yan Zhu, Antonio Torralba, Joshua B. Tenenbaum, Russ Tedrake

ICRA 2019 / Paper / Project Page

Combining Physical Simulators and Object-Based Networks for Control

Anurag Ajay, Maria Bauza, Jiajun Wu, Nima Fazeli, Joshua B. Tenenbaum, Alberto Rodriguez, Leslie P. Kaelbling

ICRA 2019 / Paper / Project Page

Video Enhancement with Task-Oriented Flow

Tianfan Xue, Baian Chen, Jiajun Wu, Donglai Wei, William T. Freeman

IJCV 2019 / Paper / Project Page

Visual Dynamics: Stochastic Future Generation via Layered Cross Convolutional Networks

Tianfan Xue*, Jiajun Wu*, Katherine L. Bouman, William T. Freeman

TPAMI 2019 / Paper / Project Page

2018

Learning to Reconstruct Shapes from Unseen Classes

Xiuming Zhang*, Zhoutong Zhang*, Chengkai Zhang, Joshua B. Tenenbaum, William T. Freeman, Jiajun Wu

NeurIPS 2018 / Paper / Project Page

Oral Presentation

Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding

Kexin Yi*, Jiajun Wu*, Chuang Gan, Antonio Torralba, Pushmeet Kohli, Joshua B. Tenenbaum

NeurIPS 2018 / Paper / Project Page

Spotlight Presentation

Learning to Exploit Stability for 3D Scene Parsing

Yilun Du, Zhijian Liu, Hector Basevi, Ales Leonardis, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu

NeurIPS 2018 / Paper / Project Page

Visual Object Networks: Image Generation with Disentangled 3D Representation

Jun-Yan Zhu, Zhoutong Zhang, Chengkai Zhang, Jiajun Wu, Antonio Torralba, Joshua B. Tenenbaum, William T. Freeman

NeurIPS 2018 / Paper / Project Page

3D-Aware Scene Manipulation via Inverse Graphics

Shunyu Yao*, Tzu-Ming Harry Hsu*, Jun-Yan Zhu, Jiajun Wu, Antonio Torralba, William T. Freeman, Joshua B. Tenenbaum

NeurIPS 2018 / Paper / Project Page

Augmenting Physical Simulators with Stochastic Neural Networks:
Case Study of Planar Pushing and Bouncing

Anurag Ajay, Jiajun Wu, Nima Fazeli, Maria Bauza, Leslie P. Kaelbling, Joshua B. Tenenbaum, Alberto Rodriguez

IROS 2018 / Paper / Project Page

Best Paper Award on Cognitive Robotics

3D Shape Perception from Monocular Vision, Touch, and Shape Priors

Shaoxiong Wang*, Jiajun Wu*, Xingyuan Sun, Wenzhen Yuan, William T. Freeman, Joshua B. Tenenbaum, Edward H. Adelson

IROS 2018 / Paper / Project Page

3D Interpreter Networks for Viewer-Centered Wireframe Modeling

Jiajun Wu*, Tianfan Xue*, Joseph J. Lim, Yuandong Tian, Joshua B. Tenenbaum, Antonio Torralba, William T. Freeman

IJCV 2018 / Paper / Project Page

Learning Sight from Sound: Ambient Sound Provides Supervision for Visual Learning

Andrew Owens, Jiajun Wu, Josh H. McDermott, William T. Freeman, Antonio Torralba

IJCV 2018 / Paper / Project Page

Physical Primitive Decomposition

Zhijian Liu, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu

ECCV 2018 / Paper / Project Page

Learning Shape Priors for Single-View 3D Completion and Reconstruction

Jiajun Wu*, Chengkai Zhang*, Xiuming Zhang, Zhoutong Zhang, William T. Freeman, Joshua B. Tenenbaum

ECCV 2018 / Paper / Project Page

Seeing Tree Structure from Vibration

Tianfan Xue*, Jiajun Wu*, Zhoutong Zhang, Chengkai Zhang, Joshua B. Tenenbaum, William T. Freeman

ECCV 2018 / Paper / Project Page

Neurocomputational Modeling of Human Physical Scene Understanding

Ilker Yildirim*, Kevin Smith*, Mario Belledonne*, Jiajun Wu, Joshua B. Tenenbaum

CCN 2018 / Paper

Oral Presentation

MoSculp: Interactive Visualization of Shape and Time

Xiuming Zhang, Tali Dekel, Tianfan Xue, Andrew Owens, Qiurui He, Jiajun Wu, Stefanie Mueller, William T. Freeman

UIST 2018 / Paper / Project Page / MIT News / Forbes / BBC

Unsupervised Learning of Latent Physical Properties Using Perception-Prediction Networks

David Zheng, Vinson Luo, Jiajun Wu, Joshua B. Tenenbaum

UAI 2018 / Paper / Project Page

Oral Presentation

Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling

Xingyuan Sun*, Jiajun Wu*, Xiuming Zhang, Zhoutong Zhang, Chengkai Zhang, Tianfan Xue, Joshua B. Tenenbaum, William T. Freeman

CVPR 2018 / Paper / Project Page

Attention Clusters: Purely Attention Based Local Feature Integration for Video Classification

Xiang Long, Chuang Gan, Gerard de Melo, Jiajun Wu, Xiao Liu, Shilei Wen

CVPR 2018 / Paper

2017

Learning to See Physics via Visual De-animation

Jiajun Wu, Erika Lu, Pushmeet Kohli, William T. Freeman, Joshua B. Tenenbaum

NeurIPS 2017 / Paper / Project Page / MIT News

Spotlight Presentation

MarrNet: 3D Shape Reconstruction via 2.5D Sketches

Jiajun Wu*, Yifan Wang*, Tianfan Xue, Xingyuan Sun, William T. Freeman, Joshua B. Tenenbaum

NeurIPS 2017 / Paper / Project Page

Self-Supervised Intrinsic Image Decomposition

Michael Janner, Jiajun Wu, Tejas D. Kulkarni, Ilker Yildirim, Joshua B. Tenenbaum

NeurIPS 2017 / Paper / Project Page

Best Paper Award at the New England Computer Vision Workshop

Shape and Material from Sound

Zhoutong Zhang, Qiujia Li, Zhengjia Huang, Jiajun Wu, Joshua B. Tenenbaum, William T. Freeman

NeurIPS 2017 / Paper / Project Page

Spotlight Presentation

Generative Modeling of Audible Shapes for Object Perception

Zhoutong Zhang*, Jiajun Wu*, Qiujia Li, Zhengjia Huang, James Traer, Josh H. McDermott, Joshua B. Tenenbaum, William T. Freeman

ICCV 2017 / Paper / Project Page

Raster-to-Vector: Revisiting Floorplan Transformation

Chen Liu, Jiajun Wu, Pushmeet Kohli, Yasutaka Furukawa

ICCV 2017 / Paper / Project Page

Neural Scene De-rendering

Jiajun Wu, Joshua B. Tenenbaum, Pushmeet Kohli

CVPR 2017 / Paper / Project Page

Synthesizing 3D Shapes via Modeling Multi-View Depth Maps and Silhouettes with Deep Generative Networks

Amir A. Soltani, Haibin Huang, Jiajun Wu, Tejas D. Kulkarni, Joshua B. Tenenbaum

CVPR 2017 / Paper / Project Page

2016

Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks

Tianfan Xue*, Jiajun Wu*, Katherine L. Bouman, William T. Freeman

NeurIPS 2016 / Paper / Project Page

Oral Presentation

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Jiajun Wu*, Chengkai Zhang*, Tianfan Xue, William T. Freeman, Joshua B. Tenenbaum

NeurIPS 2016 / Paper / Project Page / MIT Technology Review

Single Image 3D Interpreter Network

Jiajun Wu*, Tianfan Xue*, Joseph J. Lim, Yuandong Tian, Joshua B. Tenenbaum, Antonio Torralba, William T. Freeman

ECCV 2016 / Paper / Project Page

Oral Presentation

Ambient Sound Provides Supervision for Visual Learning

Andrew Owens, Jiajun Wu, Josh H. McDermott, William T. Freeman, Antonio Torralba

ECCV 2016 / Paper / Project Page

Oral Presentation

Physics 101: Learning Physical Object Properties from Unlabeled Videos

Jiajun Wu, Joseph J. Lim, Hongyi Zhang, Joshua B. Tenenbaum, William T. Freeman

BMVC 2016 / Paper / Project Page

Deep Multi-Modal Image Correspondence Learning

Chen Liu, Jiajun Wu, Pushmeet Kohli, Yasutaka Furukawa

arXiv 2016 / Paper / Project Page

A Comparative Evaluation of Approximate Probabilistic Simulation and Deep Neural Networks as Accounts of Human Physical Scene Understanding

Renqiao Zhang*, Jiajun Wu*, Chengkai Zhang, William T. Freeman, Joshua B. Tenenbaum

CogSci 2016 / Paper / Project Page

Oral Presentation

2015

Galileo: Perceiving Physical Object Properties by Integrating a Physics Engine with Deep Learning

Jiajun Wu*, Ilker Yildirim*, Joseph J. Lim, William T. Freeman, Joshua B. Tenenbaum

NeurIPS 2015 / Paper / Project Page / Demo / MIT News / VICE / WIRED / Engadget / Popular Science / The Boston Globe / PCWorld / TechRadar

Deep Multiple Instance Learning for Image Classification and Auto-Annotation

Jiajun Wu, Yinan Yu, Chang Huang, Kai Yu

CVPR 2015 / Paper / Project Page

Unsupervised Object Class Discovery via Saliency-Guided Multiple Class Learning

Jun-Yan Zhu, Jiajun Wu, Yan Xu, Eric Chang, Zhuowen Tu

TPAMI 2015 / Paper / Project Page

2014 and before

MILCut: A Sweeping Line Multiple Instance Learning Paradigm for Interactive Image Segmentation

Jiajun Wu*, Yibiao Zhao*, Jun-Yan Zhu, Siwei Luo, Zhuowen Tu

CVPR 2014 / Paper / Project Page

Harvesting Motion Patterns in Still Images from the Internet

Jiajun Wu, Yining Wang, Zhulin Li, Zhuowen Tu

CogSci 2014 / Paper / Project Page

Oral Presentation

Reverse Image Segmentation: A High-Level Solution to a Low-Level Task

Jiajun Wu, Jun-Yan Zhu, Zhuowen Tu

BMVC 2014 / Paper / Project Page

Harvesting Mid-Level Visual Concepts from Large-Scale Internet Images

Quannan Li, Jiajun Wu, Zhuowen Tu

CVPR 2013 / Paper / Project Page

Unsupervised Object Class Discovery via Saliency-Guided Multiple Class Learning

Jun-Yan Zhu, Jiajun Wu, Yichen Wei, Eric Chang, Zhuowen Tu

CVPR 2012 / Paper / Project Page

A Classification Approach to Coreference in Discharge Summaries: 2011 I2b2 Challenge

Yan Xu, Jiahua Liu, Jiajun Wu, Yue Wang, Zhuowen Tu, Jian-Tao Sun, Junichi Tsujii, Eric Chang

JAMIA 2012 / Paper

Physical Scene Understanding

NEUROK: Generative 4D Neural Object Kinematics

Chen Geng*, Guangzhao He*, Yue Gao*, Yunzhi Zhang, Shangzhe Wu, Jiajun Wu

CVPR 2026 / Paper

AnyLift: Scaling Motion Reconstruction from Internet Videos via 2D Diffusion

Hongjie Li*, Heng Yu*, Jiaman Li, Hong-Xing Yu, Ehsan Adeli, C. Karen Liu, Jiajun Wu

CVPR 2026 / Paper

ART: Articulated Reconstruction Transformer

Zizhang Li, Cheng Zhang, Zhengqin Li, Henry Howard-Jenkins, Zhaoyang Lv, Chen Geng, Jiajun Wu, Richard Newcombe, Jakob Engel, Zhao Dong

CVPR 2026 / Paper / Project Page

ENACT: Evaluating Embodied Cognition with World Modeling of Egocentric Interaction

Qineng Wang*, Wenlong Huang*, Yu Zhou, Hang Yin, Tianwei Bao, Jianwen Lyu, Weiyu Liu, Ruohan Zhang†, Jiajun Wu†, Li Fei-Fei†, Manling Li†

ICLR 2026 / Paper / Project Page

Discovering Hybrid World Representations with Co-Evolving Foundation Models

Jiajun Wu, Yunzhi Zhang, Hong-Xing Yu, Joy Hsu, Jiayuan Mao

AAAI 2026 Emerging Trends in AI / Paper

Self-Supervised Learning of Motion Concepts by Optimizing Counterfactuals

Stefan Stojanov*, David Wendt*, Seungwoo Kim*, Rahul Venkatesh*, Kevin Feigelis, Jiajun Wu, Daniel L. K. Yamins

NeurIPS 2025 / Paper / Project Page

Spotlight Presentation

Seeing the Wind from a Falling Leaf

Zhiyuan Gao*, Jiageng Mao*, Hong-Xing Yu, Haozhe Lou, Emily Yue-ting Jia, Jernej Barbic, Jiajun Wu, Yue Wang

NeurIPS 2025 / Paper / Project Page

Taming Generative Video Models for Zero-Shot Optical Flow Extraction

Seungwoo Kim*, Khai Loong Aw*, Klemen Kotar*, Cristobal Eyzaguirre, Wanhee Lee, Yunong Liu, Jared Watrous, Stefan Stojanov, Juan Carlos Niebles, Jiajun Wu, Daniel L. K. Yamins

NeurIPS 2025 / Paper / Project Page

Web-Scale Collection of Video Data for 4D Animal Reconstruction

Brian Nlong Zhao, Jiajun Wu†, Shangzhe Wu†

NeurIPS 2025 Datasets and Benchmarks / Paper

Weakly-Supervised Learning of Dense Functional Correspondences

Stefan Stojanov*, Linan Zhao*, Yunzhi Zhang, Daniel L. K. Yamins, Jiajun Wu

ICCV 2025 / Paper / Project Page

Birth and Death of a Rose

Chen Geng, Yunzhi Zhang, Shangzhe Wu, Jiajun Wu

CVPR 2025 / Paper / Project Page

Oral Presentation

FluidNexus: 3D Fluid Reconstruction and Prediction from a Single Video

Yue Gao*, Hong-Xing Yu*, Bo Zhu, Jiajun Wu

CVPR 2025 / Paper / Project Page

Oral Presentation

Lifting Motion to the 3D World via 2D Diffusion

Jiaman Li, C. Karen Liu†, Jiajun Wu†

CVPR 2025 / Paper / Project Page

Highlight

Category-Agnostic Neural Object Rigging

Guangzhao He*, Chen Geng*, Shangzhe Wu, Jiajun Wu

CVPR 2025 / Paper / Project Page

Digital Twin Catalog: A Large-Scale Photorealistic 3D Object Digital Twin Dataset

CVPR 2025 / Paper / Project Page

Highlight

PGC: Physics-Based Gaussian Cloth from a Single Pose

Michelle Guo, Matt Jen-Yuan Chiang, Igor Santesteban, Nikolaos Sarafianos, Hsiao-yu Chen, Oshri Halimi, Aljaz Bozic, Shunsuke Saito, Jiajun Wu, C. Karen Liu, Tuur Stuyck, Egor Larionov

CVPR 2025 / Paper / Project Page

Highlight

Anymate: A Dataset and Baselines for Learning 3D Object Rigging

Yufan Deng*, Yuhao Zhang*, Chen Geng, Shangzhe Wu†, Jiajun Wu†

SIGGRAPH 2025 / Paper / Project Page

Unsupervised Discovery of Object-Centric Neural Fields

Rundong Luo*, Hong-Xing Yu*, Jiajun Wu

TMLR 2025 / Paper / Project Page

Physical Scene Understanding

Jiajun Wu

AI Magazine 2024 / Paper

IKEA Manuals at Work: 4D Grounding of Assembly Instructions on Internet Videos

Yunong Liu, Cristobal Eyzaguirre, Manling Li, Shubh Khanna, Juan Carlos Niebles, Vineeth Ravi, Saumitra Mishra, Weiyu Liu†, Jiajun Wu†

NeurIPS 2024 Datasets and Benchmarks / Paper / Project Page

MARPLE: A Benchmark for Long-Horizon Inference

Emily Jin*, Zhuoyi Huang*, Jan-Philipp Fränken, Weiyu Liu, Hannah Cha, Erik Brockbank, Sarah A. Wu, Ruohan Zhang, Jiajun Wu, Tobias Gerstenberg

NeurIPS 2024 Datasets and Benchmarks / Paper / Project Page

An Eulerian Vortex Method on Flow Maps

Sinan Wang, Yitong Deng, Molin Deng, Hong-Xing Yu, Junwei Zhou, Duowen Chen, Taku Komura, Jiajun Wu, Bo Zhu

TOG (SIGGRAPH Asia) 2024 / Paper / Project Page

Ponymation: Learning Articulated 3D Animal Motions from Unlabeled Online Videos

Keqiang Sun*, Dor Litvak*, Yunzhi Zhang, Hongsheng Li, Jiajun Wu†, Shangzhe Wu†

ECCV 2024 / Paper / Project Page

3D Congealing: 3D-Aware Image Alignment in the Wild

Yunzhi Zhang, Zizhang Li, Amit Raj, Andreas Engelhardt, Yuanzhen Li, Tingbo Hou, Jiajun Wu, Varun Jampani

ECCV 2024 / Paper / Project Page

Reconstruction and Simulation of Elastic Objects with Spring-Mass 3D Gaussians

Licheng Zhong, Hong-Xing Yu, Jiajun Wu, Yunzhu Li

ECCV 2024 / Paper / Project Page

Learning the 3D Fauna of the Web

Zizhang Li*, Dor Litvak*, Ruining Li, Yunzhi Zhang, Tomas Jakab, Christian Rupprecht, Shangzhe Wu†, Andrea Vedaldi†, Jiajun Wu†

CVPR 2024 / Paper / Project Page

Neural Polynomial Gabor Fields for Macro Motion Analysis

Chen Geng*, Hong-Xing Yu*, Sida Peng, Xiaowei Zhou, Jiajun Wu

ICLR 2024 / Paper / Project Page

Physically Grounded Vision-Language Models for Robotic Manipulation

Jensen Gao, Bidipta Sarkar, Fei Xia, Ted Xiao, Jiajun Wu, Brian Ichter, Anirudha Majumdar, Dorsa Sadigh

ICRA 2024 / Paper / Project Page

CityPulse: Fine-Grained Assessment of Urban Change with Street View Time Series

Tianyuan Huang, Zejia Wu, Jiajun Wu, Jackelyn Hwang, Ram Rajagopal

AAAI 2024 AI for Social Impact / Paper

Unsupervised 3D Scene Representation Learning via Movable Object Inference

Honglin Chen*, Wanhee Lee*, Hong-Xing Yu, Rahul Venkatesh, Joshua B. Tenenbaum, Daniel M. Bear, Jiajun Wu, Daniel L. K. Yamins

TMLR 2024 / Paper

Inferring Hybrid Neural Fluid Fields from Videos

Hong-Xing Yu*, Yang Zheng*, Yuan Gao, Yitong Deng, Bo Zhu, Jiajun Wu

NeurIPS 2023 / Paper / Project Page

Siamese Masked Autoencoders

Agrim Gupta, Jiajun Wu, Jia Deng, Li Fei-Fei

NeurIPS 2023 / Paper / Project Page

Oral Presentation

3D Copy-Paste: Physically Plausible Object Insertion for Monocular 3D Detection

Yunhao Ge, Hong-Xing Yu, Cheng Zhao, Yuliang Guo, Xinyu Huang, Liu Ren, Laurent Itti, Jiajun Wu

NeurIPS 2023 / Paper / Project Page

Stanford-ORB: A Real-World 3D Object Inverse Rendering Benchmark

Zhengfei Kuang*, Yunzhi Zhang*, Hong-Xing Yu, Samir Agarwala, Shangzhe Wu, Jiajun Wu

NeurIPS 2023 Datasets and Benchmarks / Paper / Project Page

Are These the Same Apple? Comparing Images Based on Object Intrinsics

Klemen Kotar*, Stephen Tian*, Hong-Xing Yu, Daniel L. K. Yamins, Jiajun Wu

NeurIPS 2023 Datasets and Benchmarks / Paper / Project Page

Object Motion Guided Human Motion Synthesis

Jiaman Li, Jiajun Wu†, C. Karen Liu†

TOG (SIGGRAPH Asia) 2023 / Paper / Project Page

Fluid Simulation on Neural Flow Maps

Yitong Deng, Hong-Xing Yu, Diyang Zhang, Jiajun Wu, Bo Zhu

TOG (SIGGRAPH Asia) 2023 / Paper / Project Page

Best Paper Award

Ego-Body Pose Estimation via Ego-Head Pose Estimation

Jiaman Li, C. Karen Liu†, Jiajun Wu†

CVPR 2023 / Paper / Project Page

Best Paper Award Candidate

Accidental Light Probes

Hong-Xing Yu, Samir Agarwala, Charles Herrmann, Richard Szeliski, Noah Snavely, Jiajun Wu, Deqing Sun

CVPR 2023 / Paper / Project Page

CIRCLE: Capture In Rich Contextual Environments

Joao Pedro Araujo, Jiaman Li, Karthik Vetrivel, Rishi Agarwal, Deepak Gopinath, Jiajun Wu, Alexander Clegg, C. Karen Liu

CVPR 2023 / Paper / Project Page

Learning Vortex Dynamics for Fluid Inference and Prediction

Yitong Deng, Hong-Xing Yu, Jiajun Wu, Bo Zhu

ICLR 2023 / Paper / Project Page

Differentiable Physics Simulation of Dynamics-Augmented Neural Objects

Simon Le Cleac'h, Hong-Xing Yu, Michelle Guo, Taylor Howell, Ruohan Gao, Jiajun Wu, Zachary Manchester, Mac Schwager

RA-L 2023 / Paper / Video

Learning Object-Centric Neural Scattering Functions for Free-viewpoint Relighting and Scene Composition

Hong-Xing Yu*, Michelle Guo*, Alireza Fathi, Yen-Yu Chang, Eric Ryan Chan, Ruohan Gao, Thomas Funkhouser, Jiajun Wu

TMLR 2023 / Paper / Project Page

Unsupervised Discovery and Composition of Object Light Fields

Cameron Smith, Hong-Xing Yu, Sergey Zakharov, Fredo Durand, Joshua B. Tenenbaum, Jiajun Wu, Vincent Sitzmann

TMLR 2023 / Paper / Project Page

Benchmarking Rigid Body Contact Models

Michelle Guo, Yifeng Jiang, Andrew Spielberg, Jiajun Wu, C. Karen Liu

L4DC 2023 / Paper / Project Page

Interaction Modeling with Multiplex Attention

Fan-Yun Sun, Isaac Kauvar, Mykel Kochenderfer, Ruohan Zhang, Jiajun Wu, Jiachen Li, Nick Haber

NeurIPS 2022 / Paper / Project Page

Unsupervised Segmentation in Real-World Images via Spelke Object Inference

Honglin Chen, Rahul Venkatesh, Yoni Friedman, Jiajun Wu, Joshua B. Tenenbaum, Daniel L. K. Yamins†, Daniel M. Bear†

ECCV 2022 / Paper / Project Page

Oral Presentation

Rotationally Equivariant 3D Object Detection

Hong-Xing Yu, Jiajun Wu, Li Yi

CVPR 2022 / Paper / Project Page

Unsupervised Discovery of Object Radiance Fields

Hong-Xing Yu, Leonidas J. Guibas, Jiajun Wu

ICLR 2022 / Paper / Project Page

Single-Shot Scene Reconstruction

Sergey Zakharov, Rares Ambrus, Dennis Park, Vitor Guizilini, Wadim Kehl, Fredo Durand, Joshua B. Tenenbaum, Vincent Sitzmann, Jiajun Wu, Adrien Gaidon

CoRL 2021 / Paper

Unsupervised Discovery of 3D Physical Objects from Video

Yilun Du, Kevin Smith, Tomer D. Ullman, Joshua B. Tenenbaum, Jiajun Wu

ICLR 2021 / Paper / Project Page

Learning Physical Graph Representations from Visual Scenes

Daniel M. Bear, Chaofei Fan, Damian Mrowca, Yunzhu Li, Seth Alter, Aran Nayebi, Jeremy Schwartz, Li Fei-Fei, Jiajun Wu, Joshua B. Tenenbaum, Daniel L. K. Yamins

NeurIPS 2020 / Paper / Project Page

Oral Presentation

Visual Grounding of Learned Physical Models

Yunzhu Li, Toru Lin*, Kexin Yi*, Daniel M. Bear, Daniel L. K. Yamins, Jiajun Wu, Joshua B. Tenenbaum, Antonio Torralba

ICML 2020 / Paper / Project Page

The Fine Structure of Surprise in Intuitive Physics: When, Why, and How Much?

Kevin Smith, Lingjie Mei, Shunyu Yao, Jiajun Wu, Elizabeth S. Spelke, Joshua B. Tenenbaum, Tomer D. Ullman

CogSci 2020 / Paper

Learning to See the Physical World

PhD Dissertation 2019

ACM Doctoral Dissertation Award Honorable Mention
Joint AAAI/ACM SIGAI Doctoral Dissertation Award
MIT George M. Sprowls PhD Thesis Award in Artificial Intelligence and Decision-Making

Modeling Expectation Violation in Intuitive Physics with Coarse Probabilistic Object Representations

Kevin Smith*, Lingjie Mei*, Shunyu Yao*, Jiajun Wu, Elizabeth S. Spelke, Joshua B. Tenenbaum, Tomer D. Ullman

NeurIPS 2019 / Paper / Project Page / MIT News

Entity Abstraction in Visual Model-Based Reinforcement Learning

Rishi Veerapaneni*, John D. Co-Reyes*, Michael Chang*, Michael Janner, Chelsea Finn, Jiajun Wu, Joshua B. Tenenbaum, Sergey Levine

CoRL 2019 / Paper / Project Page

Unsupervised Discovery of Parts, Structure, and Dynamics

Zhenjia Xu*, Zhijian Liu*, Chen Sun, Kevin Murphy, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu

ICLR 2019 / Paper / Project Page

Stochastic Prediction of Multi-Agent Interactions from Partial Observations

Chen Sun, Per Karlsson, Jiajun Wu, Joshua B. Tenenbaum, Kevin Murphy

ICLR 2019 / Paper

An Integrative Computational Architecture for Object-Driven Cortex

Ilker Yildirim, Jiajun Wu, Nancy Kanwisher, Joshua B. Tenenbaum

CONEUR 2019 / Paper

Learning to Reconstruct Shapes from Unseen Classes

Xiuming Zhang*, Zhoutong Zhang*, Chengkai Zhang, Joshua B. Tenenbaum, William T. Freeman, Jiajun Wu

NeurIPS 2018 / Paper / Project Page

Oral Presentation

Learning to Exploit Stability for 3D Scene Parsing

Yilun Du, Zhijian Liu, Hector Basevi, Ales Leonardis, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu

NeurIPS 2018 / Paper / Project Page

3D-Aware Scene Manipulation via Inverse Graphics

Shunyu Yao*, Tzu-Ming Harry Hsu*, Jun-Yan Zhu, Jiajun Wu, Antonio Torralba, William T. Freeman, Joshua B. Tenenbaum

NeurIPS 2018 / Paper / Project Page

3D Interpreter Networks for Viewer-Centered Wireframe Modeling

Jiajun Wu*, Tianfan Xue*, Joseph J. Lim, Yuandong Tian, Joshua B. Tenenbaum, Antonio Torralba, William T. Freeman

IJCV 2018 / Paper / Project Page

Physical Primitive Decomposition

Zhijian Liu, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu

ECCV 2018 / Paper / Project Page

Seeing Tree Structure from Vibration

Tianfan Xue*, Jiajun Wu*, Zhoutong Zhang, Chengkai Zhang, Joshua B. Tenenbaum, William T. Freeman

ECCV 2018 / Paper / Project Page

Neurocomputational Modeling of Human Physical Scene Understanding

Ilker Yildirim*, Kevin Smith*, Mario Belledonne*, Jiajun Wu, Joshua B. Tenenbaum

CCN 2018 / Paper

Oral Presentation

Unsupervised Learning of Latent Physical Properties Using Perception-Prediction Networks

David Zheng, Vinson Luo, Jiajun Wu, Joshua B. Tenenbaum

UAI 2018 / Paper / Project Page

Oral Presentation

Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling

Xingyuan Sun*, Jiajun Wu*, Xiuming Zhang, Zhoutong Zhang, Chengkai Zhang, Tianfan Xue, Joshua B. Tenenbaum, William T. Freeman

CVPR 2018 / Paper / Project Page

Learning to See Physics via Visual De-animation

Jiajun Wu, Erika Lu, Pushmeet Kohli, William T. Freeman, Joshua B. Tenenbaum

NeurIPS 2017 / Paper / Project Page / MIT News

Spotlight Presentation

MarrNet: 3D Shape Reconstruction via 2.5D Sketches

Jiajun Wu*, Yifan Wang*, Tianfan Xue, Xingyuan Sun, William T. Freeman, Joshua B. Tenenbaum

NeurIPS 2017 / Paper / Project Page

Self-Supervised Intrinsic Image Decomposition

Michael Janner, Jiajun Wu, Tejas D. Kulkarni, Ilker Yildirim, Joshua B. Tenenbaum

NeurIPS 2017 / Paper / Project Page

Best Paper Award at the New England Computer Vision Workshop

Raster-to-Vector: Revisiting Floorplan Transformation

Chen Liu, Jiajun Wu, Pushmeet Kohli, Yasutaka Furukawa

ICCV 2017 / Paper / Project Page

Neural Scene De-rendering

Jiajun Wu, Joshua B. Tenenbaum, Pushmeet Kohli

CVPR 2017 / Paper / Project Page

Physics 101: Learning Physical Object Properties from Unlabeled Videos

Jiajun Wu, Joseph J. Lim, Hongyi Zhang, Joshua B. Tenenbaum, William T. Freeman

BMVC 2016 / Paper / Project Page

A Comparative Evaluation of Approximate Probabilistic Simulation and Deep Neural Networks as Accounts of Human Physical Scene Understanding

Renqiao Zhang*, Jiajun Wu*, Chengkai Zhang, William T. Freeman, Joshua B. Tenenbaum

CogSci 2016 / Paper / Project Page

Oral Presentation

Galileo: Perceiving Physical Object Properties by Integrating a Physics Engine with Deep Learning

Jiajun Wu*, Ilker Yildirim*, Joseph J. Lim, William T. Freeman, Joshua B. Tenenbaum

NeurIPS 2015 / Paper / Project Page / Demo / MIT News / VICE / WIRED / Engadget / Popular Science / The Boston Globe / PCWorld / TechRadar

Multi-Modal Perception

VLIC: Vision-Language Models As Perceptual Judges for Human-Aligned Image Compression

Kyle Sargent, Ruiqi Gao, Philipp Henzler, Charles Herrmann, Aleksander Holynski, Li Fei-Fei, Jiajun Wu, Jason Y. Zhang

CVPR 2026 / Paper / Project Page

Neuro-Symbolic Decoding of Neural Activity

Yanchen Wang*, Joy Hsu*, Ehsan Adeli†, Jiajun Wu†

ICLR 2026 / Paper / Project Page

X-Capture: An Open-Source Portable Device for Multi-Sensory Learning

Samuel Clarke, Suzannah Wistreich, Yanjie Ze, Jiajun Wu

ICCV 2025 / Paper / Project Page

Hearing Anything Anywhere

Mason Wang*, Ryosuke Sawata*, Samuel Clarke, Ruohan Gao, Shangzhe Wu, Jiajun Wu

CVPR 2024 / Paper / Project Page

ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understanding

Le Xue, Ning Yu, Shu Zhang, Junnan Li, Roberto Martin-Martin, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, Silvio Savarese

CVPR 2024 / Paper / Project Page

DiffSound: Differentiable Modal Sound Rendering and Inverse Rendering for Diverse Inference Tasks

Xutong Jin*, Chenxi Xu*, Ruohan Gao, Jiajun Wu, Guoping Wang, Sheng Li

SIGGRAPH 2024 / Paper / Project Page

Whodunnit? Inferring What Happened from Multimodal Evidence

Sarah A. Wu*, Erik Brockbank*, Hannah Cha, Jan-Philipp Fränken, Emily Jin, Zhuoyi Huang, Weiyu Liu, Ruohan Zhang, Jiajun Wu, Tobias Gerstenberg

CogSci 2024 / Paper

SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensing

Zhecheng Wang, Rajanie Prabha*, Tianyuan Huang*, Jiajun Wu, Ram Rajagopal

AAAI 2024 / Paper

SoundCam: A Dataset for Finding Humans Using Room Acoustics

Mason Wang, Samuel Clarke, Jui-Hsien Wang, Ruohan Gao, Jiajun Wu

NeurIPS 2023 Datasets and Benchmarks / Paper / Project Page

Holistic Evaluation of Text-to-Image Models

NeurIPS 2023 Datasets and Benchmarks / Paper / Project Page

Spotlight Presentation

NOIR: Neural Signal Operated Intelligent Robots for Everyday Activities

Ruohan Zhang*, Sharon Lee*, Minjune Hwang*, Ayano Hiranaka*, Chen Wang, Wensi Ai, Jin Jie Ryan Tan, Shreya Gupta, Yilun Hao, Gabrael Levine, Ruohan Gao, Anthony Norcia, Li Fei-Fei, Jiajun Wu

CoRL 2023 / Paper / Project Page

RealImpact: A Dataset of Impact Sound Fields for Real Objects

Samuel Clarke, Ruohan Gao, Mason Wang, Mark Rau, Julia Xu, Jui-Hsien Wang, Doug L. James, Jiajun Wu

CVPR 2023 / Paper / Project Page

Highlight

The ObjectFolder Benchmark: Multisensory Learning with Neural and Real Objects

Ruohan Gao*, Yiming Dou*, Hao Li*, Tanmay Agarwal, Jeannette Bohg, Yunzhu Li, Li Fei-Fei, Jiajun Wu

CVPR 2023 / Paper / Project Page

ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understanding

Le Xue, Mingfei Gao, Chen Xing, Roberto Martin-Martin, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, Silvio Savarese

CVPR 2023 / Paper / Project Page

An Extensible Multi-Modal Multi-Task Object Dataset with Materials

Trevor Standley, Ruohan Gao, Dawn Chen, Jiajun Wu, Silvio Savarese

ICLR 2023 / Paper / Project Page

Sonicverse: A Multisensory Simulation Platform for Embodied Household Agents that See and Hear

Ruohan Gao*, Hao Li*, Gokul Dharan, Zhuzhu Wang, Chengshu Li, Fei Xia, Silvio Savarese, Li Fei-Fei, Jiajun Wu

ICRA 2023 / Paper / Project Page

See, Hear, and Feel: Smart Sensory Fusion for Robotic Manipulation

Hao Li*, Yizhi Zhang*, Junzhe Zhu, Shaoxiong Wang, Michelle A. Lee, Huazhe Xu, Edward H. Adelson, Li Fei-Fei, Ruohan Gao†, Jiajun Wu†

CoRL 2022 / Paper / Project Page

MOMA-LRG: Language-Refined Graphs for Multi-Object Multi-Actor Activity Parsing

Zelun Luo, Zane Durante*, Linden Li*, Wanze Xie, Ruochen Liu, Emily Jin, Zhuoyi Huang, Lun Yu Li, Jiajun Wu, Juan Carlos Niebles, Ehsan Adeli, Li Fei-Fei

NeurIPS 2022 Datasets and Benchmarks / Paper / Project Page

ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer

Ruohan Gao*, Zilin Si*, Yen-Yu Chang*, Samuel Clarke, Jeannette Bohg, Li Fei-Fei, Wenzhen Yuan, Jiajun Wu

CVPR 2022 / Paper / Project Page

Revisiting the "Video" in Video-Language Understanding

Shyamal Buch, Cristobal Eyzaguirre, Adrien Gaidon, Jiajun Wu, Li Fei-Fei, Juan Carlos Niebles

CVPR 2022 / Paper / Project Page

Oral Presentation

ObjectFolder: A Dataset of Objects with Implicit Visual, Auditory, and Tactile Representations

Ruohan Gao, Yen-Yu Chang*, Shivani Mall*, Li Fei-Fei, Jiajun Wu

CoRL 2021 / Paper / Project Page

DiffImpact: Differentiable Rendering and Identification of Impact Sounds

Samuel Clarke, Negin Heravi, Mark Rau, Ruohan Gao, Jiajun Wu, Doug L. James, Jeannette Bohg

CoRL 2021 / Paper / Project Page

Oral Presentation

Learning Temporal Dynamics from Cycles in Narrated Video

Dave Epstein, Jiajun Wu, Cordelia Schmid, Chen Sun

ICCV 2021 / Paper / Project Page

On the Opportunities and Risks of Foundation Models

Center for Research on Foundation Models

arXiv 2021 / Paper / Project Page

Deep Audio Priors Emerge from Harmonic Convolutional Networks

Zhoutong Zhang, Yunyun Wang, Chuang Gan, Jiajun Wu, Joshua B. Tenenbaum, Antonio Torralba, William T. Freeman

ICLR 2020 / Paper / Project Page

Look, Listen, and Act: Towards Audio-Visual Embodied Navigation

Chuang Gan*, Yiwei Zhang*, Jiajun Wu, Boqing Gong, Joshua B. Tenenbaum

ICRA 2020 / Paper / Project Page

A Computational Model for Combinatorial Generalization in Physical Perception from Sound

Yunyun Wang, Chuang Gan, Max H. Siegel, Zhoutong Zhang, Jiajun Wu, Joshua B. Tenenbaum

CCN 2019 / Paper

3D Shape Perception from Monocular Vision, Touch, and Shape Priors

Shaoxiong Wang*, Jiajun Wu*, Xingyuan Sun, Wenzhen Yuan, William T. Freeman, Joshua B. Tenenbaum, Edward H. Adelson

IROS 2018 / Paper / Project Page

Learning Sight from Sound: Ambient Sound Provides Supervision for Visual Learning

Andrew Owens, Jiajun Wu, Josh H. McDermott, William T. Freeman, Antonio Torralba

IJCV 2018 / Paper / Project Page

Attention Clusters: Purely Attention Based Local Feature Integration for Video Classification

Xiang Long, Chuang Gan, Gerard de Melo, Jiajun Wu, Xiao Liu, Shilei Wen

CVPR 2018 / Paper

Shape and Material from Sound

Zhoutong Zhang, Qiujia Li, Zhengjia Huang, Jiajun Wu, Joshua B. Tenenbaum, William T. Freeman

NeurIPS 2017 / Paper / Project Page

Spotlight Presentation

Generative Modeling of Audible Shapes for Object Perception

Zhoutong Zhang*, Jiajun Wu*, Qiujia Li, Zhengjia Huang, James Traer, Josh H. McDermott, Joshua B. Tenenbaum, William T. Freeman

ICCV 2017 / Paper / Project Page

Deep Multi-Modal Image Correspondence Learning

Chen Liu, Jiajun Wu, Pushmeet Kohli, Yasutaka Furukawa

arXiv 2016 / Paper / Project Page

Visual Generation

Choreographing a World of Dynamic Objects

Yanzhe Lyu*, Chen Geng*, Karthik Dharmarajan, Yunzhi Zhang, Hadi Alzayer, Shangzhe Wu, Jiajun Wu

CVPR 2026 / Paper / Project Page

WonderZoom: Multi-Scale 3D World Generation

Jin Cao*, Hong-Xing Yu*, Jiajun Wu

CVPR 2026 / Paper / Project Page

PerpetualWonder: Long-Horizon Action-Conditioned 4D Scene Generation

Jiahao Zhan*, Zizhang Li*, Hong-Xing Yu, Jiajun Wu

CVPR 2026 / Paper / Project Page

Highlight

Coupled Diffusion Sampling for Training-Free Multi-View Image Editing

Hadi Alzayer, Yunzhi Zhang, Chen Geng, Jia-Bin Huang, Jiajun Wu

CVPR 2026 / Paper / Project Page

Talk2Move: Reinforcement Learning for Text-Instructed Object-Level Geometric Transformation in Scenes

Jing Tan, Zhaoyang Zhang, Yantao Shen, Jiarui Cai, Shuo Yang, Jiajun Wu, Wei Xia, Zhuowen Tu, Stefano Soatto

CVPR 2026 / Paper / Project Page

Highlight

Product of Experts for Visual Generation

Yunzhi Zhang, Carson Murtuza-Lanier, Zizhang Li, Yilun Du, Jiajun Wu

ICLR 2026 / Paper / Project Page

GenFusion: Feed-forward Human Performance Capture via Progressive Canonical Space Updates

Youngjoong Kwon, Yao He*, Heejung Choi*, Chen Geng, Zhengmao Liu, Jiajun Wu, Ehsan Adeli

ICLR 2026 / Paper / Project Page

Deep Sketch-Based 3D Modeling: A Survey

Alberto Tono, Jiajun Wu, Gordon Wetzstein, Iro Armeni, Martin Fischer, Hariharan Subramonyam, James Landay

CGF 2026 / Paper / Project Page

ZeroHSI: Zero-Shot 4D Human-Scene Interaction by Video Generation

Hongjie Li*, Hong-Xing Yu*, Jiaman Li, Jiajun Wu

3DV 2026 / Paper / Project Page

3D-Generalist: Vision-Language-Action Models for Crafting 3D Worlds

3DV 2026 / Paper / Project Page

WonderPlay: Dynamic 3D Scene Generation from a Single Image and Actions

Zizhang Li*, Hong-Xing Yu*, Wei Liu, Yin Yang, Charles Herrmann, Gordon Wetzstein, Jiajun Wu

ICCV 2025 / Paper / Project Page

Highlight

WorldScore: Unified Evaluation Benchmark for World Generation

Haoyi Duan*, Hong-Xing Yu*, Sirui Chen, Li Fei-Fei, Jiajun Wu

ICCV 2025 / Paper / Project Page

Flow to the Mode: Mode-Seeking Diffusion Autoencoders for State-of-the-Art Image Tokenization

Kyle Sargent, Kyle Hsu, Li Fei-Fei, Justin Johnson, Jiajun Wu

ICCV 2025 / Paper / Project Page

The Scene Language: Representing Scenes with Programs, Words, and Embeddings

Yunzhi Zhang, Zizhang Li, Matt Zhou, Shangzhe Wu, Jiajun Wu

CVPR 2025 / Paper / Project Page

Highlight

WonderWorld: Interactive 3D Scene Generation from a Single Image

Hong-Xing Yu*, Haoyi Duan*, Charles Herrmann, William T. Freeman, Jiajun Wu

CVPR 2025 / Paper / Project Page

Highlight

LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Models

Fan-Yun Sun*, Weiyu Liu*, Siyi Gu, Dylan Lim, Goutam Bhat, Federico Tombari, Manling Li, Nick Haber, Jiajun Wu

CVPR 2025 / Paper / Project Page

Diffusion Self-Distillation for Zero-Shot Customized Image Generation

Shengqu Cai, Eric Ryan Chan, Yunzhi Zhang, Leonidas J. Guibas, Jiajun Wu, Gordon Wetzstein

CVPR 2025 / Paper / Project Page

Range, not Independence, Drives Modularity in Biologically Inspired Representations

Will Dorrell*, Kyle Hsu*, Luke Hollingsworth†, Jin Hwa Lee†, Jiajun Wu, Chelsea Finn, Peter E. Latham, Tim Behrens, James C. R. Whittington

ICLR 2025 / Paper / Project Page

PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generation

Tianyuan Zhang, Hong-Xing Yu, Rundi Wu, Brandon Y. Feng, Changxi Zheng, Noah Snavely, Jiajun Wu, William T. Freeman

ECCV 2024 / Paper / Project Page

Oral Presentation

Controllable Human-Object Interaction Synthesis

Jiaman Li, Alexander Clegg, Roozbeh Mottaghi, Jiajun Wu, Xaiver Puig†, C. Karen Liu†

ECCV 2024 / Paper / Project Page

Oral Presentation

WonderJourney: Going from Anywhere to Everywhere

Hong-Xing Yu, Haoyi Duan, Junhwa Hur, Kyle Sargent, Michael Rubinstein, William T. Freeman, Forrester Cole, Deqing Sun, Noah Snavely, Jiajun Wu, Charles Herrmann

CVPR 2024 / Paper / Project Page

ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Real Image

Kyle Sargent, Zizhang Li, Tanmay Shah, Charles Herrmann, Hong-Xing Yu, Yunzhi Zhang, Eric Ryan Chan, Dmitry Lagun, Li Fei-Fei, Deqing Sun, Jiajun Wu

CVPR 2024 / Paper / Project Page

Tripod: Three Complementary Inductive Biases for Disentangled Representation Learning

Kyle Hsu*, Jubayer Ibn Hamid*, Kaylee Burns, Chelsea Finn, Jiajun Wu

ICML 2024 / Paper / Project Page

Language-Informed Visual Concept Learning

Sharon Lee*, Yunzhi Zhang*, Shangzhe Wu, Jiajun Wu

ICLR 2024 / Paper / Project Page

Patched Denoising Diffusion Models For High-Resolution Image Synthesis

Zheng Ding*, Mengqi Zhang*, Jiajun Wu, Zhuowen Tu

ICLR 2024 / Paper / Project Page

RoboCraft: Learning to See, Simulate, and Shape Elasto-Plastic Objects in 3D with Graph Networks

Haochen Shi*, Huazhe Xu*, Zhiao Huang, Yunzhu Li, Jiajun Wu

IJRR 2024 / Paper / Project Page

Partial-View Object View Synthesis via Filtering Inversion

Fan-Yun Sun*, Jonathan Tremblay, Valts Blukis, Kevin Lin, Danfei Xu, Boris Ivanovic, Peter Karkus, Stan Birchfield, Dieter Fox, Ruohan Zhang, Yunzhu Li, Jiajun Wu, Marco Pavone, Nick Haber

3DV 2024 / Paper / Project Page

Spotlight Presentation

Disentanglement via Latent Quantization

Kyle Hsu, Will Dorrell, James C. R. Whittington, Jiajun Wu, Chelsea Finn

NeurIPS 2023 / Paper / Project Page

VQ3D: Learning a 3D-Aware Generative Model on ImageNet

Kyle Sargent, Jing Yu Koh, Han Zhang, Huiwen Chang, Charles Herrmann, Pratul Srinivasan, Jiajun Wu, Deqing Sun

ICCV 2023 / Paper / Project Page

Best Paper Award Finalist

Rendering Humans from Object-Occluded Monocular Videos

Tiange Xiang, Adam Sun, Jiajun Wu, Ehsan Adeli, Li Fei-Fei

ICCV 2023 / Paper / Project Page

Seeing a Rose in Five Thousand Ways

Yunzhi Zhang, Shangzhe Wu, Noah Snavely, Jiajun Wu

CVPR 2023 / Paper / Project Page

3D Neural Field Generation using Triplane Diffusion

J. Ryan Shue*, Eric Ryan Chan*, Ryan Po*, Zachary Ankner*, Jiajun Wu, Gordon Wetzstein

CVPR 2023 / Paper / Project Page

Putting People in Their Place: Affordance-Aware Human Insertion into Scenes

Sumith Kulal, Tim Brooks, Alex Aiken, Jiajun Wu, Jimei Yang, Jingwan Lu, Alexei A. Efros, Krishna Kumar Singh

CVPR 2023 / Paper / Project Page

Physically Plausible Animation of Human Upper Body from a Single Image

Ziyuan Huang*, Zhengping Zhou*, Yung-Yu Chuang, Jiajun Wu, C. Karen Liu

WACV 2023 / Paper

Scene Synthesis from Human Motion

Sifan Ye*, Yixing Wang*, Jiaman Li, Dennis Park, C. Karen Liu, Huazhe Xu†, Jiajun Wu†

SIGGRAPH Asia 2022 / Paper / Project Page

Video Extrapolation in Space and Time

Yunzhi Zhang, Jiajun Wu

ECCV 2022 / Paper / Project Page

SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations

Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, Stefano Ermon

ICLR 2022 / Paper / Project Page

3D Shape Generation and Completion Through Point-Voxel Diffusion

Linqi Zhou, Yilun Du, Jiajun Wu

ICCV 2021 / Paper / Project Page

Oral Presentation

Neural Radiance Flow for 4D View Synthesis and Video Processing

Yilun Du, Yinan Zhang, Hong-Xing Yu, Joshua B. Tenenbaum, Jiajun Wu

ICCV 2021 / Paper / Project Page

pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis

Eric Ryan Chan*, Marco Monteiro*, Petr Kellnhofer, Jiajun Wu, Gordon Wetzstein

CVPR 2021 / Paper / Project Page

Oral Presentation

KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control

Tomas Jakab, Richard Tucker, Ameesh Makadia, Jiajun Wu, Noah Snavely, Angjoo Kanazawa

CVPR 2021 / Paper / Project Page

Oral Presentation

De-rendering the World's Revolutionary Artefacts

Shangzhe Wu, Ameesh Makadia, Jiajun Wu, Noah Snavely, Richard Tucker, Angjoo Kanazawa

CVPR 2021 / Paper / Project Page

Repopulating Street Scenes

Yifan Wang, Andrew Liu, Richard Tucker, Jiajun Wu, Brian Curless, Steven M. Seitz, Noah Snavely

CVPR 2021 / Paper / Project Page

Probabilistic Video Prediction from Noisy Data with a Posterior Confidence

Yunbo Wang, Jiajun Wu, Mingsheng Long, Joshua B. Tenenbaum

CVPR 2020 / Paper / Project Page

End-to-End Optimization of Scene Layout

Andrew Luo, Zhoutong Zhang, Jiajun Wu, Joshua B. Tenenbaum

CVPR 2020 / Paper / Project Page

Oral Presentation

Learning Generative Models of 3D Structures

Siddhartha Chaudhuri, Daniel Ritchie, Jiajun Wu, Kai Xu, Hao Zhang

Eurographics STAR 2020 / Paper

Video Enhancement with Task-Oriented Flow

Tianfan Xue, Baian Chen, Jiajun Wu, Donglai Wei, William T. Freeman

IJCV 2019 / Paper / Project Page

Visual Dynamics: Stochastic Future Generation via Layered Cross Convolutional Networks

Tianfan Xue*, Jiajun Wu*, Katherine L. Bouman, William T. Freeman

TPAMI 2019 / Paper / Project Page

Visual Object Networks: Image Generation with Disentangled 3D Representation

Jun-Yan Zhu, Zhoutong Zhang, Chengkai Zhang, Jiajun Wu, Antonio Torralba, Joshua B. Tenenbaum, William T. Freeman

NeurIPS 2018 / Paper / Project Page

Learning Shape Priors for Single-View 3D Completion and Reconstruction

Jiajun Wu*, Chengkai Zhang*, Xiuming Zhang, Zhoutong Zhang, William T. Freeman, Joshua B. Tenenbaum

ECCV 2018 / Paper / Project Page

MoSculp: Interactive Visualization of Shape and Time

Xiuming Zhang, Tali Dekel, Tianfan Xue, Andrew Owens, Qiurui He, Jiajun Wu, Stefanie Mueller, William T. Freeman

UIST 2018 / Paper / Project Page / MIT News / Forbes / BBC

Synthesizing 3D Shapes via Modeling Multi-View Depth Maps and Silhouettes with Deep Generative Networks

Amir A. Soltani, Haibin Huang, Jiajun Wu, Tejas D. Kulkarni, Joshua B. Tenenbaum

CVPR 2017 / Paper / Project Page

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Jiajun Wu*, Chengkai Zhang*, Tianfan Xue, William T. Freeman, Joshua B. Tenenbaum

NeurIPS 2016 / Paper / Project Page / MIT Technology Review

Visual Reasoning

Building Intelligent Agents with Neuro-Symbolic Concepts

Jiayuan Mao, Joshua B. Tenenbaum, Jiajun Wu

CACM 2026 / Paper

Self-Consistency for LLM-Based Motion Trajectory Generation and Verification

Jiaju Ma, R. Kenny Jones, Jiajun Wu, Maneesh Agrawala

CVPR 2026 / Paper / Project Page

Spatial Mental Modeling from Limited Views

ICLR 2026 / Paper / Project Page

Theory of Space: Can Foundation Models Construct Spatial Beliefs Through Active Exploration?

Pingyue Zhang*, Zihan Huang*, Yue Wang*, Jieyu Zhang*, Letian Xue, Zihan Wang, Qineng Wang, Keshigeyan Chandrasegaran, Ruohan Zhang, Yejin Choi, Ranjay Krishna, Jiajun Wu, Li Fei-Fei, Manling Li

ICLR 2026 / Paper / Project Page

A Tool Bottleneck Framework for Clinically-Informed and Interpretable Medical Image Understanding

Christina Liu*, Alan Wang*, Joy Hsu, Jiajun Wu, Ehsan Adeli

MIDL 2026 / Paper / Project Page

From Programs to Poses: Factored Real-World Scene Generation via Learned Program Libraries

Joy Hsu, Emily Jin, Jiajun Wu, Niloy J. Mitra

NeurIPS 2025 / Paper / Project Page

VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents

NeurIPS 2025 / Paper / Project Page

Re-thinking Temporal Search for Long-Form Video Understanding

Jinhui Ye*, Zihan Wang*, Haosen Sun, Keshigeyan Chandrasegaran, Zane Durante, Cristobal Eyzaguirre, Yonatan Bisk, Juan Carlos Niebles, Ehsan Adeli, Li Fei-Fei, Jiajun Wu, Manling Li

CVPR 2025 / Paper / Project Page

Understanding Complexity in VideoQA via Visual Program Generation

Cristobal Eyzaguirre, Igor Vasiljevic, Achal Dave, Jiajun Wu, Rares Ambrus, Thomas Kollar, Juan Carlos Niebles, Pavel Tokmakov

ICML 2025 / Paper / Project Page

Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas

Shiqi Chen, Tongyao Zhu, Ruochen Zhou, Jinghan Zhang, Siyang Gao, Juan Carlos Niebles, Mor Geva, Junxian He, Jiajun Wu, Manling Li

ICML 2025 / Paper / Project Page

What Makes a Maze Look Like a Maze?

Joy Hsu, Jiayuan Mao, Joshua B. Tenenbaum, Noah D. Goodman, Jiajun Wu

ICLR 2025 / Paper / Project Page

Predicate Hierarchies Improve Few-Shot State Classification

Emily Jin*, Joy Hsu*, Jiajun Wu

ICLR 2025 / Paper / Project Page

Visually Descriptive Language Model for Vector Graphics Reasoning

Zhenhailong Wang, Joy Hsu, Xingyao Wang, Kuan-Hao Huang, Manling Li, Jiajun Wu, Heng Ji

TMLR 2025 / Paper / Project Page

HourVideo: 1-Hour Video-Language Understanding

Keshigeyan Chandrasegaran, Agrim Gupta, Lea M. Hadzic, Taran Kota, Jimming He, Cristobal Eyzaguirre, Zane Durante, Manling Li, Jiajun Wu, Li Fei-Fei

NeurIPS 2024 Datasets and Benchmarks / Paper / Project Page

Streaming Detection of Queried Event Start

Cristobal Eyzaguirre, Eric Tang, Shyamal Buch, Adrien Gaidon, Jiajun Wu, Juan Carlos Niebles

NeurIPS 2024 Datasets and Benchmarks / Paper / Project Page

Naturally Supervised 3D Visual Grounding with Language-Regularized Concept Learners

Chun Feng*, Joy Hsu*, Weiyu Liu, Jiajun Wu

CVPR 2024 / Paper / Project Page

Learning Planning Abstractions from Language

Weiyu Liu*, Geng Chen*, Joy Hsu, Jiayuan Mao†, Jiajun Wu†

ICLR 2024 / Paper / Project Page

What's Left? Concept Grounding with Logic-Enhanced Foundation Models

Joy Hsu*, Jiayuan Mao*, Joshua B. Tenenbaum, Jiajun Wu

NeurIPS 2023 / Paper / Project Page

Compositional Diffusion-Based Continuous Constraint Solvers

Zhutian Yang, Jiayuan Mao, Yilun Du, Jiajun Wu, Joshua B. Tenenbaum, Tomas Lozano-Perez, Leslie P. Kaelbling

CoRL 2023 / Paper / Project Page

Editing Motion Graphics Videos via Program Induction and Transformation

Sharon Zhang, Jiaju Ma, Jiajun Wu, Daniel Ritchie, Maneesh Agrawala

TOG (SIGGRAPH Asia) 2023 / Paper / Project Page

Tree-Structured Shading Decomposition

Chen Geng*, Hong-Xing Yu*, Sharon Zhang, Maneesh Agrawala, Jiajun Wu

ICCV 2023 / Paper / Project Page

NS3D: Neuro-Symbolic Grounding of 3D Objects and Relations

Joy Hsu, Jiayuan Mao, Jiajun Wu

CVPR 2023 / Paper / Project Page

Motion Question Answering via Modular Motion Programs

Mark Endo*, Joy Hsu*, Jiaman Li, Jiajun Wu

ICML 2023 / Paper / Project Page

Modeling Dynamic Environments with Scene Graph Memory

Andrey Kurenkov, Michael Lingelbach, Tanmay Agarwal, Emily Jin, Chengshu Li, Ruohan Zhang, Li Fei-Fei, Jiajun Wu, Silvio Savarese, Roberto Martin-Martin

ICML 2023 / Paper / Project Page

Programmatically Grounded, Compositionally Generalizable Robotic Manipulation

Renhao Wang, Jiayuan Mao, Joy Hsu, Hang Zhao, Jiajun Wu, Yang Gao

ICLR 2023 / Paper / Project Page

Notable Top 25%

Neurosymbolic Models for Computer Graphics

Daniel Ritchie, Paul Guerrero, R. Kenny Jones, Niloy J. Mitra, Adriana Schulz, Karl D. D. Willis, Jiajun Wu

Eurographics STAR 2023 / Paper

Task-Driven Graph Attention for Hierarchical Relational Object Navigation

Michael Lingelbach, Chengshu Li, Minjune Hwang, Andrey Kurenkov, Alan Lou, Roberto Martin-Martin, Ruohan Zhang, Li Fei-Fei, Jiajun Wu

ICRA 2023 / Paper

DisCo: Improving Compositional Generalization in Visual Reasoning through Distribution Coverage

Joy Hsu, Jiayuan Mao, Jiajun Wu

TMLR 2023 / Paper / Project Page

Learning Rational Subgoals from Demonstrations and Instructions

Zhezheng Luo*, Jiayuan Mao*, Jiajun Wu, Tomas Lozano-Perez, Joshua B. Tenenbaum, Leslie P. Kaelbling

AAAI 2023 / Paper / Project Page

Unsupervised Learning of Shape Programs with Repeatable Implicit Parts

Boyang Deng*, Sumith Kulal*, Zhengyang Dong, Congyue Deng, Yonglong Tian, Jiajun Wu

NeurIPS 2022 / Paper / Project Page

E-MAPP: Efficient Multi-Agent Reinforcement Learning with Parallel Program Guidance

Can Chang, Ni Mu, Jiajun Wu, Ling Pan, Huazhe Xu

NeurIPS 2022 / Paper / Project Page

Spotlight Presentation

IKEA-Manual: Seeing Shape Assembly Step by Step

Ruocheng Wang, Yunzhi Zhang, Jiayuan Mao, Ran Zhang, Chin-Yi Cheng, Jiajun Wu

NeurIPS 2022 Datasets and Benchmarks / Paper / Project Page

CLEVRER-Humans: Describing Physical and Causal Events the Human Way

Jiayuan Mao*, Xuelin Yang*, Xikun Zhang, Noah D. Goodman, Jiajun Wu

NeurIPS 2022 Datasets and Benchmarks / Paper / Project Page

Geoclidean: Few-Shot Generalization in Euclidean Geometry

Joy Hsu, Jiajun Wu, Noah D. Goodman

NeurIPS 2022 Datasets and Benchmarks / Paper / Project Page

Sparse and Local Networks for Hypergraph Reasoning

Guangxuan Xiao, Leslie P. Kaelbling, Jiajun Wu, Jiayuan Mao

LoG 2022 / Paper / Project Page

Translating a Visual LEGO Manual to a Machine-Executable Plan

Ruocheng Wang, Yunzhi Zhang, Jiayuan Mao, Chin-Yi Cheng, Jiajun Wu

ECCV 2022 / Paper / Project Page

Programmatic Concept Learning for Human Motion Description and Synthesis

Sumith Kulal*, Jiayuan Mao*, Alex Aiken†, Jiajun Wu†

CVPR 2022 / Paper / Project Page

Grammar-Based Grounded Lexicon Learning

Jiayuan Mao, Haoyue Shi, Jiajun Wu, Roger P. Levy, Joshua B. Tenenbaum

NeurIPS 2021 / Paper / Project Page

Language-Mediated, Object-Centric Representation Learning

Ruocheng Wang*, Jiayuan Mao*, Samuel J. Gershman†, Jiajun Wu†

ACL 2021 Findings / Paper / Project Page

Temporal and Object Quantification Networks

Jiayuan Mao*, Zhezheng Luo*, Chuang Gan, Joshua B. Tenenbaum, Jiajun Wu, Leslie P. Kaelbling, Tomer D. Ullman

IJCAI 2021 / Paper / Project Page

Hierarchical Motion Understanding via Motion Programs

Sumith Kulal*, Jiayuan Mao*, Alex Aiken, Jiajun Wu

CVPR 2021 / Paper / Project Page

Grounding Physical Concepts of Objects and Events Through Dynamic Visual Reasoning

Zhenfang Chen, Jiayuan Mao, Jiajun Wu, Kwan-Yee K. Wong, Joshua B. Tenenbaum, Chuang Gan

ICLR 2021 / Paper / Project Page

Augmenting Policy Learning with Routines Discovered from a Single Demonstration

Zelin Zhao, Chuang Gan, Jiajun Wu, Xiaoxiao Guo, Joshua B. Tenenbaum

AAAI 2021 / Paper / Project Page

Multi-Plane Program Induction with 3D Box Priors

Yikai Li*, Jiayuan Mao*, Xiuming Zhang, William T. Freeman, Joshua B. Tenenbaum, Noah Snavely, Jiajun Wu

NeurIPS 2020 / Paper / Project Page

Object-Centric Diagnosis of Visual Reasoning

Jianwei Yang, Jiayuan Mao, Jiajun Wu, Devi Parikh, David D. Cox, Joshua B. Tenenbaum, Chuang Gan

arXiv 2020 / Paper

Perspective Plane Program Induction from a Single Image

Yikai Li*, Jiayuan Mao*, Xiuming Zhang, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu

CVPR 2020 / Paper / Project Page

CLEVRER: Collision Events for Video Representation and Reasoning

Kexin Yi*, Chuang Gan*, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, Joshua B. Tenenbaum

ICLR 2020 / Paper / Project Page / MIT Technology Review

Spotlight Presentation

Visual Concept-Metaconcept Learning

Chi Han*, Jiayuan Mao*, Chuang Gan, Joshua B. Tenenbaum, Jiajun Wu

NeurIPS 2019 / Paper / Project Page

Program-Guided Image Manipulators

Jiayuan Mao*, Xiuming Zhang*, Yikai Li, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu

ICCV 2019 / Paper / Project Page

Neurally-Guided Structure Inference

Sidi Lu*, Jiayuan Mao*, Joshua B. Tenenbaum, Jiajun Wu

ICML 2019 / Paper / Project Page

The Neuro-Symbolic Concept Learner:
Interpreting Scenes, Words, and Sentences from Natural Supervision

Jiayuan Mao, Chuang Gan, Pushmeet Kohli, Joshua B. Tenenbaum, Jiajun Wu

ICLR 2019 / Paper / Project Page / MIT News / MIT Technology Review

Oral Presentation

Learning to Infer and Execute 3D Shape Programs

Yonglong Tian, Andrew Luo, Xingyuan Sun, Kevin Ellis, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu

ICLR 2019 / Paper / Project Page

Learning to Describe Scenes with Programs

Yunchao Liu, Zheng Wu, Daniel Ritchie, William T. Freeman, Joshua B. Tenenbaum, Jiajun Wu

ICLR 2019 / Paper

Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding

Kexin Yi*, Jiajun Wu*, Chuang Gan, Antonio Torralba, Pushmeet Kohli, Joshua B. Tenenbaum

NeurIPS 2018 / Paper / Project Page

Spotlight Presentation

Robotics and Embodied AI

ROSETTA: Constructing Code-Based Reward from Unconstrained Language Preference

Sanjana Srivastava*, Kangrui Wang*, Yung-Chieh Chan*, Tianyuan Dai, Manling Li, Ruohan Zhang, Mengdi Xu, Jiajun Wu, Li Fei-Fei

ICLR 2026 / Paper / Project Page

MoMaGen: Generating Demonstrations Under Soft and Hard Constraints for Multi-Step Bimanual Mobile Manipulation

Chengshu Li*, Mengdi Xu*, Arpit Bahety*, Hang Yin*, Yunfan Jiang, Huang Huang, Josiah Wong, Sujay Garlanka, Cem Gokmen, Ruohan Zhang, Weiyu Liu, Jiajun Wu, Roberto Martin-Martin, Li Fei-Fei

ICLR 2026 / Paper / Project Page

Learning Composable Skills by Discovering Spatial and Temporal Structure with Foundation Models

Neil Nie, Wenlong Huang, Jiayuan Mao, Li Fei-Fei, Weiyu Liu†, Jiajun Wu†

ICRA 2026 / Paper / Project Page

TWIST2: Scalable, Portable, and Holistic Humanoid Data Collection System

Yanjie Ze, Siheng Zhao, Weizhuo Wang, Angjoo Kanazawa†, Rocky Duan†, Pieter Abbeel†, Guanya Shi†, Jiajun Wu†, C. Karen Liu†

ICRA 2026 / Paper / Project Page

IMPASTO: Integrating Model-Based Planning with Learned Dynamics Models for Robotic Oil Painting Reproduction

Yingke Wang, Hao Li, Yifeng Zhu, Hong-Xing Yu, Ken Goldberg, Li Fei-Fei, Jiajun Wu, Yunzhu Li, Ruohan Zhang

ICRA 2026 / Paper / Project Page

Dream2Flow: Bridging Video Generation and Open-World Manipulation with 3D Object Flow

Karthik Dharmarajan, Wenlong Huang, Jiajun Wu, Li Fei-Fei†, Ruohan Zhang†

ICRA 2026 / Paper / Project Page

Retargeting Matters: General Motion Retargeting for Humanoid Motion Tracking

Joao Pedro Araujo*, Yanjie Ze*, Pei Xu*, Jiajun Wu†, C. Karen Liu†

ICRA 2026 / Paper / Project Page

Phys2Real: Fusing VLM Priors with Interactive Online Adaptation for Uncertainty-Aware Sim-to-Real Manipulation

Maggie Wang, Stephen Tian, Aiden Swann, Ola Shorinwa, Jiajun Wu, Mac Schwager

ICRA 2026 / Paper / Project Page

10 Open Challenges Steering the Future of Vision-Language-Action Models

Soujanya Poria, Navonil Majumder, Chia-Yu Hung, Amir Ali Bagherzadeh, Chuan Li, Kenneth Kwok, Ziwei Wang, Cheston Tan, Jiajun Wu, David Hsu

AAAI 2026 Senior Member Presentation / Paper

A Review of Learning-Based Dynamics Models for Robotic Manipulation

Bo Ai, Stephen Tian, Haochen Shi, Yixuan Wang, Tobias Pfaff, Cheston Tan, Henrik I. Christensen, Hao Su, Jiajun Wu, Yunzhu Li

Science Robotics 2025 / Paper

Model-Based Policy Adaptation for Closed-Loop End-to-End Autonomous Driving

Haohong Lin, Yunzhi Zhang, Wenhao Ding, Jiajun Wu, Ding Zhao

NeurIPS 2025 / Paper

DexSkin: High-Coverage Conformable Robotic Skin for Learning Contact-Rich Manipulation

Suzannah Wistreich*, Baiyu Shi*, Stephen Tian*, Samuel Clarke, Michael Nath, Chengyi Xu, Zhenan Bao, Jiajun Wu

CoRL 2025 / Paper / Project Page

Oral Presentation

TWIST: Teleoperated Whole-Body Imitation System

Yanjie Ze*, Zixuan Chen*, Joao Pedro Araujo, Zi-ang Cao, Xue Bin Peng, Jiajun Wu†, C. Karen Liu†

CoRL 2025 / Paper / Project Page

BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activities

Yunfan Jiang, Ruohan Zhang, Josiah Wong, Chen Wang, Yanjie Ze, Hang Yin, Cem Gokmen, Shuran Song, Jiajun Wu, Li Fei-Fei

CoRL 2025 / Paper / Project Page

Generalizable Humanoid Manipulation with 3D Diffusion Policies

Yanjie Ze, Zixuan Chen, Wenhao Wang, Tianyi Chen, Xialin He, Ying Yuan, Xue Bin Peng, Jiajun Wu

IROS 2025 / Paper / Project Page

Learning Smooth Humanoid Locomotion Through Lipschitz-Constrained Policies

Zixuan Chen*, Xialin He*, Yen-Jen Wang*, Qiayuan Liao, Yanjie Ze, Zhongyu Li, S. Shankar Sastry, Jiajun Wu, Koushil Sreenath, Saurabh Gupta, Xue Bin Peng

IROS 2025 / Paper / Project Page

UAD: Unsupervised Affordance Distillation for Generalization in Robotic Manipulation

Yihe Tang, Wenlong Huang, Yingke Wang, Chengshu Li, Roy Yuan, Ruohan Zhang, Jiajun Wu, Li Fei-Fei

ICRA 2025 / Paper / Project Page

Best Paper Award Finalist

FactorSim: Generative Simulation via Factorized Representation

Fan-Yun Sun, S. I. Harini, Angela Yi, Yihan Zhou, Alex Zook, Jonathan Tremblay, Logan Cross, Jiajun Wu, Nick Haber

NeurIPS 2024 / Paper / Project Page

Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making

Manling Li*, Shiyu Zhao*, Qineng Wang*, Kangrui Wang*, Yu Zhou*, Sanjana Srivastava, Cem Gokmen, Tony Lee, Li Erran Li, Ruohan Zhang, Weiyu Liu, Percy Liang, Li Fei-Fei, Jiayuan Mao, Jiajun Wu

NeurIPS 2024 Datasets and Benchmarks / Paper / Project Page

Oral Presentation

Learning Compositional Behaviors from Demonstration and Language

Weiyu Liu*, Neil Nie*, Ruohan Zhang, Jiayuan Mao†, Jiajun Wu†

CoRL 2024 / Paper / Project Page

View-Invariant Policy Learning via Zero-Shot Novel View Synthesis

Stephen Tian, Blake Wulfe, Kyle Sargent, Katherine Liu, Sergey Zakharov, Vitor Guizilini, Jiajun Wu

CoRL 2024 / Paper / Project Page

Automated Creation of Digital Cousins for Robust Policy Learning

Tianyuan Dai*, Josiah Wong*, Yunfan Jiang, Chen Wang, Cem Gokmen, Ruohan Zhang, Jiajun Wu, Li Fei-Fei

CoRL 2024 / Paper / Project Page

D3Fields: Dynamic 3D Descriptor Fields for Zero-Shot Generalizable Rearrangement

Yixuan Wang*, Mingtong Zhang*, Zhuoran Li*, Tarik Kelestemur, Katherine Driggs-Campbell, Jiajun Wu, Li Fei-Fei, Yunzhu Li

CoRL 2024 / Paper / Project Page

Oral Presentation

TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction

Yunfan Jiang, Chen Wang, Ruohan Zhang, Jiajun Wu, Li Fei-Fei

CoRL 2024 / Paper / Project Page

Evaluating Real-World Robot Manipulation Policies in Simulation

CoRL 2024 / Paper / Project Page

Foundation Models in Robotics: Applications, Challenges, and the Future

IJRR 2024 / Paper / Project Page

BEHAVIOR Vision Suite: Customizable Dataset Generation via Simulation

CVPR 2024 / Paper / Project Page

Highlight

Holodeck: Language Guided Generation of 3D Embodied AI Environments

CVPR 2024 / Paper / Project Page

RoboPack: Learning Tactile-Informed Dynamics Models for Dense Packing

Bo Ai*, Stephen Tian*, Haochen Shi, Yixuan Wang, Cheston Tan, Yunzhu Li, Jiajun Wu

RSS 2024 / Paper / Project Page

DROID: A Large-Scale In-the-Wild Robot Manipulation Dataset

DROID Dataset Team

RSS 2024 / Paper / Project Page

D5RL: Diverse Datasets for Data-Driven Deep Reinforcement Learning

Rafael Rafailov*, Kyle Hatch*, Anikait Singh, Laura Smith, Aviral Kumar, Ilya Kostrikov, Philippe Hansen-Estruch, Victor Kolev, Philip J. Ball, Jiajun Wu, Chelsea Finn, Sergey Levine

RLC 2024 / Paper

RoboCraft: Learning to See, Simulate, and Shape Elasto-Plastic Objects in 3D with Graph Networks

Haochen Shi*, Huazhe Xu*, Zhiao Huang, Yunzhu Li, Jiajun Wu

IJRR 2024 / Paper / Project Page

Learning to Design 3D Printable Adaptations on Everyday Objects for Robot Manipulation

Michelle Guo, Ziang Liu, Stephen Tian, Zhaoming Xie, Jiajun Wu, C. Karen Liu

ICRA 2024 / Paper / Project Page

Open X-Embodiment: Robotic Learning Datasets and RT-X Models

Open X-Embodiment Collaboration

ICRA 2024 / Paper / Project Page

Best Paper Award

Efficient Imitation Learning with Conservative World Models