Hello! I am a Research Scientist at Ai2 on the PRIOR team, based in Seattle, WA.

My research interests lie in computer vision and machine learning, with a focus on their applications to real-world problems. In particular, I study multimodal foundation models grounded in vision and language. Prior to joining Ai2, I received my Ph.D. in Computer Science and Engineering from Seoul National University where I was advised by Prof. Gunhee Kim.

Employment

work Allen Institute for AI, Seattle, WA, USA ^{2024.08 - Current}

• Research Scientist on the PRIOR team

work Allen Institute for AI, Seattle, WA, USA ^{2023.01 - 2024.07}

• Postdoctoral Young Investigator on the PRIOR team

work Allen Institute for AI, Seattle, WA, USA ^{2023.01 - Current}

• Research Intern on the PRIOR team

Education

school Seoul National University (SNU), Seoul, South Korea ^{2017.03 - 2023.02}

• Ph.D. in Computer Science and Engineering
• Advisor: Prof. Gunhee Kim
• Thesis: Improving Efficiency in Large-Scale Self-Supervised Video Representation Learning

school Seoul National University (SNU), Seoul, South Korea ^{2010.03 - 2017.02}

• B.S. in Computer Science and Engineering (minor in Statistics)

Publications

• Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding
  Christopher Clark^♥*, Jieyu Zhang^♥*, Zixian Ma^♥*, Jae Sung Park^♥, Mohammadreza Salehi^♥, Rohun Tripathi^♥, Sangho Lee^♥, Zhongzheng Ren, Chris Dongjoo Kim, Yinuo Yang, Vincent Shao, Yue Yang, Weikai Huang, Ziqi Gao, Taira Anderson, Jianrui Zhang, Jitesh Jain, George Stoica, Winson Han, Ali Farhadi and Ranjay Krishna^♥ (^♥: core contributor, *: equal contribution)
  Technical Report 2025
  [paper] [demo] [blog post] [code] [models] [data]

• SAGE: Training Smart Any-Horizon Agents for Long Video Reasoning with Reinforcement Learning
  Jitesh Jain, Jialuo Li, Zixian Ma, Jieyu Zhang, Chris Dongjoo Kim, Sangho Lee, Rohun Tripathi, Tanmay Gupta, Christopher Clark^ and Humphrey Shi^ (^: equal advising)
  Preprint 2025
  [paper] [project page] [code] [models&data]

• MolmoAct: Action Reasoning Models that can Reason in Space
  Jason Lee^♥*, Jiafei Duan^♥*, Haoquan Fang^♥*, Yuquan Deng^♥, Shuo Liu^♥, Boyang Li^♥, Bohan Fang^♥, Jieyu Zhang^♥, Yi Ru Wang^♥, Sangho Lee, Winson Han, Wilbert Pumacay, Angelica Wu, Rose Hendrix^♥, Karen Farley, Eli VanderBilt, Ali Farhadi, Dieter Fox^♥ and Ranjay Krishna^♥ (^♥: core contributor, *: equal contribution)
  Technical Report 2025
  CoRL 2025 Workshop on Making Sense of Data in Robotics (Best Paper Nominee)
  CoRL 2025 Workshop on Resource-Rational Robot Learning (Best Paper Runner-up)
  [paper] [blog post] [code] [models] [data]

• Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Multimodal Models
  Matt Deitke*, Christopher Clark*, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, Jiasen Lu, Taira Anderson, Erin Bransom, Kiana Ehsani, Huong Ngo, YenSung Chen, Ajay Patel, Mark Yatskar, Chris Callison-Burch, Andrew Head, Rose Hendrix, Favyen Bastani, Eli VanderBilt, Nathan Lambert, Yvonne Chou, Arnavi Chheda, Jenna Sparks, Sam Skjonsberg, Michael Schmitz, Aaron Sarnat, Byron Bischoff, Pete Walsh, Chris Newell, Piper Wolters, Tanmay Gupta, Kuo-Hao Zeng, Jon Borchardt, Dirk Groeneveld, Jen Dumas, Crystal Nam, Sophie Lebrecht, Caitlin Wittlif, Carissa Schoenick, Oscar Michel, Ranjay Krishna, Luca Weihs, Noah A. Smith, Hannaneh Hajishirzi, Ross Girshick, Ali Farhadi and Aniruddha Kembhavi (*: equal contribution)
  CVPR 2025 (Oral, 3.3%)
  Best Paper, Honorable Mention (top 7 out of 2,872 accepted papers)
  [paper] [demo] [blog post] [code] [models] [data]

• One Diffusion to Generate Them All
  Duong H. Le*, Tuan Pham*, Sangho Lee, Christopher Clark, Aniruddha Kembhavi, Stephan Mandt, Ranjay Krishna and Jiasen Lu (*: equal contribution)
  CVPR 2025
  [paper] [project page] [code]

• ReSpec: Relevance and Specificity Grounded Online Filtering for Learning on Video-Text Data Streams
  Chris Dongjoo Kim*, Jihwan Moon*, Sangwoo Moon, Heeseung Yun, Sihaeng Lee, Aniruddha Kembhavi, Soonyoung Lee, Gunhee Kim, Sangho Lee^ and Christopher Clark^ (*: equal contribution, ^: equal advising)
  CVPR 2025
  [paper] [code]

• Finding NeMo: Negative-mined Mosaic Augmentation for Referring Image Segmentation
  Seongsu Ha*, Chaeyun Kim*, Donghwa Kim*, Junho Lee, Sangho Lee and Joonseok Lee (*: equal contribution)
  ECCV 2024
  [paper] [project page] [code] [poster] [slides]

• Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action
  Jiasen Lu*, Christopher Clark*, Sangho Lee*, Zichen Zhang*, Savya Khosla, Ryan Marten, Derek Hoiem and Aniruddha Kembhavi (*: equal contribution)
  CVPR 2024 (highlight)
  [paper] [webpage]

• Can Language Models Laugh at YouTube Short-form Videos?
  Dayoon Ko, Sangho Lee and Gunhee Kim
  EMNLP 2023
  [paper] [code&dataset]

• ACAV100M: Automatic Curation of Large-Scale Datasets for Audio-Visual Video Representation Learning
  Sangho Lee*, Jiwan Chung*, Youngjae Yu, Gunhee Kim, Thomas Breuel, Gal Chechik and Yale Song (*: equal contribution)
  ICCV 2021
  CVPR 2021: The Third Workshop on Learning from Unlabeled Videos
  [paper] [code&dataset]

• Unsupervised Representation Learning via Neural Activation Coding
  Yookoon Park, Sangho Lee, Gunhee Kim and David Blei
  ICML 2021 (long talk)
  [paper] [poster] [slides] [code]

• Parameter Efficient Multimodal Transformers for Video Representation Learning
  Sangho Lee, Youngjae Yu, Gunhee Kim, Thomas Breuel, Jan Kautz and Yale Song
  ICLR 2021
  CVPR 2021: The Second Intertional Workshop on Large Scale Holistic Video Understanding
  [paper] [poster] [slides]

• Self-Supervised Learning of Compressed Video Representations
  Youngjae Yu*, Sangho Lee*, Gunhee Kim and Yale Song (*: equal contribution)
  ICLR 2021
  [paper] [poster] [slides]

• A Memory Network Approach for Story-based Temporal Summarization of 360° Videos
  Sangho Lee, Jinyoung Sung, Youngjae Yu and Gunhee Kim
  CVPR 2018
  ECCV 2018 Workshop on 360° Perception and Interaction
  [paper] [project page (poster/slides/bibtex)]

• A Deep Ranking Model for Spatio-temporal Highlight Detection from a 360° Video
  Youngjae Yu, Sangho Lee, Joonil Na, Jaeyoun Kang and Gunhee Kim
  AAAI 2018 (spotlight)
  [paper] [poster] [slides] [bibtex]

• A Read-Write Memory Network for Movie Story Understanding
  Seil Na, Sangho Lee, Jisung Kim and Gunhee Kim
  ICCV 2017
  ICCV 2017: The Joint Video and Language Understanding Workshop
  [paper] [code] [poster] [bibtex]

• Encoding Video and Label Priors for Multi-label Video Classification on YouTube-8M dataset
  Seil Na, Youngjae Yu, Sangho Lee, Jisung Kim and Gunhee Kim
  CVPR 2017 Workshop on YouTube-8M Large-Scale Video Understanding
  [paper] [code] [bibtex]

Awards

• Best Paper Awards (CoRL Workshop, 2025)
  • CoRL 2025 Workshop on Making Sense of Data in Robotics: Composition, Curation, and Interpretability at Scale
  • MolmoAct: Action Reasoning Models that can Reason in Space
  • Best Paper Nominee
• Best Paper Awards (CoRL Workshop, 2025)
  • CoRL 2025 Workshop on Resource-Rational Robot Learning
  • MolmoAct: Action Reasoning Models that can Reason in Space
  • Best Paper Runner-up
• Best Paper Awards (CVPR, 2025)
  • Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Multimodal Models
  • Best Paper, Honorable Mention
• Excellent Ph.D. Thesis Award (SNU CSE, 2023)
  • Selected as the best doctoral thesis by Department of Computer Science and Engineering, Seoul National University
• Naver Ph.D. Fellowship (Naver, 2021)
  • Awarded to outstanding graduate students in the field of Computer Science for their exceptional academic research
• Youlchon AI Star Fellowship (Youlchon Foundation, Nongshim Group, 2021)
  • An award for those who made distinguished research achievements in core AI fields
• MovieQA Challenge @ ICCV 2017 Workshop (ICCV Workshop, 2017)
  • ICCV 2017 Workshop on the Joining Video and Language Understanding
  • Ranked 2nd place
• Google Cloud & YouTube-8M Video Understanding Challenge (CVPR Workshop, 2017)
  • CVPR 2017 Workshop on YouTube-8M Large-Scale Video Understanding
  • Ranked 8th place out of 655 teams (Top 2%)