Jing Yu Koh (@kohjingyu) / X

Jing Yu Koh

1,546 posts

Jing Yu Koh

@kohjingyu

ML PhD @CarnegieMellon. Previously: computer use and multimodal at MSL TBD Lab, text-to-image generation @GoogleAI. Opinions my own. 🇸🇬

PIT

Joined July 2009

Pinned
Jing Yu Koh
@kohjingyu
Jun 3
Computer use agents are slow and brittle. The fix isn’t just stronger models, but also deploying them as multi-agent systems. MACU is a general Multi-Agent Computer Use framework that consistently lifts success rates by 3.4-25.5% and is up to 1.5x faster on long-horizon tasks.🧵
00:00
28K
Jing Yu Koh
@kohjingyu
Jan 28, 2025
Adding \vspace{-0.00352} beneath a figure caption so that 2 extra lines get moved up, section 3.5 flows onto the previous page, and the whole paper collapses into exactly 8 pages
122K
Jing Yu Koh
@kohjingyu
Jan 10, 2025
One of my friends in undergrad wrote a linear optimization program to maximize nutrition per dollar for their protein shake. It ended up returning that the optimal solution was just 100% peanut butter
Andrew Carr 🤸
@andrew_n_carr
Jan 10, 2025
Over break I calculated the calorical, macro, and micro optimal food storage with minimal ingredients Oats: 30-35 lbs (13.6 - 15.9 kg) Quinoa: 20-25 lbs (9 - 11.3 kg) Lentils: 70-80 lbs (31.8 - 36.3 kg) Ground Flax Seeds: 10-12 lbs (4.5 - 5.4 kg) Sunflower Seeds: 8-10 lbs (3.6
455K
Jing Yu Koh
@kohjingyu
Dec 26, 2024
When I was leaving Google to start my PhD, several people told me that having access to less resources in academia would actually encourage more creative work. I didn’t really understand it at the time: can’t you arbitrarily impose the same constraints on yourself to be efficient
DeepSeek
@deepseek_ai
Dec 26, 2024
🚀 Introducing DeepSeek-V3! Biggest leap forward yet: ⚡ 60 tokens/second (3x faster than V2!) 💪 Enhanced capabilities 🛠 API compatibility intact 🌍 Fully open-source models & papers 🐋 1/n
GIF
130K
Jing Yu Koh
@kohjingyu
May 16, 2024
Last week when presenting Parti (parti.research.google) at ICLR, I explained at least 20 times how I felt about autoregressive text-to-image generation models vs. diffusion models. So this is my take: The major benefit of autoregressive image generation models is that they
Greg Brockman
@gdb
May 15, 2024
A GPT-4o generated image — so much to explore with GPT-4o's image generation capabilities alone. Team is working hard to bring those to the world.
227K
Jing Yu Koh
@kohjingyu
Sep 27, 2023
One piece of advice that I frequently give to prospective ML PhD students is to work at a tech company or startup (ideally doing research) for at least a year before diving into a PhD program. This is invaluable if you've never worked full time before. Some reasons why: 1. You
96K
Jing Yu Koh
@kohjingyu
Feb 1, 2023
Can we ground LLMs to enable them to process and produce image-text data? Introducing 🧀FROMAGe: a model that efficiently bootstraps frozen LLMs for images. This enables LLMs to process image-text data and produce coherent image-text outputs. arxiv.org/abs/2301.13823 🧵👇
150K
Jing Yu Koh
@kohjingyu
Sep 29, 2022
It's a good day for text-to-image... 1) Make-A-Video (makeavideo.studio): text-to-video w/o paired examples 2) Phenaki (phenaki.video): text-to-video (up to a few mins long (!)), prompts can change over time. 3) DreamFusion (dreamfusionpaper.github.io): text-to-3D
Jing Yu Koh
@kohjingyu
Jun 20, 2024
LLM agents have demonstrated promise in their ability to automate computer tasks, but face challenges with multi-step reasoning and planning. Towards addressing this, we propose an inference-time tree search algorithm for LLM agents to explicitly perform exploration and
00:00
125K
Jing Yu Koh
@kohjingyu
May 30, 2023
Excited to share GILL🐟 — a method to fuse LLMs with image encoder and decoder models! GILL is one of the first approaches capable of conditioning on interleaved image-text inputs to generate image-text outputs. Paper: arxiv.org/abs/2305.17216 Website: jykoh.com/gill
128K
Jing Yu Koh
@kohjingyu
Oct 15, 2022
PhD application season is upon us! In the spirit of paying it forward, I'm making my statement of purpose (CS / ML PhDs) public: jykoh.com/jykoh_sop.pdf I learnt a ton from reading other people's SOPs, and it was a super helpful exercise to write (and rewrite) mine.
Jing Yu Koh
@kohjingyu
Sep 15, 2022
It's that time of the year again to remind everyone that ML research is not about making the world a better place. It is about publishing as many NeurIPS papers as you can.
Jing Yu Koh
@kohjingyu
Jul 3, 2024
Exciting personal update: I'm taking a leave of absence from my PhD to join the GenAI team at @AIatMeta. I'm super excited to continue working with my advisors @rsalakhu and @dan_fried, and join forces with the amazing Llama team to build multimodal LLM agents! 🦙+🤖
Russ Salakhutdinov
@rsalakhu
Jul 3, 2024
I am very excited to start working with GenAI team at @Meta, focusing on multimodal LLM agents, joining together with my amazing CMU colleagues Jing Yu Koh @kohjingyu and Daniel Fried @dan_fried!
54K
Jing Yu Koh
@kohjingyu
Jan 2, 2023
Research codebases get really cluttered over time. What are the nanoGPTs (github.com/karpathy/nanoG… by @karpathy) of other models/tasks?
83K