Bo Liu (@cranialxix) / X

Bo Liu

43 posts

Bo Liu

@cranialxix

Senior Research Scientist @AIatMeta FAIR | CS PhD @UTCompSci | Former Research Intern @DeepMind, @Nvidia

Mountain View, CA

cranial-xix.github.io

Joined January 2018

Pinned
Bo Liu
@cranialxix
Jul 25, 2024
How to design State Space Models (SSM) from principles? We propose to view SSM's recurrence as the per-step closed-form solution to an online learning problem. To this end, we present Longhorn, a novel SSM that achieves 1.8x better sampling efficiency against Mamba.
27K
Bo Liu
@cranialxix
Oct 23, 2025
Proud to be under Yuandong for the past year : ) Just realized Yd got deactivated before he could see my reply under his badge post: You reminded me why I chose this path: seeking the truth of natural/intelligence is the most beautiful and impactful thing we can do. Yuandong,
Yuandong Tian
@tydsh
Oct 23, 2025
Several of my team members + myself are impacted by this layoff today. Welcome to connect :)
115K
Bo Liu
@cranialxix
Aug 25, 2025
Replying to @agarwl_ and @AIatMeta
It was a short but very rewarding time working with you Rishabh. Missing those RL discussions. Wish you the best of luck on your next adventure :)
12K
Bo Liu
@cranialxix
Jul 25, 2024
Replying to @cranialxix
We release our code following the NanoGPT style for the convenience of the community. It contains the one-file implementation of: LLaMA, RetNet, GLA, RWKV, Mamba, and Longhorn, and uses the OpenWebText dataset. Code:
github.com
GitHub - Cranial-XIX/longhorn: Official PyTorch Implementation of the Longhorn Deep State Space...
Official PyTorch Implementation of the Longhorn Deep State Space Model - Cranial-XIX/longhorn
1.1K
Bo Liu
@cranialxix
Sep 24, 2024
RWKV-7'update is pretty similar to the Longhorn model's update (arxiv.org/pdf/2407.14207), which is derived explicitly from solving online associative recall in closed form. The household transform used in the RWKV-7, (diag(w) - a \alpha^\top \beta), stems from optimizing a
BlinkDL
@BlinkDL_AI
Sep 23, 2024
RWKV-7 "Goose" 🪿 preview rc2 => Peak RNN architecture?😃Will try to squeeze more performance for the final release. Preview code: github.com/BlinkDL/RWKV-L…
1.2K
Bo Liu
@cranialxix
Nov 26, 2024
One line of code for improved training by ensuring the update aligns with the gradient. Note that there is no need to tune hyperparameters; just use those from AdamW or Lion.
Kaizhao Liang
@KyleLiang5
Nov 26, 2024
TLDR: 1⃣ line modification, satisfaction (theoretically and empirically) guaranteed 😀😀😀 Core idea: 🚨Do not update if you are not sure 👨‍💻github.com/kyleliang919/C… 🤗huggingface.co/papers/2411.16… 📚arxiv.org/abs/2411.16085 @cranialxix @lqiang67 @Tim38463182
2.2K
Bo Liu
@cranialxix
Aug 14, 2024
Interested in the continual adaptation of large AI models? Join us by submitting your work to our NeurIPS workshop :) This is a great opportunity to engage with experts and advance the dialogue on how foundation models can be dynamically updated. Deadline is Sept 9th AoE.
Arslan Chaudhry
@arslan_mac
Aug 14, 2024
[1/4] Happy to announce that we are organizing a workshop on continuous development of foundation models at NeurIPS’24. Website: sites.google.com/corp/view/cont…
3K
Bo Liu
@cranialxix
Dec 11, 2024
For imitation learning in robotics: as cheap as behavioral cloning, as expressive as diffusion policy. From the original group that designed the rectified flow.
Xixi Hu
@XixiHu12
Dec 11, 2024
🚀 Excited to share AdaFlow at #NeurIPS2024! A fast, adaptive method for training robots to act with one-step efficiency—no distillation needed! 🌟 Come check out our poster! 📍 West Ballroom A-D #7309 📅 Today 11 am - 2 pm #MachineLearning #Robotics
2.8K
Bo Liu
@cranialxix
Oct 18, 2025
A simple idea to improve RL for text diffusion - replace ELBO (lower bound) with EUBO (upper bound) for negative reward traces : )
Chenyu (Monica) Wang
@ChenyuW64562111
Oct 18, 2025
Introducing SPG: Sandwiched Policy Gradient for Masked Diffusion Language Models We propose a new policy gradient algorithm, SPG, for diffusion large language models. SPG improves the accuracy over the previous state-of-the-art RL methods by 3.6% in GSM8K, 2.6% in MATH500, 18.4%
1.3K
Bo Liu
@cranialxix
Feb 4, 2025
If you are interested in learning/using flow/diffusion models, please check this thread from the original author of rectified flow (RF). It contains: 1. a tutorial blog (to quickly get a sense of what RF is and some interesting findings we had lately) 2. a codebase (a minimal
Qiang Liu
@lqiang67
Feb 3, 2025
🚀 New Rectified Flow materials (WIP)! 📖 Tutorials: rectifiedflow.github.io 💻 Code: github.com/lqiang67/recti… 📜 Notes: cs.utexas.edu/~lqiang/PDF/fl… Contributions from @RunlongLiao, @XixiHu12, @cranialxix, and many others! 🔥 Let us know your thoughts! 🚀
1.6K
Bo Liu
@cranialxix
Jul 25, 2024
Replying to @cranialxix
The HF paper page link:
Paper page - Longhorn: State Space Models are Amortized Online Learners
From huggingface.co
7K
Bo Liu
@cranialxix
Jul 25, 2024
Replying to @cranialxix
Unlike previous SSMs, Longhorn shares the same architecture as Mamba, except that we replace the Mamba SSM with the Longhorn's SSM. We believe this forms a fair comparison. Moreover, as Longhorn's update is derived, it does not require a separately parameterized forget gate.
727
Bo Liu
@cranialxix
Jul 26, 2024
Replying to @NobodyObviously
Great question! A major problem for more generalized approaches (including RNN like LSTM) is that these approaches are not in general parallelizable. On the other hand, we see that even a linear autoregressive decoder trained on CoT data can be turing complete (check
Maksym Andriushchenko @ ICML 🇰🇷
@maksym_andr
Sep 16, 2023
It's really surprising how far one can go with *linear* predictors in the autoregressive setting. Interesting theory and experiments on TinyStories: a linear model (with 162M params :-) ) can generate totally coherent text with few grammatical mistakes. arxiv.org/abs/2309.06979
291
Bo Liu
@cranialxix
Dec 10, 2023
Replying to @yifengzhu_ut
See you at poster #1414, 10:45AM Tue :)
138