Hi, I am Lean Wang, a PhD student at the Institute of Computational Linguistics (ICL), Peking University. I am in the LANCO group, supervised by Prof. Xu Sun.

I am now dedicated to creating a better LLM! I once studied mechanistic interpretation, yet I am now focusing on improving the foundation language model.

Education

• PhD Student, Peking University, 2023 - Now
• B.S. in Intelligence Science and Technology (Summa Cum Laude), Peking University, 2019 - 2023.

Internship

• DeepSeek-AI, Research Intern, Jan. 2024 - Now

Preprints

• Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts
  Lean Wang, Huazuo Gao, Chenggang Zhao, Xu Sun, Damai Dai
  Adopted by DeepSeek V3 [arxiv]

Selected Publications

(# denotes Equal Contribution)

• Towards Codable Watermarking for Injecting Multi-bits Information to LLMs
  Lean Wang#, Wenkai Yang#, Deli Chen#, Hao Zhou, Yankai Lin, Fandong Meng, Jie Zhou, Xu Sun
  ICLR 2024 [arxiv, code]

• Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning
  Lean Wang, Lei Li, Damai Dai, Deli Chen, Hao Zhou, Fandong Meng, Jie Zhou, Xu Sun
  EMNLP 2023 Best Long Paper Award [arxiv, code]

Academic Service

• Teaching Assistant:
  • Probability Theory and Statistics (A) (2023 Spring, PKU)
• As a Reviewer
  • ARR Reviewer

Awards

• Best Long Paper Award, EMNLP, 2023