{"@attributes":{"version":"2.0"},"channel":{"title":"@varunneal","link":"https:\/\/varunneal.github.io","description":"Last 10 notes on @varunneal","generator":"Quartz -- quartz.jzhao.xyz","item":[{"title":"Essays","link":"https:\/\/varunneal.github.io\/essays\/","guid":"https:\/\/varunneal.github.io\/essays\/","description":"Personal catalog of essays, aphorisms, arrows.","pubDate":"Fri, 03 Jul 2026 08:09:21 GMT"},{"title":"Projects","link":"https:\/\/varunneal.github.io\/projects","guid":"https:\/\/varunneal.github.io\/projects","description":"(I\u2019ve been working on \u2026) Machine Learning Modded NanoGPT - an LLM speedrunning competition where I have set several records 1.","pubDate":"Fri, 03 Jul 2026 08:09:21 GMT"},{"title":"welcome","link":"https:\/\/varunneal.github.io\/","guid":"https:\/\/varunneal.github.io\/","description":"I\u2019m Varun. I am a Machine Learning researcher and engineer.","pubDate":"Fri, 03 Jul 2026 08:09:21 GMT"},{"title":"Loss-Free MoE Routing","link":"https:\/\/varunneal.github.io\/essays\/loss-free-moe","guid":"https:\/\/varunneal.github.io\/essays\/loss-free-moe","description":"Overview: I posit that the the two essential constraints on the MoE router are load balancing and rowwise orthogonality.","pubDate":"Tue, 30 Jun 2026 00:00:00 GMT"},{"title":"Heterogeneous Iteration for Spectral Powers","link":"https:\/\/varunneal.github.io\/essays\/spectral-powers","guid":"https:\/\/varunneal.github.io\/essays\/spectral-powers","description":"Extremely fast approximations of matrix powers U\u03a3^p V^T via a basis of polynomial iterates.","pubDate":"Fri, 22 May 2026 00:00:00 GMT"},{"title":"[Translation] MoE Odyssey (MoE\u73af\u6e38\u8bb0)","link":"https:\/\/varunneal.github.io\/essays\/moe","guid":"https:\/\/varunneal.github.io\/essays\/moe","description":"Author: Jianlin Su (\u82cf\u5251\u6797) | kexue.fm This is a compilation and AI-generated translation of Jianlin Su\u2019s wonderful series of blog posts \u201cMoE\u73af\u6e38\u8bb0\u201d.","pubDate":"Sun, 17 May 2026 00:00:00 GMT"},{"title":"Verses on Training","link":"https:\/\/varunneal.github.io\/essays\/verses-on-training","guid":"https:\/\/varunneal.github.io\/essays\/verses-on-training","description":"Arrows of Wisdom On training (Buddha): 1 My mind is without leaks, free, long cultivated, and well-trained.","pubDate":"Fri, 01 May 2026 00:00:00 GMT"},{"title":"Muon in Modded NanoGPT","link":"https:\/\/varunneal.github.io\/essays\/muon","guid":"https:\/\/varunneal.github.io\/essays\/muon","description":"How the Muon optimizer is used in Modded NanoGPT NanoGPT. Key techniques: adaptive Muon, cautious weight decay batch scheduling, faster orthogonalization (polar express). Key concepts: gradient variance\/curvature estimation, learning rate tuning, GPU optimization.","pubDate":"Mon, 24 Nov 2025 00:00:00 GMT"},{"title":"Hephaestus","link":"https:\/\/varunneal.github.io\/essays\/hephaestus","guid":"https:\/\/varunneal.github.io\/essays\/hephaestus","description":"THREE thousand years ago, Homer strained his memory and recalled what was said about that great inventor Hephaestus\u2026 May 2025 Meta pays $14bn for a majority stake in Scale AI.","pubDate":"Fri, 31 Oct 2025 00:00:00 GMT"},{"title":"Blood from a Stone","link":"https:\/\/varunneal.github.io\/essays\/blood-from-a-stone","guid":"https:\/\/varunneal.github.io\/essays\/blood-from-a-stone","description":"This post is a follow-up to two peaks. (1) p-values killed the cat Science has killed magic.","pubDate":"Thu, 22 May 2025 00:00:00 GMT"}]}}