Muon: The Most Underrated Breakthrough in LLMs
bycloud · 19:29 · Watch on YouTube
Muon is a matrix-aware optimizer, proposed by Keller Jordan in December 2024, that orthogonalizes momentum updates so transformer weight matrices learn across more independent directions. The video argues it is the fi...