Vladimir Bogachev

LoRA meets Riemannion: Muon Optimizer for Parametrization-independent Low-Rank Adapters

This work presents a novel, fully Riemannian framework for Low-Rank Adaptation (LoRA) that geometrically treats low-rank adapters by optimizing them directly on the fixed-rank manifold. This formulation eliminates the parametrization ambiguity present in standard Euclidean optimizers. Our framework…