Maxim Rakhuba — accepted AI-conference papers

2026

LoRA meets Riemannion: Muon Optimizer for Parametrization-independent Low-Rank Adapters

Vladimir Bogachev, Vladimir Aletov, Alexander Molozhavenko, Denis Bobkov, Vera Soboleva, Aibek Alanov +1

ICLR 2026poster

This work presents a novel, fully Riemannian framework for Low-Rank Adaptation (LoRA) that geometrically treats low-rank adapters by optimizing them directly on the fixed-rank manifold. This formulation eliminates the parametrization ambiguity present in standard Euclidean optimizers. Our framework…

Cited by 0 Source ↗Scholar ↗

2026

Matrix-Free Two-to-Infinity and One-to-Two Norms Estimation

Askar Tsyganov, Evgeny Frolov, Sergey Samsonov, Maxim Rakhuba

AAAI 2026technical

In this paper, we propose new randomized algorithms for estimating the two-to-infinity and one-to-two norms in a matrix-free setting, using only matrix-vector multiplications. Our methods are based on appropriate modifications of Hutchinson

Cited by 0 Source ↗PDF ↗Scholar ↗

2026

OrthoFuse: Training-free Riemannian Fusion of Orthogonal Style-Concept Adapters for Diffusion Models

Ali Aliev, Kamil Garifullin, Nikolay Yudin, Vera Soboleva, Alexander Molozhavenko, Ivan Oseledets +2

CVPR 2026

In a rapidly growing field of model training there is a constant practical interest in parameter-efficient fine-tuning and various techniques that use a small amount of training data to adapt the model to a narrow task. However, there is an open question: how to combine several adapters tuned for di

Cited by 0 Source ↗code ↗Scholar ↗

2025

COALA: Numerically Stable and Efficient Framework for Context-Aware Low-Rank Approximation

Uliana Parkina, Maxim Rakhuba

NeurIPS 2025poster

Recent studies suggest that context-aware low-rank approximation is a useful tool for compression and fine-tuning of modern large-scale neural networks. In this type of approximation, a norm is weighted by a matrix of input activations, significantly improving metrics over the unweighted case. Neve…

Cited by 0 Source ↗Scholar ↗

2025

Knowledge Graph Completion with Mixed Geometry Tensor Factorization

Viacheslav Yusupov, Maxim Rakhuba, Evgeny Frolov

AISTATS 2025poster

In this paper, we propose a new geometric approach for knowledge graph completion via low rank tensor approximation. We augment a pretrained and well-established Euclidean model based on a Tucker tensor decomposition with a novel hyperbolic interaction term. This correction enables more nuanced cap…

Cited by 0 Source ↗code ↗Scholar ↗

2025

ProcrustesGPT: Compressing LLMs with Structured Matrices and Orthogonal Transformations

Ekaterina Grishina, Mikhail Gorbunov, Maxim Rakhuba

ACL 2025finding

Large language models (LLMs) demonstrate impressive results in natural language processing tasks but require a significant amount of computational and memory resources. Structured matrix representations are a promising way for reducing the number of parameters of these models. However, it seems unre…

Cited by 0 Source ↗PDF ↗code ↗Scholar ↗

2024

Group and Shuffle: Efficient Structured Orthogonal Parametrization

Mikhail Gorbunov, Kolya Yudin, Vera Soboleva, Aibek Alanov, Alexey Naumov, Maxim Rakhuba

NeurIPS 2024poster

The increasing size of neural networks has led to a growing demand for methods of efficient finetuning. Recently, an orthogonal finetuning paradigm was introduced that uses orthogonal matrices for adapting the weights of a pretrained model. In this paper, we introduce a new class of structured matri…

Cited by 10 Source ↗PDF ↗Scholar ↗

2024

Tight and Efficient Upper Bound on Spectral Norm of Convolutional Layers

Ekaterina Grishina*, Mikhail Gorbunov, Maxim Rakhuba

ECCV 2024poster

"Controlling the spectral norm of the Jacobian matrix, which is related to the convolution operation, has been shown to improve generalization, training stability and robustness in CNNs. Existing methods for computing the norm either tend to overestimate it or their performance may deteriorate quick…

Cited by 1 Source ↗PDF ↗code ↗Scholar ↗

2024

Training a Tucker Model With Shared Factors: a Riemannian Optimization Approach

Ivan Peshekhonov, Aleksey Arzhantsev, Maxim Rakhuba

AISTATS 2024poster

Factorization of a matrix into a product of two rectangular factors, is a classic tool in various machine learning applications. Tensor factorizations generalize this concept to more than two dimensions. In applications, where some of the tensor dimensions have the same size or encode the same objec…

Cited by 3 Source ↗PDF ↗code ↗Scholar ↗

2022

Towards Practical Control of Singular Values of Convolutional Layers

Alexandra Senderovich, Ekaterina Bulatova, Anton Obukhov, Maxim Rakhuba

NeurIPS 2022accept

In general, convolutional neural networks (CNNs) are easy to train, but their essential properties, such as generalization error and adversarial robustness, are hard to control. Recent research demonstrated that singular values of convolutional layers significantly affect such elusive properties and…

Cited by 12 Source ↗PDF ↗code ↗Scholar ↗

2021

Cherry-Picking Gradients: Learning Low-Rank Embeddings of Visual Data via Differentiable Cross-Approximation

Mikhail Usvyatsov, Anastasia Makarova, Rafael Ballester-Ripoll, Maxim Rakhuba, Andreas Krause, Konrad Schindler

ICCV 2021poster

We propose an end-to-end trainable framework that processes large-scale visual data tensors by looking at a fraction of their entries only. Our method combines a neural network encoder with a tensor train decomposition to learn a low-rank latent encoding, coupled with cross-approximation (CA) to lea…

Cited by 6 PDF ↗code ↗Scholar ↗

2021

Spectral Tensor Train Parameterization of Deep Learning Layers

Anton Obukhov, Maxim Rakhuba, Alexander Liniger, Zhiwu Huang, Stamatios Georgoulis, Dengxin Dai +1

AISTATS 2021poster

We study low-rank parameterizations of weight matrices with embedded spectral properties in the Deep Learning context. The low-rank property leads to parameter efficiency and permits taking computational shortcuts when computing mappings. Spectral properties are often subject to constraints in optim…

Cited by 13 Source ↗PDF ↗code ↗Scholar ↗

2020

T-Basis: a Compact Representation for Neural Networks

Anton Obukhov, Maxim Rakhuba, Stamatios Georgoulis, Menelaos Kanakis, Dengxin Dai, Luc Van Gool

ICML 2020poster

We introduce T-Basis, a novel concept for a compact representation of a set of tensors, each of an arbitrary shape, which is often seen in Neural Networks. Each of the tensors in the set is modeled using Tensor Rings, though the concept applies to other Tensor Networks. Owing its name to the T-shape…

Cited by 30 Source ↗PDF ↗code ↗Scholar ↗