Uliana Parkina

COALA: Numerically Stable and Efficient Framework for Context-Aware Low-Rank Approximation

Recent studies suggest that context-aware low-rank approximation is a useful tool for compression and fine-tuning of modern large-scale neural networks. In this type of approximation, a norm is weighted by a matrix of input activations, significantly improving metrics over the unweighted case. Neve…