Moritz Thoma

Advancing SVD-based LLM Compression via Layer-Wise Error Model Search

Low-rank SVD-based compression offers a powerful strategy to reduce the computational costs of Large language models (LLMs); however, existing methods commonly encounter two recurring obstacles: (i) global rank allocation, where uncalibrated error proxies fail to account for complex error propagatio…