Lizhi Peng

PACEAttention: Principled and Adaptive Feature Compression-Expansion Grounded in the Geometry of $\text{MCR}^2$

The maximal coding rate reduction ($\text{MCR}^2$) objective is proposed for learning low-dimensional subspace representations and for principled deep model design, where layer structures are derived by unrolling its optimization steps. However, existing methods motivated by this objective do not fu…