2026
RMT-KD: RANDOM MATRIX THEORETIC CAUSAL KNOWLEDGE DISTILLATION
ICASSP 2026poster
Large deep learning models such as BERT and ResNet achieve state-of-the-art performance but are costly to deploy at the edge due to their size and compute demands. We present RMT-KD, a compression method that leverages Random Matrix Theory (RMT) for knowledge distillation to iteratively reduce netwo…