Andrzej Banburski-Fahey

Neural Collapse in Deep Homogeneous Classifiers and The Role of Weight Decay

Neural Collapse is a phenomenon recently discovered in deep classifiers where the last layer activations collapse onto their class means, while the means and last layer weights take on the structure of dual equiangular tight frames. In this paper we present results showing the role of weight decay i…