4 accepted papers
Supervised classifier learning has a theoretical optimum — Neural Collapse (NC) — yet standard training does not reach it in practice. We trace this failure to a geometric limitation: cross-entropy is invariant to joint rescaling of features and weights, leaving radial degrees of freedom unconstrain…
Sparse autoencoders (SAEs) have emerged as a promising method for interpreting neural network representations by decomposing activations into sparse combinations of dictionary atoms. However, SAEs assume that features combine additively through linear reconstruction, an assumption that cannot captur…
What do different contrastive learning (CL) losses actually optimize for? Although multiple CL methods have demonstrated remarkable representation learning capabilities, the differences in their inner workings remain largely opaque. In this work, we analyse several CL families and prove that, under…
Despite the considerable research output on Multimodal Learning for Affect-related tasks, most of the current methods are very complex in terms of the number of trainable parameters, and thus do not constitute effective solutions for real-life applications. In this work we try to alleviate this gap…