AIConfPaper
← Search

Maria Refinetti

7 accepted papers

2024

Forward Learning with Top-Down Feedback: Empirical and Analytical Characterization

Ravi Francesco Srinivasan, Francesca Mignacco, Martino Sorbaro, Maria Refinetti, Avi Cooper, Gabriel Kreiman +1
ICLR 2024poster

"Forward-only" algorithms, which train neural networks while avoiding a backward pass, have recently gained attention as a way of solving the biologically unrealistic aspects of backpropagation. Here, we first address compelling challenges related to the "forward-only" rules, which include reducing…

Cited by 21Source ↗PDF ↗Scholar ↗
2023

Neural networks trained with SGD learn distributions of increasing complexity

Maria Refinetti, Alessandro Ingrosso, Sebastian Goldt
ICML 2023poster

The uncanny ability of over-parameterised neural networks to generalise well has been explained using various "simplicity biases". These theories postulate that neural networks avoid overfitting by first fitting simple, linear classifiers before learning more complex, non-linear functions. Meanwhile…

Cited by 42Source ↗PDF ↗code ↗Scholar ↗
2022

Fluctuations, Bias, Variance & Ensemble of Learners: Exact Asymptotics for Convex Losses in High-Dimension

Bruno Loureiro, Cedric Gerbelot, Maria Refinetti, Gabriele Sicuro, Florent Krzakala
ICML 2022spotlight

From the sampling of data to the initialisation of parameters, randomness is ubiquitous in modern Machine Learning practice. Understanding the statistical fluctuations engendered by the different sources of randomness in prediction is therefore key to understanding robust generalisation. In this man…

Cited by 36Source ↗PDF ↗Scholar ↗
2022

The dynamics of representation learning in shallow, non-linear autoencoders

Maria Refinetti, Sebastian Goldt
ICML 2022spotlight

Autoencoders are the simplest neural network for unsupervised learning, and thus an ideal framework for studying feature learning. While a detailed understanding of the dynamics of linear autoencoders has recently been obtained, the study of non-linear autoencoders has been hindered by the technical…

Cited by 24Source ↗PDF ↗code ↗Scholar ↗
2021

Align, then memorise: the dynamics of learning with feedback alignment

Maria Refinetti, Stéphane D’Ascoli, Ruben Ohana, Sebastian Goldt
ICML 2021spotlight

Direct Feedback Alignment (DFA) is emerging as an efficient and biologically plausible alternative to backpropagation for training deep neural networks. Despite relying on random feedback weights for the backward pass, DFA successfully trains state-of-the-art models such as Transformers. On the othe…

Cited by 61Source ↗PDF ↗code ↗Scholar ↗
2021

Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeed

Maria Refinetti, Sebastian Goldt, Florent Krzakala, Lenka Zdeborova
ICML 2021spotlight

A recent series of theoretical works showed that the dynamics of neural networks with a certain initialisation are well-captured by kernel methods. Concurrent empirical work demonstrated that kernel methods can come close to the performance of neural networks on some image classification tasks. Thes…

Cited by 105Source ↗PDF ↗code ↗Scholar ↗
2020

Double Trouble in Double Descent: Bias and Variance(s) in the Lazy Regime

Stéphane D’Ascoli, Maria Refinetti, Giulio Biroli, Florent Krzakala
ICML 2020poster

Deep neural networks can achieve remarkable generalization performances while interpolating the training data. Rather than the U-curve emblematic of the bias-variance trade-off, their test error often follows a "double descent"—a mark of the beneficial role of overparametrization. In this work, we d…

Cited by 191Source ↗PDF ↗code ↗Scholar ↗