Jan Hazla — accepted AI-conference papers

2022

A Johnson-Lindenstrauss Framework for Randomly Initialized CNNs

Ido Nachum, Jan Hazla, Michael Gastpar, Anatoly Khina

ICLR 2022poster

How does the geometric representation of a dataset change after the application of each randomly initialized layer of a neural network? The celebrated Johnson-Lindenstrauss lemma answers this question for linear fully-connected neural networks (FNNs), stating that the geometry is essentially preserv…

Cited by 11 Source ↗PDF ↗Scholar ↗

2022

An Initial Alignment between Neural Network and Target is Needed for Gradient Descent to Learn

Emmanuel Abbe, Elisabetta Cornacchia, Jan Hazla, Christopher Marquis

ICML 2022spotlight

This paper introduces the notion of “Initial Alignment” (INAL) between a neural network at initialization and a target function. It is proved that if a network and a Boolean target function do not have a noticeable INAL, then noisy gradient descent with normalized i.i.d. initialization will not lear…

Cited by 15 Source ↗PDF ↗Scholar ↗