Ichiro Hashimoto

Directional Convergence, Benign Overfitting of Gradient Descent in leaky ReLU two-layer Neural Networks

In this paper, we provide sufficient conditions of benign overfitting of fixed width leaky ReLU two-layer neural network classifiers trained on mixture data via gradient descent. Our results are derived by establishing directional convergence of the network parameters and classification error bound…