2025
Feature Averaging: An Implicit Bias of Gradient Descent Leading to Non-Robustness in Neural Networks
ICLR 2025poster
In this work, we investigate a particular implicit bias in gradient descent training, which we term “Feature Averaging,” and argue that it is one of the principal factors contributing to the non-robustness of deep neural networks. We show that, even when multiple discriminative features are present…