AIConfPaper
← Search

Frank Schneider

5 accepted papers

2025

Accelerating neural network training: An analysis of the AlgoPerf competition

Priya Kasimbeg, Frank Schneider, Runa Eschenhagen, Juhan Bae, Chandramouli Shama Sastry, Mark Saroufim +8
ICLR 2025poster

The goal of the AlgoPerf: Training Algorithms competition is to evaluate practical speed-ups in neural network training achieved solely by improving the underlying training algorithms. In the external tuning ruleset, submissions must provide workload-agnostic hyperparameter search spaces, while in t…

Cited by 0Source ↗PDF ↗Scholar ↗
2023

Kronecker-Factored Approximate Curvature for Modern Neural Network Architectures

Runa Eschenhagen, Alexander Immer, Richard E Turner, Frank Schneider, Philipp Hennig
NeurIPS 2023spotlight

The core components of many modern neural network architectures, such as transformers, convolutional, or graph neural networks, can be expressed as linear layers with *weight-sharing*. Kronecker-Factored Approximate Curvature (K-FAC), a second-order optimisation method, has shown promise to speed up…

Cited by 55Source ↗PDF ↗Scholar ↗
2021

Cockpit: A Practical Debugging Tool for the Training of Deep Neural Networks

Frank Schneider, Felix Dangel, Philipp Hennig
NeurIPS 2021poster

When engineers train deep learning models, they are very much "flying blind". Commonly used methods for real-time training diagnostics, such as monitoring the train/test loss, are limited. Assessing a network's training process solely through these performance indicators is akin to debugging softwar…

Cited by 13Source ↗PDF ↗Scholar ↗
2021

Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers

Robin M Schmidt, Frank Schneider, Philipp Hennig
ICML 2021spotlight

Choosing the optimizer is considered to be among the most crucial design decisions in deep learning, and it is not an easy one. The growing literature now lists hundreds of optimization methods. In the absence of clear theoretical guidance and conclusive empirical evidence, the decision is often mad…

Cited by 232Source ↗PDF ↗Scholar ↗