2026
Predicting Large Model Test Losses with a Noisy Quadratic System
ICML 2026poster
We introduce a predictive model that estimates the pre-training loss of large models from model size ($N$), batch size ($B$) and number of weight updates ($K$). This is the first loss prediction model that can handle changing batch size. The model outperforms Chinchilla's loss model, a model of the …