2026
$\mu$pscaling small models: Principled warm starts and hyperparameter transfer
ICML 2026poster
Modern large-scale neural networks are often trained and released in multiple sizes to accommodate diverse inference budgets. To improve efficiency, recent work has explored *model upscaling*: initializing larger models from trained smaller ones in order to transfer knowledge and accelerate converge…