Mufan Li — accepted AI-conference papers

2025

Don't be lazy: CompleteP enables compute-efficient deep transformers

Nolan Simran Dey, Bin Claire Zhang, Lorenzo Noci, Mufan Li, Blake Bordelon, Shane Bergsma +3

NeurIPS 2025poster

We study compute efficiency of LLM training when using different parameterizations, i.e., rules for adjusting model and optimizer hyperparameters (HPs) as model size changes. Some parameterizations fail to transfer optimal base HPs (such as learning rate) across changes in model depth, requiring pra…

Cited by 0 Source ↗code ↗Scholar ↗

2021

The future is log-Gaussian: ResNets and their infinite-depth-and-width limit at initialization

Mufan Li, Mihai Nica, Daniel M. Roy

NeurIPS 2021poster

Theoretical results show that neural networks can be approximated by Gaussian processes in the infinite-width limit. However, for fully connected networks, it has been previously shown that for any fixed network width, $n$, the Gaussian approximation gets worse as the network depth, $d$, increases.…

Cited by 45 Source ↗PDF ↗Scholar ↗