Uncovering Scaling Laws for Large Language Models via Inverse Problems
Arun Verma, Zhaoxuan Wu, Zijian Zhou, Xiaoqiang Lin, Zhiliang Chen, Rachael Hwee Ling Sim, Rui Qiao, Jingtan Wang
Abstract
Large Language Models (LLMs) are large-scale pretrained models that have achieved remarkable success across diverse domains. These successes have been driven by unprecedented complexity and scale in both data and computations. However, due to the high costs of training such models, brute-force trial-and-error approaches to improve LLMs are not feasible. Inspired by the success of inverse problems in uncovering fundamental scientific laws, this position paper advocates that inverse problems can also efficiently uncover scaling laws that guide the building of LLMs to achieve the desirable performance with significantly better cost-effectiveness.
BibTeX
@inproceedings{emnlp2025_uncoveringscalin,
title = {Uncovering Scaling Laws for Large Language Models via Inverse Problems},
author = {Arun Verma and Zhaoxuan Wu and Zijian Zhou and Xiaoqiang Lin and Zhiliang Chen and Rachael Hwee Ling Sim and Rui Qiao and Jingtan Wang and Nhung Bui and Xinyuan Niu and Wenyang Hu and Gregory Kang Ruey Lau and Zi-Yu Khoo and Zitong Zhao and Xinyi Xu and Apivich Hemachandra and See-Kiong Ng and Bryan Kian Hsiang Low},
booktitle = {EMNLP 2025},
year = {2025}
}