MeRino: Entropy-Driven Design for Generative Language Models on IoT Devices
Youpeng Zhao, Ming Lin, Huadong Tang, Qiang Wu, Jun Wang
Abstract
Generative Large Language Models (LLMs) stand as a revolutionary advancement in the modern era of artificial intelligence (AI). However, scaling down LLMs for resource-constrained hardware, such as Internet-of-Things (IoT) devices requires non-trivial efforts and domain knowledge. In this paper, we propose a novel information-entropy framework for designing mobile-friendly generative language models. The whole design procedure involves solving a mathematical programming (MP) problem, which can be done on the CPU within minutes, making it nearly zero-cost. We evaluate our designed models, termed MeRino, across fourteen NLP downstream tasks, showing their competitive performance against the state-of-the-art autoregressive transformer models under the mobile setting. Notably, MeRino achieves similar or better performance on both language modeling and zero-shot learning tasks, compared to the 350M parameter OPT while being 4.9x faster on NVIDIA Jetson Nano with 5.5x reduction in model size.
BibTeX
@article{Zhao_Lin_Tang_Wu_Wang_2025, title={MeRino: Entropy-Driven Design for Generative Language Models on IoT Devices}, volume={39}, url={https://ojs.aaai.org/index.php/AAAI/article/view/34445}, DOI={10.1609/aaai.v39i21.34445}, abstractNote={Generative Large Language Models (LLMs) stand as a revolutionary advancement in the modern era of artificial intelligence (AI). However, scaling down LLMs for resource-constrained hardware, such as Internet-of-Things (IoT) devices requires non-trivial efforts and domain knowledge. In this paper, we propose a novel information-entropy framework for designing mobile-friendly generative language models. The whole design procedure involves solving a mathematical programming (MP) problem, which can be done on the CPU within minutes, making it nearly zero-cost. We evaluate our designed models, termed MeRino, across fourteen NLP downstream tasks, showing their competitive performance against the state-of-the-art autoregressive transformer models under the mobile setting. Notably, MeRino achieves similar or better performance on both language modeling and zero-shot learning tasks, compared to the 350M parameter OPT while being 4.9x faster on NVIDIA Jetson Nano with 5.5x reduction in model size.}, number={21}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Zhao, Youpeng and Lin, Ming and Tang, Huadong and Wu, Qiang and Wang, Jun}, year={2025}, month={Apr.}, pages={22840-22848} }