PocketLLM: Ultimate Compression of Large Language Models via Meta Networks

Ye Tian; Chengcheng Wang; Jing Han; Yehui Tang; Kai Han

AAAI 2026technical0 citations

PocketLLM: Ultimate Compression of Large Language Models via Meta Networks

Ye Tian, Chengcheng Wang, Jing Han, Yehui Tang, Kai Han

Abstract

As Large Language Models (LLMs) continue to grow in size, storing and transmitting them on edge devices becomes increasingly challenging. Traditional methods like quantization and pruning struggle to achieve extreme compression of LLMs without sacrificing accuracy. In this paper, we introduce PocketLLM, a novel approach to compress LLMs in a latent space via meta-networks. A simple encoder network is proposed to project the weights of LLMs into discrete latent vectors, which are then represented using a compact codebook. A lightweight decoder network is employed to map the codebook

BibTeX

@inproceedings{aaai2026_pocketllmultimat,
  title = {PocketLLM: Ultimate Compression of Large Language Models via Meta Networks},
  author = {Ye Tian and Chengcheng Wang and Jing Han and Yehui Tang and Kai Han},
  booktitle = {AAAI 2026},
  year = {2026}
}