GPT-C: Generative PrompT Compression

Abstract

Large language models (LLMs) increasingly rely on lengthy prompts to achieve complex tasks. However, longer prompts not only increase inference costs but pose significant challenges for limited context windows of LLMs. Prompt compression aims to accelerate inference in long-text scenarios while preserving performance in various tasks. Conventional extractive compression methods often lead to semantic incoherence and loss of primitive key information. To this end, we propose a Generative PrompT Compression (GPT-C) paradigm. In the absence of appropriate datasets, we design a Collaborative Ordered Agent (Co-Agent) framework to distill high-quality datasets. We then develop a task-agnostic, length-adaptive reinforcement learning strategy that controls compression length and enhances the generalization across diverse tasks. Extensive experiments demonstrate that GPT-C achieves performance comparable to the original prompts while utilizing only 20% of the tokens.