Soosung Kim

GSRQ: Gain-Shape Residual Quantization for Sub-1-bit KV Cache

The deployment of Large Language Models (LLMs) with extended context windows is increasingly constrained by the linear growth of Key-Value (KV) cache memory. Vector Quantization (VQ) is a key enabler for pushing KV cache storage toward the sub-1-bit regime; in particular, Residual Quantization (RQ) …