Zeyu XING — accepted AI-conference papers

2026

Beyond Speedup - Utilizing KV Cache for Sampling and Reasoning

Zeyu XING, Xing Li, Hui-Ling Zhen, Mingxuan Yuan, Sinno Jialin Pan

ICLR 2026poster

KV caches, typically used only to speed up autoregressive decoding, encode contextual information that can be reused for downstream tasks at no extra cost. We propose treating the KV cache as a lightweight representation, eliminating the need to recompute or store full hidden states. Despite being w…

Cited by 0 Source ↗Scholar ↗

2025

KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference

Xing Li, Zeyu XING, Yiming Li, Linping Qu, Hui-Ling Zhen, Yiwu Yao +3

ICML 2025poster

KV cache quantization can improve Large Language Models (LLMs) inference throughput and latency in long contexts and large batch-size scenarios while preserving LLMs effectiveness. However, current methods have three unsolved issues: overlooking layer-wise sensitivity to KV cache quantization, high…

Cited by 0 Source ↗PDF ↗code ↗Scholar ↗