Zhangyu Chen — accepted AI-conference papers

2026

DualMap: Enabling Both Cache Affinity and Load Balancing for Distributed LLM Serving

Ying Yuan, Pengfei Zuo, Bo Wang, Zhangyu Chen, Zhipeng Tan, Zhou Yu

ICLR 2026poster

In large language model (LLM) serving, reusing the key-value (KV) cache of prompts across requests is a key technique for reducing time-to-first-token (TTFT) and lowering serving costs. Cache-affinity scheduling, which co-locates requests with the same prompt prefix to maximize KV cache reuse, often…

Cited by 0 Source ↗code ↗Scholar ↗

2026

TileSparse: Arithmetic-Intensity-Aware Sparse Attention for Compute-Bound LLM Decoding

Chao Wang, Pengfei Zuo, Zhangyu Chen, Qihui Zhou, Tsung-Yi Ho, Ming-Chang Yang

ICML 2026poster

Sparse attention has emerged as a vital technique for long-context inference in Large Language Models (LLMs), effectively accelerating memory-bound decoding by reducing memory access for non-essential keys. However, the assumption that decoding attention is memory-bound has been shattered. The proli…

Cited by 0 Source ↗Scholar ↗