2025
HiFC: High-efficiency Flash-based KV Cache Swapping for Scaling LLM Inference
NeurIPS 2025poster
Large‑language‑model inference with long contexts often produces key–value (KV) caches whose footprint exceeds the capacity of high‑bandwidth memory on a GPU. Prior LLM inference frameworks such as vLLM mitigate this pressure by swapping KV cache pages to host DRAM. However, the high cost of large D…