3 accepted papers
Designing sparse attention for 2-dimensional image data in diffusion models and vision-language models requires reconciling spatial locality with hardware-efficient execution, a fundamental trade-off that existing methods struggle to resolve. Prior approaches preserve 2D structure through handcrafte…
Retrieval-augmented generation (RAG) for long-context question answering is bottlenecked by inference-time prefilling over large retrieved contexts. A common strategy is to precompute key–value (KV) caches for individual documents and selectively recompute a small subset of tokens to restore global …
Diffusion models excel in high-fidelity image generation but face scalability limits due to transformers’ quadratic attention complexity. Plug-and-play token reduction methods like ToMeSD and ToFu reduce FLOPs by merging redundant tokens in generated images but rely on GPU-inefficient operations (e.…