OmniSparse: Training-Aware Fine-Grained Sparse Attention for Long-Video MLLMs
Existing sparse attention methods primarily target inference-time acceleration by selecting critical tokens under predefined sparsity patterns. However, they often fail to bridge the training–inference gap and lack the capacity for fine-grained token selection across multiple dimensions—such as quer