Haojuan Li

SegMo: Co-Designing Content-Aware Sparsity and Locally-Cohesive Segment Parallelism for Efficient VLM Inference

Video Large Language Models (VideoLLMs) face a fundamental performance bottleneck: the token explosion intrinsic to video inputs. The resulting O(N^2) prefill cost makes conventional Transformer inference prohibitively expensive at scale. Existing attempts fall into a hard accuracy-latency dilemma: