Linfeng Yin

LFS: Learnable Frame Selector for Event-Aware and Temporally Diverse Video Captioning

Video captioning models convert frames into visual tokens and generate descriptions with large language models (LLMs). Since encoding all frames is prohibitively expensive, uniform sampling is the default choice, but it enforces equal temporal coverage while ignoring the uneven events distribution.