Online Residual-Based Key Frame Sampling with Self-Coach Mechanism and Adaptive Multi-Level Feature Fusion
Rui Zhang, Yang Hua, Tao Song, Zhengui Xue, Ruhui Ma, Haibing Guan
Abstract
Key frame sampling is a common component in video tasks. Putting more effort into key frames, rather than processing all frames equally, can significantly reduce computational costs and improve processing efficiency. This paper presents ORSampler, an adaptive Online Residual-based key frame Sampler. ORSampler relies on feature residuals to sample key frames and decouples from subsequent video tasks. To facilitate ORSampler, a self-coached mechanism is designed to speed up learning, and an adaptive multi-level feature fusion is proposed to fit the diversity of subsequent video tasks. OR-Sampler has a fast inference speed and can work online. Extensive experiments on two typical video tasks verify the effectiveness and generality of our proposed ORSampler.
BibTeX
@inproceedings{icassp2023_onlineresidualba,
title = {Online Residual-Based Key Frame Sampling with Self-Coach Mechanism and Adaptive Multi-Level Feature Fusion},
author = {Rui Zhang and Yang Hua and Tao Song and Zhengui Xue and Ruhui Ma and Haibing Guan},
booktitle = {ICASSP 2023},
year = {2023}
}