Internal Location Assistance for Temporal Action Proposal Generation

Abstract

Temporal action proposal generation (TAPG) aims to locate action instances in untrimmed videos for video analysis tasks. In this paper, we propose a novel approach called Internal Location Assistance Net (ILAN) to take advantage of the internal action point instead of only utilizing the start and end points themselves. Specifically, instead of solely predicting the action’s start and end positions, we also predict extra internal points which could be the one-eighth, one-fourth or center points etc. Then the pair of left and right internal positions of the action are matched to generate center region proposals. Then the predicted center region is constrained to lie in the predicted overall action region. And both confidences of the overall action region and the center region are combined to get the final action proposal. Besides, we incorporate the window transformer to enhance the feature extraction for capturing more precise action boundaries. Extensive experiments are conducted on two popular benchmark datasets: THUMOS14 and ActivityNet-v1.3. The experimental results demonstrate that the proposed method outperforms the state-of-the-arts.