Minimalistic Video Saliency Prediction via Efficient Decoder & Spatio Temporal Action Cues
Rohit Girmaji, Siddharth Jain, Bhav Beri, Sarthak Bansal, Vineet Gandhi
Abstract
This paper introduces ViNet-S, a 36MB model based on the ViNet architecture with a U-Net design, featuring a lightweight decoder that significantly reduces model size and parameters without compromising performance. Additionally, ViNet-A (148MB) incorporates spatio-temporal action localization (STAL) features, differing from traditional video saliency models that use action classification backbones. Our studies show that an ensemble of ViNet-S and ViNet-A, by averaging predicted saliency maps, achieves state-of-the-art performance on three visual-only and six audio-visual saliency datasets, outperforming transformer-based models in both parameter efficiency and real-time performance, with ViNet-S reaching over 1000fps.
BibTeX
@inproceedings{icassp2025_minimalisticvide,
title = {Minimalistic Video Saliency Prediction via Efficient Decoder & Spatio Temporal Action Cues},
author = {Rohit Girmaji and Siddharth Jain and Bhav Beri and Sarthak Bansal and Vineet Gandhi},
booktitle = {ICASSP 2025},
year = {2025}
}