Manuel Hochmeister

OLViT: Multi-Modal State Tracking via Attention-Based Embeddings for Video-Grounded Dialog

We present the Object Language Video Transformer (OLViT) – a novel model for video dialog operating over a multi-modal attention-based dialog state tracker. Existing video dialog models struggle with questions requiring both spatial and temporal localization within videos, long-term temporal reasoni…