Mingyeong Song

SIREN: SPATIALLY-INFORMED RECONSTRUCTION OF BINAURAL AUDIO WITH VISION

Binaural audio delivers spatial cues essential for immersion, yet most consumer videos are monaural due to capture constraints. We introduce SIREN, a visually guided mono to binaural framework that explicitly predicts left and right channels. A ViT-based encoder learns dual-head self-attention to pr…