BAH Dataset for Ambivalence/Hesitancy Recognition in Videos for Behavioural Change
Manuela González-González, Soufiane Belharbi, Muhammad Osama Zeeshan, Masoumeh Sharafi, Muhammad Haseeb Aslam, Marco Pedersoli, Alessandro Lameiras Koerich, Simon L Bacon
Abstract
This paper introduces the Behavioral Ambivalence/Hesitancy (BAH) dataset collected for the Ambivalence/Hesitancy (A/H) recognition task in videos. In particular, this task involves recognizing conflicting emotions linked to A/H from question-and-answer videos captured for behavior analysis. The dataset contains videos from 224 subjects with different age, ethnicity collected across 9 Canadian provinces via webcam through our developed web platform. Each user answers to 7 questions that we designed to induce Ambivalence/Hesitancy. Each video captures the response for one question in the subject's environment totaling 1,118 videos for a total duration of 8.26 hours with 1.5 hours of A/H. BAH is a first and unique dataset for Ambivalence/Hesitancy recognition. Our behavioral team annotated timestamp segment where A/H occurs providing frame- and video-level annotation, in addition to used cues for annotation such as face, audio, body, and language. Video transcripts and their timestamps as well as per-frame cropped and aligned faces are also included. This work offers initial baselines for A/H recognition in videos at frame- and video-level with different analysis with single and multimodal setups. The data, code, and pretrained weights are publicly accessible.
BibTeX
@inproceedings{
gonzalez-gonzalez2026bah,
title={{BAH} Dataset for Ambivalence/Hesitancy Recognition in Videos for Behavioural Change},
author={Manuela Gonz{\'a}lez-Gonz{\'a}lez and Soufiane Belharbi and Muhammad Osama Zeeshan and Masoumeh Sharafi and Muhammad Haseeb Aslam and Marco Pedersoli and Alessandro Lameiras Koerich and Simon L Bacon and Eric Granger},
booktitle={The Fourteenth International Conference on Learning Representations},
year={2026},
url={https://openreview.net/forum?id=jYDHVscRO3}
}