SeeHear: Signer Diarisation and a New Dataset
Samuel Albanie, Gül Varol, Liliane Momeni, Triantafyllos Afouras, Andrew Brown, Chuhan Zhang, Ernesto Coto, Necati Cihan Camgöz
Abstract
In this work, we propose a framework to collect a large-scale, diverse sign language dataset that can be used to train automatic sign language recognition models.The first contribution of this work is SDTrack, a generic method for signer tracking and diarisation in the wild. Our second contribution is SeeHear, a dataset of 90 hours of British Sign Language (BSL) content featuring more than 1000 signers, and including interviews, monologues and debates. Using SDTrack, the SeeHear dataset is annotated with 35K active signing tracks, with corresponding signer identities and subtitles, and 40K automatically localised sign labels. As a third contribution, we provide benchmarks for signer diarisation and sign recognition on SeeHear.
BibTeX
@inproceedings{icassp2021_seehearsignerdia,
title = {SeeHear: Signer Diarisation and a New Dataset},
author = {Samuel Albanie and Gül Varol and Liliane Momeni and Triantafyllos Afouras and Andrew Brown and Chuhan Zhang and Ernesto Coto and Necati Cihan Camgöz and Ben Saunders and Abhishek Dutta and Neil Fox and Richard Bowden and Bencie Woll and Andrew Zisserman},
booktitle = {ICASSP 2021},
year = {2021}
}