Keyword Search for Sign Language

Nazif Can Tamer; Murat Saraçlar

ICASSP 2020accepted0 citations

Keyword Search for Sign Language

Abstract

Keyword search is the search for a written query in an archive, which is often assumed to be a collection from a spoken language. Yet, the main languages of the Deaf, i.e. sign languages, are mostly neglected in this definition due to being visual languages. In this paper, we propose a keyword search (KWS) system for a sign language. In this technique, we first extract body and hand joints from the frames of a sign language sentence and represent it with a unified spatio-temporal graph of skeleton joints. We then train our graph-convolutional-network encoder, query embedding, and selection mechanism together in a weakly supervised, end-to-end fashion. Experimental results are reported on RWTH-PHOENIX-Weather 2014T dataset.

BibTeX

@inproceedings{icassp2020_keywordsearchfor,
  title = {Keyword Search for Sign Language},
  author = {Nazif Can Tamer and Murat Saraçlar},
  booktitle = {ICASSP 2020},
  year = {2020}
}