Word-Level ASL Recognition and Trigger Sign Detection with RF Sensors
Mohammad Mahbubur Rahman, Emre Kurtoglu, Robiulhossain Mdrafi, Ali Cafer Gürbüz, Evie Malaia, Chris S. Crawford, Darrin J. Griffin, Sevgi Zubeyde Gurbuz
Abstract
Current research in the recognition of American Sign Language (ASL) has focused on perception using video or wearable gloves. However, deaf ASL users have expressed concern about the invasion of privacy with video, as well as the interference with daily activity and restrictions on movement presented by wearable gloves. In contrast, RF sensors can mitigate these issues as it is a non-contact ambient sensor that is effective in the dark and can penetrate clothes, while only recording speed and distance. Thus, this paper investigates RF sensing as an alternative sensing modality for ASL recognition to facilitate interactive devices and smart environments for the deaf and hard-of-hearing. In particular, the recognition of up to 20 ASL signs, sequential classification of signing mixed with daily activity, and detection of a trigger sign to initiate human-computer interaction (HCI) via RF sensors is presented. Results yield %91.3 ASL word-level classification accuracy, %92.3 sequential recognition accuracy, 0.93 trigger recognition rate.
BibTeX
@inproceedings{icassp2021_wordlevelaslreco,
title = {Word-Level ASL Recognition and Trigger Sign Detection with RF Sensors},
author = {Mohammad Mahbubur Rahman and Emre Kurtoglu and Robiulhossain Mdrafi and Ali Cafer Gürbüz and Evie Malaia and Chris S. Crawford and Darrin J. Griffin and Sevgi Zubeyde Gurbuz},
booktitle = {ICASSP 2021},
year = {2021}
}