Exploring Acted Sleepy Speech to Advance Real-World Sleepiness Estimation and Cognitive Degradation Detection

Abstract

Accurate estimation of sleepiness levels is crucial for managing sleep-related health risks and preventing cognitive degradation that can lead to accidents in the workplace. However, using machine learning (ML) to estimate these levels from speech remains challenging, with reported weak correlations (<0.40) with ground-truth sleepiness levels. Developing effective ML models requires high-quality sleepiness data from noticeably sleepy individuals, but collecting such data through prolonged sleep deprivation is both risky and costly. We propose that acted (feigned) sleepy speech can effectively represent realistic sleepiness and be used to train ML models for estimating sleepiness levels and detecting sleepiness-associated cognitive performance degradation. Our study demonstrates that: (1) human listeners perceive acted sleepy speech as sleepier than both non-sleepy and genuinely sleepy speech, and (2) ML models trained on acted speech can accurately estimate sleepiness levels in individuals who have been awake for 25 hours, achieving a correlation of 0.54 with ground-truth sleepiness levels while using fewer samples. Furthermore, the acted sleepy speech-based ML model detects cognitive performance degradation (F1 score = 0.80) in sleep-deprived individuals, outperforming models trained on real sleepy speech (F1 score = 0.32). Our approach provides efficient, effective, and scalable solutions to not only update benchmarks but also enhance the capabilities of real-world AI applications, from voice assistants to AI agents, ultimately supporting human health, workplace safety, and daily tasks.