Impact of Temporal Precision on Speech Synthesis Accuracy From Electrocorticographic Brain Signals
Qinwan Rabbani, Matthew S. Fifer, Nathan E. Crone, Laureano Moro-Velázquez
Abstract
Accurately time-aligned spectral targets are essential for training electrocorticographic (ECoG) brain-computer interfaces (BCIs) intended for real-time speech output. This alignment is particularly challenging with "silent speech," in which speech occurs without phonation, or in the extreme, without articulation. Crafting suitably precise targets for silent speech is complex and error-prone due to multiple sources of temporal imprecision. We investigated how these temporal inaccuracies impact deep neural network performance in synthesizing speech from a BCI clinical trial participant who retained some speech capability. By simulating silent speech conditions through distortions in known acoustic target timings, we observed significant performance degradations at both phonetic and syllabic timescales. Accuracy-based measures offered a more reliable assessment of intelligibility than traditional metrics like the short-time objective intelligibility index, which may overestimate performance in low-precision contexts. These results underscore the need for advanced alignment techniques with precise phonetic and syllabic guarantees for silent speech BCIs focused on providing immediate output.
BibTeX
@inproceedings{icassp2025_impactoftemporal,
title = {Impact of Temporal Precision on Speech Synthesis Accuracy From Electrocorticographic Brain Signals},
author = {Qinwan Rabbani and Matthew S. Fifer and Nathan E. Crone and Laureano Moro-Velázquez},
booktitle = {ICASSP 2025},
year = {2025}
}