Mental-Perceiver: Audio-Textual Multi-Modal Learning for Estimating Mental Disorders
Jinghui Qin, Changsong Liu, Tianchi Tang, Dahuang Liu, Minghao Wang, Qianying Huang, Rumin Zhang
Abstract
Mental disorders, such as anxiety and depression, have become a global concern that affects people of all ages. Early detection and treatment are crucial to mitigate the negative effects these disorders can have on daily life. Although AI-based detection methods show promise, progress is hindered by the lack of publicly available large-scale datasets. To address this, we introduce the Multi-Modal Psychological assessment corpus (MMPsy), a large-scale dataset containing audio recordings and transcripts from Mandarin-speaking adolescents undergoing automated anxiety/depression assessment interviews. MMPsy also includes self-reported anxiety/depression evaluations using standardized psychological questionnaires. Leveraging this dataset, we propose Mental-Perceiver, a deep learning model for estimating mental disorders from audio and textual data. Extensive experiments on MMPsy and the DAIC-WOZ dataset demonstrate the effectiveness of Mental-Perceiver in anxiety and depression detection.
BibTeX
@article{Qin_Liu_Tang_Liu_Wang_Huang_Zhang_2025, title={Mental-Perceiver: Audio-Textual Multi-Modal Learning for Estimating Mental Disorders}, volume={39}, url={https://ojs.aaai.org/index.php/AAAI/article/view/34687}, DOI={10.1609/aaai.v39i23.34687}, abstractNote={Mental disorders, such as anxiety and depression, have become a global concern that affects people of all ages. Early detection and treatment are crucial to mitigate the negative effects these disorders can have on daily life. Although AI-based detection methods show promise, progress is hindered by the lack of publicly available large-scale datasets. To address this, we introduce the Multi-Modal Psychological assessment corpus (MMPsy), a large-scale dataset containing audio recordings and transcripts from Mandarin-speaking adolescents undergoing automated anxiety/depression assessment interviews. MMPsy also includes self-reported anxiety/depression evaluations using standardized psychological questionnaires. Leveraging this dataset, we propose Mental-Perceiver, a deep learning model for estimating mental disorders from audio and textual data. Extensive experiments on MMPsy and the DAIC-WOZ dataset demonstrate the effectiveness of Mental-Perceiver in anxiety and depression detection.}, number={23}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Qin, Jinghui and Liu, Changsong and Tang, Tianchi and Liu, Dahuang and Wang, Minghao and Huang, Qianying and Zhang, Rumin}, year={2025}, month={Apr.}, pages={25029-25037} }