Evaluation Measures for Depression Prediction and Affective Computing
Sadari Jayawardena, Julien Epps, Eliathamby Ambikairajah
Abstract
A variety of evaluation measures are being used to validate systems in depression prediction and affective computing. Among them, the most common measures focus on the error between the ground truth and predictions. However, when the ground truth is ordinal such as in psychiatric scores, ranking information is more important than the actual error. Therefore, this study systematically analyses the properties of classification, error-based and ranking measures particularly using classification accuracy, root mean square error (RMSE) and Spearman rank correlation coefficient, with the aim of identifying suitable measures for evaluating depression prediction and affective computing. For the purpose of analysis, we employed both synthetic data and real depression prediction systems evaluated with the AVEC2017 depression corpus. Outcomes of the experiments suggest that RMSE and classification accuracy, which are frequently used, are not sensitive to ordering and that rank correlation measures are more appropriate for depression prediction, which is an ordinal problem.
BibTeX
@inproceedings{icassp2019_evaluationmeasur,
title = {Evaluation Measures for Depression Prediction and Affective Computing},
author = {Sadari Jayawardena and Julien Epps and Eliathamby Ambikairajah},
booktitle = {ICASSP 2019},
year = {2019}
}