PAIR: Prompt-Aware margIn Ranking for Counselor Reflection Scoring in Motivational Interviewing
Do June Min, Verónica Pérez-Rosas, Kenneth Resnicow, Rada Mihalcea
Abstract
Counselor reflection is a core verbal skill used by mental health counselors to express understanding and affirmation of the client’s experience and concerns. In this paper, we propose a system for the analysis of counselor reflections. Specifically, our system takes as input one dialog turn containing a client prompt and a counselor response, and outputs a score indicating the level of reflection in the counselor response. We compile a dataset consisting of different levels of reflective listening skills, and propose the Prompt-Aware margIn Ranking (PAIR) framework that contrasts positive and negative prompt and response pairs using specially designed multi-gap and prompt-aware margin ranking losses. Through empirical evaluations and deployment of our system in a real-life educational environment, we show that our analysis model outperforms several baselines on different metrics, and can be used to provide useful feedback to counseling trainees.
BibTeX
@inproceedings{min-etal-2022-pair,
title = "{PAIR}: Prompt-Aware marg{I}n Ranking for Counselor Reflection Scoring in Motivational Interviewing",
author = "Min, Do June and
P{\'e}rez-Rosas, Ver{\'o}nica and
Resnicow, Kenneth and
Mihalcea, Rada",
editor = "Goldberg, Yoav and
Kozareva, Zornitsa and
Zhang, Yue",
booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing",
month = dec,
year = "2022",
address = "Abu Dhabi, United Arab Emirates",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.emnlp-main.11/",
doi = "10.18653/v1/2022.emnlp-main.11",
pages = "148--158"
}