Your Mileage May Vary: How Empathy and Demographics Shape Human Preferences in LLM Responses
Yishan Wang, Amanda Cercas Curry, Flor Miriam Plaza-del-Arco
Abstract
As large language models (LLMs) increasingly assist in subjective decision-making (e.g., moral reasoning, advice), it is critical to understand whose preferences they align with—and why. While prior work uses aggregate human judgments, demographic variation and its linguistic drivers remain underexplored. We present a comprehensive analysis of how demographic background and empathy level correlate with preferences for LLM-generated dilemma responses, alongside a systematic study of predictive linguistic features (e.g., agency, emotional tone). Our findings reveal significant demographic divides and identify markers (e.g., power verbs, tentative phrasing) that predict group-level differences. These results underscore the need for demographically informed LLM evaluation.
BibTeX
@inproceedings{emnlp2025_yourmileagemayva,
title = {Your Mileage May Vary: How Empathy and Demographics Shape Human Preferences in LLM Responses},
author = {Yishan Wang and Amanda Cercas Curry and Flor Miriam Plaza-del-Arco},
booktitle = {EMNLP 2025},
year = {2025}
}