LLM-Driven Expressive Robot Motion Via Proxy-Based Optimization

Abstract

Nonverbal communication through expressive robot motion significantly enhances human-robot interaction (HRI), yet designing motions that are both individually legible and collectively distinguishable remains challenging. Existing large language model (LLM)-based methods can successfully generate expressive robot motions but can fail to adequately differentiate motions for communicating closely related functional robot states, resulting in perceptual confusion. To address this, we introduce a hybrid optimization framework combining an LLM-derived accuracy proxy with distance metrics to autonomously produce legible and distinct expressive robot motions. We explore two distance metrics: the Earth Mover's Distance (EMD) and a kinematic trajectory distance. A validation experiment demonstrates the LLM-generated accuracy proxy's capability to mimic human perceptual responses (90.4% alignment), offering a scalable method for evaluating expressive robot motions without human observers. Results from a user study (<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$N=79$</tex-math></inline-formula>) demonstrate that the proposed optimization method significantly enhanced state classification accuracy over LLM-generated motions. The inclusion of the kinematic distance metric provided the greatest improvement in legibility. The trade-off between motion clarity and distance between motions emerged as a critical design parameter. Our codebase is available on GitHub at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/liamreneroy/LLM_vocab_optimization</uri>