Investigating the Role of Implicit Signals in Adaptive User-Aware Human-Robot Interactions

Abstract

Our work investigates how social robots can act in a user-aware manner by adapting their behaviour to users' personal characteristics and preferences without unnecessarily exposing them to frustration through the robot's actions. In particular, we investigate how implicit social signals inadvertently exhibited by users (e.g. facial expressions) during interactions can be incorporated into user-aware decision-making models while accounting for the systematic limitations of implicit feedback signals (e.g. inconsistency, noise, culture and individual-dependence). Doing so, we develop a user-aware adaptive decision-making and learning framework for human-robot interactions, building on implicit signal processing, cue-based intent inference, and multiarmed bandit learning techniques. Evaluating our approach, we conduct a user study where participants interact with a Pepper robot in a cafeteria style interaction scenario, with the robot providing recommendations and taking orders while adapting its behaviour to individual users. The experimental results demonstrate our proposed model's success in adapting its behaviour (i.e. conversational style) to users with different personal characteristics, while receiving 80% positive user feedback, and user questionnaire responses reporting higher perceived usefulness than baseline approaches. Questionnaire responses also illustrate positive user impressions of implicit signal based approaches while highlighting the importance of accounting for their limitations in learning models. In addition, we provide a dataset of over 5 hours of human and robot behaviour data extracted from multimodal recordings captured as part of our user study.