Social Robot Haru Assisting Dynamic Group Discussion with Autonomous Eye Gaze Behavior

Abstract

Due to recent advances in large language models and robotics, social robots will potentially play an important role in people’s daily lives soon, and are expected to improve dynamic multi-party group discussions in social scenarios. In this paper, we developed a system to assist dynamic group discussion with our social robot Haru. Our system is composed of three modules: a Dialogue Assistance module via integrating Haru with large language models which facilitates Haru to be an embodied chatbot; a Balancing and Welcoming Behavior module to improve users’ engagement and welcome new users to join the discussion with verbal behaviors; an Autonomous Eye Gazing module to show politeness during group discussion, e.g., gazing to the talking user or the less-engaging user to encourage her, looking to the new comer when she joins the discussion, gazing via eyeball movement when the current speaking user is close to the previous one. The autonomous eye gazing behavior was first trained via deep reinforcement learning in simulation and transferred to physical Haru in the real world. Results of our user study with 50 subjects show the significant performance of our system in assisting dynamic group discussion.