MotionVL: Vision-Language Supervision for Reinforcement Learning of Humanoid Motion

Abstract

Reward function design remains a fundamental challenge in reinforcement learning of humanoid robots, where handcrafted rewards often fail to capture human-like behavior, limit generalization, and require costly manual tuning. Addressing this bottleneck is essential for enabling scalable, adaptive, interpretable, and robust humanoid motion control. In this work, MotionVL, a novel framework for humanoid motion reinforcement learning that leverages multimodal large models to automate reward generation and semantic supervision, is proposed. A Vision-Language Model (VLM) encodes visual observations into structured natural language descriptions, while a Large Language Model (LLM) generates task-aligned reward functions based on these descriptions and the given instructions. This closed-loop design supports dynamic reward optimization and behavior alignment across diverse motor skills. We validate MotionVL through simulations of bipedal jumping, single-leg balancing, running, and real-world deployment of straight-leg walking, which demonstrates higher task success rates, improved robustness to external pushes, better energy efficiency, and superior stability and human-likeness over handcrafted and LLM-only baselines, thereby establishing a scalable paradigm for language-informed humanoid motion learning.