LACY: A Vision-Language Model-Based Language-Action Cycle for Self-Improving Robotic Manipulation

Abstract

Learning generalizable policies for robotic manipulation increasingly relies on large-scale models that excel at mapping language instructions to actions (L2A). However, this unidirectional training paradigm often produces policies that can execute tasks without deeper contextual understanding, thereby limiting their ability to generalize and to explain their behavior. We argue that the complementary skill of mapping actions back to language (A2L) is essential for developing more holistic and robust grounding. An agent capable of both acting and explaining its actions can form richer internal representa- tions and, critically, unlock new paradigms for self-supervised learning. In this paper, we introduce LACY (Language-Action CYcle), a unified framework that learns such bidirectional mappings within a single vision-language model. LACY is jointly trained on three synergistic tasks: generating parameterized actions from language (L2A), explaining observed actions in language (A2L), and verifying semantic consistency between language pairs (L2C). The framework enables a self-improving cycle that autonomously generates new training data by chain- ing the L2A and A2L modules in an L2A2L pipeline. The L2C module then filters this data using an active data augmentation strategy that selectively targets low-confidence cases, thereby improving the model efficiently without requiring additional human annotations. Extensive experiments on pick-and-place tasks in both simulation and the real world demonstrate that LACY substantially improves task success rates by over 56.46% compared to baseline methods and yields more robust language- action grounding for robotic manipulation.