Semantic-Aware Behavior Optimization With Safety Reinforcement Feedback for Language-Conditioned Manipulation

Abstract

Language-conditioned manipulation (LCM) couples vision, language, and control to enable natural instruction following, making it a promising direction in embodied AI research. Researchers have combined imitation learning (IL) with reinforcement learning (RL), where IL provides sample-efficient initialization and RL refines the policy through feedback to maximize long-term returns. Despite progress, safe LCM execution faces two challenges: (1) a modality gap that disrupts semantic-spatial correspondence, degrading action quality, and (2) sparse safety costs in high-dimensional states that make constraint estimation and semantic–safety alignment difficult. In this paper, we present Semantic-Aware Behavior Optimization with Safety Reinforcement Feedback, a novel framework that mitigates multimodal misalignment through context-aware initialization and enforces semantic–safety consistency during execution. We introduce spatial–semantic priors to align language goals with spatial observations and embed safety cues. Specifically, a goal prior aligns modalities for context-aware initialization during behavioral cloning, while a constraint prior combined with a feasibility metric provides learnable supervision for safety reinforcement fine-tuning. Evaluations show our framework achieves a 6.7% relative improvement in simulation and effectively reduces safety violations in real-world tasks across diverse scenarios.