Autoregressive Meta-Actions for Unified Controllable Trajectory Generation in Autonomous Driving

Abstract

Generating trajectories from high-level commands is critical for autonomous driving, but prevailing methods suffer from a flaw we term semantic misalignment. By associating long trajectories with a single, static meta-action (e.g., “lane change”), these methods corrupt training data during maneuver transitions, hindering the learning of robust command-following. To resolve this, we propose the Autoregressive Meta-Action (AMA) framework, a novel formulation that operates at a frame-wise level. It autoregressively predicts the joint distribution of the next-step meta-action and its corresponding state, ensuring strict semantic alignment at every timestep. We implement this framework using a flexible modular architecture where lightweight modules are fine-tuned atop a pre-trained foundation model. This modular design drastically reduces training costs for new commands and enables seamless switching between autonomous and command-following modes. We support our research by releasing an expanded Waymo Open Motion Dataset with dense, frame-level meta-action labels and validate our method’s efficacy on real-world test vehicles.