EDAIL: Adversarial Imitation Learning Via Exploration-Driven Data Augmentation

Abstract

Adversarial Imitation Learning (AIL) is a prominent paradigm in imitation learning that enables policy acquisition from expert demonstrations without relying on manually crafted reward functions. Although AIL has achieved promising results in certain scenarios, many existing methods suffer from mode collapse and training instability when expert demonstrations are limited. Given that agent–environment interactions are often abundant, we focus on effectively leveraging such interaction data to address the above challenges. In this paper, we propose a novel adversarial imitation learning framework called Exploration-Driven Adversarial Imitation Learning (EDAIL). First, we introduce exploratory policies that augment the discriminator’s training data with high-confidence state-action pairs generated by the agent, thereby improving coverage of the solution space under sparse expert data. Second, we design an asymmetric surrogate reward function that shifts the reward-penalty boundary to mitigate discriminator bias caused by class imbalance, enabling more reliable policy optimization. We evaluate our method on six simulated tasks, including robotic manipulation, locomotion, and navigation, using only 1% and 10% of the datasets employed in prior baselines as expert demonstrations. Experimental results show that our method outperforms the baselines, demonstrating both the effectiveness and robustness of our method. In particular, it achieves a success rate of 94% on the FetchPush task using only 1% of expert demonstrations, representing an absolute improvement of 19 points over the state-of-the-art method. Our code will be available at https://github.com/lipengcheng-nudt/EDAIL.