A Deep Reinforcement Learning Framework and Methodology for Reducing the Sim-to-Real Gap in ASV Navigation

Luis F. W. Batista; Junghwan Ro; Antoine Richard; Pete Schroepfer; Seth Hutchinson; Cedric Pradalier

IROS 2024poster3 citations

A Deep Reinforcement Learning Framework and Methodology for Reducing the Sim-to-Real Gap in ASV Navigation

Luis F. W. Batista, Junghwan Ro, Antoine Richard, Pete Schroepfer, Seth Hutchinson, Cedric Pradalier

Abstract

Despite the increasing adoption of Deep Reinforcement Learning (DRL) for Autonomous Surface Vehicles (ASVs), there still remain challenges limiting real-world deployment. In this paper, we first integrate buoyancy and hydrodynamics models into a modern Reinforcement Learning framework to reduce training time. Next, we show how system identification coupled with domain randomization improves the RL agent performance and narrows the sim-to-real gap. Real-world experiments for the task of capturing floating waste show that our approach lowers energy consumption by 13.1% while reducing task completion time by 7.4%. These findings, supported by sharing our open-source implementation, hold the potential to impact the efficiency and versatility of ASVs, contributing to environmental conservation efforts.

BibTeX

@inproceedings{iros2024_adeepreinforceme,
  title = {A Deep Reinforcement Learning Framework and Methodology for Reducing the Sim-to-Real Gap in ASV Navigation},
  author = {Luis F. W. Batista and Junghwan Ro and Antoine Richard and Pete Schroepfer and Seth Hutchinson and Cedric Pradalier},
  booktitle = {IROS 2024},
  year = {2024}
}