Focus-Then-Reuse: Fast Adaptation in Visual Perturbation Environments
Jiahui Wang, Chao Chen, Jiacheng Xu, Zongzhang Zhang, Yang Yu
Abstract
Visual reinforcement learning has shown promise in various real-world applications. However, deploying policies in complex real-world environments with visual perturbations remains a significant challenge. We notice that humans tend to filter information at the object level prior to decision-making, facilitating efficient skill transfer across different contexts. Inspired by this, we introduce Focus-Then-Reuse (FTR), a method utilizing a novel object selection mechanism to focus on task-relevant objects, and directly reuse the simulation-trained policy on them. The training of the object selection mechanism integrates prior knowledge from a vision-language model and feedback from the environment. Experimental results on challenging tasks based on DeepMind Control Suite and Franka Emika Robotics demonstrate that FTR enables rapid adaptation in visual perturbation environments and achieves state-of-the-art performance. The source code is available at https://github.com/LAMDA-RL/FTR.
BibTeX
@inproceedings{
wang2025focusthenreuse,
title={Focus-Then-Reuse: Fast Adaptation in Visual Perturbation Environments},
author={Jiahui Wang and Chao Chen and Jiacheng Xu and Zongzhang Zhang and Yang Yu},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
year={2025},
url={https://openreview.net/forum?id=I4fBSpDOha}
}