PEMTRS: Perception-Enhanced Memory With Temporal Region Selection for Vision-Based Multirotor Navigation

Abstract

End-to-end deep learning methods for autonomous aerial vehicles like multirotors have been successfully applied to high-agility flight. However, existing end-to-end navigation methods for multirotors typically rely solely on the robot's current observation and state, neglecting explicit modeling of temporal sequence information. Yet trajectory prediction is inherently temporal: past observations provide cues about future feasible regions, and this temporal feasibility is still not explicitly exploited in current end-to-end navigation frameworks. To overcome these limitations, we propose the PEMTRS framework, which enables the multirotor to fly with thinking by building a perception-enhanced memory structure and employing a temporal region selector to highlight regions of interest for traversability. By jointly modeling spatial perception and temporal information, PEMTRS generalizes better within the evaluated environment class than state-of-the-art end-to-end methods. To achieve more efficient training, we adopt an implicit learning strategy that replaces expert-labeled demonstration trajectories with cost-based learning signals to guide network optimization. In addition, we design an adaptive time-allocation network to dynamically adjust the time intervals between waypoints and a residual VAE to reconstruct obstacle depth for perception enhancement. Finally, simulation and real-world experiments validate the effectiveness and efficiency of the proposed method.