AoI-MDP: An AoI Optimized Markov Decision Process Dedicated in the Underwater Task (Student Abstract)
Yimian Ding, Jingzehua Xu, Yiyuan Yang, Guanwen Xie, Xinqi Wang, Shuai Zhang
Abstract
Ocean exploration places high demands on autonomous underwater vehicles, especially when there's observation delay. We propose age of information optimized Markov decision process (AoI-MDP) to enhance underwater tasks by modeling observation delay as signal delay and including it in the state space. AoI-MDP also introduces wait time in the action space and integrates AoI with reward functions, optimizing information freshness and decision-making using reinforcement learning. Simulations show AoI-MDP outperforms the standard MDP, demonstrating superior performance, feasibility, and generalization in underwater tasks. To accelerate relevant research, we have made the codes available as open-source at https://github.com/Xiboxtg/AoI-MDP.
BibTeX
@article{Ding_Xu_Yang_Xie_Wang_Zhang_2025, title={AoI-MDP: An AoI Optimized Markov Decision Process Dedicated in the Underwater Task (Student Abstract)}, volume={39}, url={https://ojs.aaai.org/index.php/AAAI/article/view/35247}, DOI={10.1609/aaai.v39i28.35247}, abstractNote={Ocean exploration places high demands on autonomous underwater vehicles, especially when there’s observation delay. We propose age of information optimized Markov decision process (AoI-MDP) to enhance underwater tasks by modeling observation delay as signal delay and including it in the state space. AoI-MDP also introduces wait time in the action space and integrates AoI with reward functions, optimizing information freshness and decision-making using reinforcement learning. Simulations show AoI-MDP outperforms the standard MDP, demonstrating superior performance, feasibility, and generalization in underwater tasks. To accelerate relevant research, we have made the codes available as open-source at https://github.com/Xiboxtg/AoI-MDP.}, number={28}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Ding, Yimian and Xu, Jingzehua and Yang, Yiyuan and Xie, Guanwen and Wang, Xinqi and Zhang, Shuai}, year={2025}, month={Apr.}, pages={29348-29350} }