A Spatiotemporal Brain Activity Visualization and Assessment Framework for Human-Robot Cognitive Interaction Training

Abstract

Accurately assessing brain activity to modulate training parameters online is crucial for improving the human-robot cognitive interaction (HRCI) performance in closed-loop brain training. The major challenge for this technique lies in how to accurately model and characterize the intrinsic behavior of brain activity in HRCI process, which typically exhibits a dynamic manner across spatial and temporal scales. In this study, we propose a dynamic perspective to visualize the spatiotemporal evolution of brain activity during HRCI process, thus enabling assessment of brain states and adaptive modulation during rehabilitation. A novel framework is developed to model the spatiotemporal dynamics of brain activity by integrating deterministic learning with neural population theory. It demonstrates a remarkable capability to mine and visualize the complex nonlinear dynamics of brain activity, encompassing both temporal evolution and spatial connectivity patterns. The proposed model not only visualizes of spatiotemporal brain dynamics but also enables online assessment of brain states, which can facilitate optimal modulation of HRCI process and improve the brain training efficiency. The method is validated using a panoramic virtual reality system. Results show that our method improves the accuracy of brain activity assessment by 8.86%, effectively demonstrating that it accurately visualizes spatiotemporal brain dynamics and enhances training outcomes when integrated with HRCI.