Deep Reinforcement Learning-Based Levitation Control of Wireless Capsule Endoscope by Robotically Driven Permanent Magnet

Abstract

Magnetic levitation control provides a promising solution for wireless capsule endoscopy by minimizing tissue pressure and reducing patient discomfort and risks. Compared to electromagnetic actuation systems, using permanent magnets as the actuation source provides stronger magnetic fields at a lower cost. However, permanent magnet-based actuation systems are highly nonlinear and necessitate complex system modeling. In this study, we propose a deep reinforcement learning (DRL)-based control method for permanent magnet levitation. This approach utilizes DRL to learn optimal control strategies in complex and dynamic environments, without the need for detailed modeling of nonlinear physical phenomena such as magnetic interactions, manipulator dynamics, and capsule-environment interactions. A simulation environment was developed where a manipulator equipped with a permanent magnet actuates the internal magnet of a capsule. A multistage reward function and a recurrent neural network with memory capabilities were designed to improve control stability and accuracy. After Sim-to-Sim transfer, the proposed method successfully controlled five degrees of freedom, achieving navigation accuracies of 1.68 mm in the training environment and 9.18 mm in the testing environment. The system maintained stable performance and high accuracy while supporting dynamic tracking at speeds of up to 30 mm/s. Additionally, the method demonstrated significant resistance to disturbances.