3DGS-Holo-Inspector: A Mixed Reality UAV Controller with 3D Gaussian Splatting Localization for Infrastructure Inspection

Abstract

Unmanned aerial vehicles (UAVs) are increasingly used for infrastructure inspection, but conventional joystick and first-person-view (FPV) controllers remain unintuitive, error-prone, and cognitively demanding, particularly in cluttered or safety-critical environments. We present 3DGS-Holo- Inspector, a Mixed Reality (MR) UAV controller that combines holographic goal-setting with autonomous UAV navigation. Using natural hand gestures, operators can define and preview navigation goals directly in MR before flight, ensuring precise and safe data capture at inspection viewpoints. The system complements existing inspection pipelines by leveraging pre-built 3D maps (e.g., photogrammetry or LiDAR reconstructions) to enable refinement of regions of interest (ROIs) where coverage is incomplete or the detail is insufficient. Robust headset–UAV alignment is achieved through a LiDAR–RGB 3D Gaussian Splatting (3DGS) localization backbone, which provides dense, markerless, and persistent spatial registration in both indoor and outdoor settings. Once goals are placed, the UAV autonomously navigates to the specified pose, with real-time telemetry and live video overlaid in MR to enhance situational awareness. Experimental validation using a ModalAI Starling UAV and Microsoft HoloLens 2 demonstrated accurate UAV-goal alignment, achieving a positional Root Mean Square Error (RMSE) of 0.090 m (median = 0.084 m) indoors and 0.119 m (median = 0.118 m) outdoors, with orientation (yaw) RMSEs of 1.491 ◦ (median = 1.400 ◦ ) and 2.233 ◦ (median = 2.268 ◦ ), respectively. These results confirm that 3DGS-Holo-Inspector provides reliable MR-based UAV control, augmenting inspection workflows by enabling safe, intuitive, and high-precision UAV operations in real-world environments.