Vision-Guided Robotic Grinding with Deep Learning-Based Bead Segmentation and Digital Twin Verification

Abstract

Weld bead grinding is a critical post-processing step in metal fabrication, yet conventional robotic grinding based on teach-pendant programming lacks adaptability to variations in bead geometry and position. This paper presents a vision-guided robotic grinding system that combines deep learning-based weld bead segmentation, automated grinding path generation, and digital twin-based pre-verification. A U-Net model with a ResNet34 encoder and ImageNet pre-training segments weld bead regions from RGB images captured by an Intel RealSense D415 camera mounted on a Staubli RX160 manipulator, achieving a mean Intersection over Union (IoU) of 0.9311 and a Dice coefficient of 0.9641. The segmented bead contours are transformed into the robot coordinate frame through hand-eye calibration and forward kinematics, enabling automated generation of grinding waypoints along the bead centerline. The CHOMP algorithm plans collision-free trajectories within MoveIt, and all planned motions are validated in a digital twin environment built on NVIDIA Isaac Sim 5.0, integrated with ROS through a distributed multi-container architecture. Experimental results demonstrate that the proposed system effectively generates adaptive grinding paths for varying weld bead geometries and verifies them in simulation before physical deployment.