Q-Learning-based Optimal Force-Tracking Control of Grinding Robots in Uncertain Environments

Abstract

This paper proposes a novel Q-learning-based dual-loop force tracking control framework for robot grinding tasks in uncertain environments. A complete system state-space model is established, incorporating interaction dynamics and the desired force. By augmenting the system state, a discount cost function is defined to quantify the tracking errors of the force and reference trajectory. The modified Q-learning method is systematically designed to iteratively compute the optimal control gain in a model-free manner. To mitigate force overshoot during the transition from free space to contact space, a force reference model and a transition mechanism for the control gain are designed. Simulations and experiments validate the method’s effectiveness in precise force tracking with minimal overshoot and robustness to environmental variations.