Dynamics-Based Visual Tracking of Robot Pose With Camera Extrinsic Online Calibration
Jingxuan Zhang, Kai Nie, Beixian Lai, Zhiwen Li, Yongping Pan
Abstract
Visual servoing provides the flexibility of robot control in dynamic environments, but three-dimensional (3-D) robot visual servoing is challenging due to the 2-D nature of the image space. Adaptive homography-based visual servoing (HBVS) is effective for 3-D robot pose control under various parametric uncertainties, but the robot dynamics is seldom considered under uncalibrated camera parameters. This paper proposes a dynamics-based adaptive HBVS method for 3-D pose tracking of robot manipulators using an eye-to-hand monocular camera with unknown extrinsic parameters, where a composite learning law is designed to achieve parameter convergence without the stringent persistent excitation condition. Compared with existing adaptive HBVS methods, the proposed method accounts for robot dynamics in the control design to enhance control performance, calibrates extrinsic parameters online under a weak condition of interval excitation, and does not require measuring the rotation of a reference plane or finding valid vanishing points. Experiments on an articulated robot with 7 degrees of freedom, named Franka Emika Panda, have verified that the proposed method performs well for both camera extrinsic online calibration and robot 3-D pose tracking.
BibTeX
@inproceedings{ral2026_dynamicsbasedvis,
title = {Dynamics-Based Visual Tracking of Robot Pose With Camera Extrinsic Online Calibration},
author = {Jingxuan Zhang and Kai Nie and Beixian Lai and Zhiwen Li and Yongping Pan},
booktitle = {RA-L 2026},
year = {2026}
}