3D Human Pose Estimation in Weightless Environments Using a Fisheye Camera

Abstract

Three-dimensional (3D) human pose estimation is one of the most basic tasks for human-interacting robots. Especially in weightless environments such as the International Space Station (ISS), wherein objects may move with a higher degree of freedom compared to on the ground, a camera with a wider field of view (FOV) is crucial in improving the probability of capturing surrounding humans. To this end, we propose a learning-based 3D human pose estimation (3D-HPE) using a fisheye camera targeted at weightless environments. One impediment is that 3D-HPE trained on existing datasets are incapable of addressing the adverse effects of strong fisheye distortion and weightlessness as existing human detection and pose estimation datasets are recorded on the ground using typical rectilinear cameras. To overcome this difficulty, we integrate virtual camera projection to perform a detected human-centered undistortion from fisheye to rectilinear images. We also include upside-down augmentation during training to improve the performance toward weightlessness. Our results show that these two techniques successfully mitigate the adverse effects of weightlessness and fisheye distortion.