Improving 3D Markerless Pose Estimation of Animals in the Wild using Low-Cost Cameras
Naoya Muramatsu, Zico da Silva, Daniel Joska, Fred Nicolls, Amir Patel
Abstract
Tracking the 3D motion of agile animals in the wild will enable new insight into the design of robotic controllers. However, in-field 3D pose estimation of high-speed wildlife such as cheetahs is still a challenge [1]. In this work, we aim to solve two of these challenges: unnatural pose estimates during highly occluded sequences and synchronization error between multi-view data. We expand on our previous Full Trajectory Estimation (FTE) method with two significant additions: Pairwise FTE (PW-FTE) and Shutter-delay FTE (SD-FTE). The PW-FTE expands on image-dependent pairwise terms, produced by a convolutional neural network (CNN), to infer occluded 2D keypoints, while SD-FTE uses shutter delay estimation to correct the synchronization error. Lastly, we combine both methods into PW-SD-FTE and perform a quantitative and qualitative analysis on a subset of AcinoSet, the video dataset of rapid and agile motions of cheetahs. We found that SD-FTE has significant benefits in tracking the position of the cheetah in the world frame, while PW-FTE provided a more robust 3D pose estimate during events of high occlusion. The PW-SD-FTE was found to retain both advantages, resulting in an improved baseline for AcinoSet. Code and data can be found at https://github.com/African-Robotics-Unit/AcinoSet/tree/pw_sd_fte.
BibTeX
@inproceedings{iros2022_improving3dmarke,
title = {Improving 3D Markerless Pose Estimation of Animals in the Wild using Low-Cost Cameras},
author = {Naoya Muramatsu and Zico da Silva and Daniel Joska and Fred Nicolls and Amir Patel},
booktitle = {IROS 2022},
year = {2022}
}