Spatio-Temporal Hyperbolic Aggregation Neural Network for Human Action Recognition

Abstract

Human action recognition (HAR) is a critical task in the field of robotics. Traditionally, HAR methods rely on either perceptual features from RGB images or skeletal features. While RGB-based features are typically represented in 2D Euclidean space, few approaches differentiate between methods developed for RGB data and those for skeletal features, often treating both as Euclidean representations. This conventional approach, which typically leverages standard deep learning techniques, limits the descriptive power of skeletal data, which naturally exhibits a tree-like structure. In this paper, we introduce a novel framework that, for the first time, utilizes skeletal data while preserving its inherent structure to fully capture its descriptive potential. Our proposed deep neural network embeds skeletal joints into hyperbolic space, followed by a spatio-temporal processing framework that incorporates established transformations to optimize performance while maintaining the advantages of hyperbolic analysis. Extensive experiments on publicly available datasets, including UAV-Human, UAV-Gesture, and DHG 14/28, demonstrate that our approach achieves state-of-the-art results, underscoring its ability to enhance robotic systems’ performance in dynamic environments.