Proper Velocity Neural Networks
Ziheng Chen, Zihan Su, Bernhard Schölkopf, Nicu Sebe
Abstract
Hyperbolic Neural Networks (HNNs) have shown remarkable success in representing hierarchical and tree-like structures, yet most existing work relies on the Poincaré ball and hyperboloid models. While these models admit closed-form Riemannian operators, their constrained nature potentially leads to numerical instabilities, especially near model boundaries. In this work, we explore the Proper Velocity (PV) space, an unconstrained representation of hyperbolic space rooted in Einstein’s special relativity, as a stable alternative. We first establish the complete Riemannian toolkit of the PV space. Building on this foundation, we introduce Proper Velocity Neural Networks (PVNNs) with core layers including Multinomial Logistic Regression (MLR), Fully Connected (FC), convolutional, activation, and batch normalization layers. Extensive experiments across four tasks, namely numerical stability, image classification, graph node classification, and genomic sequence learning, demonstrate the stability and effectiveness of PVNNs. The code is available at https://github.com/NickyoyoSu/PVNN.
BibTeX
@inproceedings{
chen2026proper,
title={Proper Velocity Neural Networks},
author={Ziheng Chen and Zihan Su and Bernhard Sch{\"o}lkopf and Nicu Sebe},
booktitle={The Fourteenth International Conference on Learning Representations},
year={2026},
url={https://openreview.net/forum?id=UDIYU1X3vC}
}