Learning Properties of Holomorphic Neural Networks of Dual Variables

Abstract

Artificial neural networks have become an inseparable element of human life. Researches do not stop at the current progress and try to improve neural networks and expand fields of applications. The most widespread way to make models better consists in generalization of existing methods and approaches. In this paper, we make a step in an unusual direction: we propose to use neural networks based on dual numbers. We develop a special subclass of dual-valued operators, which satisfy the equivalent of the Cauchy-Riemann equations for the dual domain. We also propose a new type of preprocessing and batch normalization, relying on peculiarities of dual numbers. We test deep holomorphic dual-valued models on music transcription and gravitational wave detection tasks and show that our holomorphic dual-valued networks achieve better inference time compared to the dual-valued models and are better than their real-valued counter-parts in sense of metrics.