Machines of Finite Depth: Towards a Formalization of Neural Networks
Pietro Vertechi, Mattia G. Bergomi
Abstract
We provide a unifying framework where artificial neural networks and their architectures can be formally described as particular cases of a general mathematical construction---machines of finite depth. Unlike neural networks, machines have a precise definition, from which several properties follow naturally. Machines of finite depth are modular (they can be combined), efficiently computable, and differentiable. The backward pass of a machine is again a machine and can be computed without overhead using the same procedure as the forward pass. We prove this statement theoretically and practically via a unified implementation that generalizes several classical architectures---dense, convolutional, and recurrent neural networks with a rich shortcut structure---and their respective backpropagation rules.
BibTeX
@article{Vertechi_Bergomi_2023, title={Machines of Finite Depth: Towards a Formalization of Neural Networks}, volume={37}, url={https://ojs.aaai.org/index.php/AAAI/article/view/26199}, DOI={10.1609/aaai.v37i8.26199}, abstractNote={We provide a unifying framework where artificial neural networks and their architectures can be formally described as particular cases of a general mathematical construction---machines of finite depth. Unlike neural networks, machines have a precise definition, from which several properties follow naturally. Machines of finite depth are modular (they can be combined), efficiently computable, and differentiable. The backward pass of a machine is again a machine and can be computed without overhead using the same procedure as the forward pass. We prove this statement theoretically and practically via a unified implementation that generalizes several classical architectures---dense, convolutional, and recurrent neural networks with a rich shortcut structure---and their respective backpropagation rules.}, number={8}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Vertechi, Pietro and Bergomi, Mattia G.}, year={2023}, month={Jun.}, pages={10061-10068} }