2020
A mathematical model for automatic differentiation in machine learning
NeurIPS 2020spotlight
Automatic differentiation, as implemented today, does not have a simple mathematical model adapted to the needs of modern machine learning. In this work we articulate the relationships between differentiation of programs as implemented in practice, and differentiation of nonsmooth functions. To this…