2017
Neural Taylor Approximations: Convergence and Exploration in Rectifier Networks
ICLR 2017workshop
Modern convolutional networks, incorporating rectifiers and max-pooling, are neither smooth nor convex. Standard guarantees therefore do not apply. Nevertheless, methods from convex optimization such as gradient descent and Adam are widely used as building blocks for deep learning algorithms. This p…