Convex Optimization of Deep Polynomial and ReLU Activation Neural Networks

Burak Bartan; Mert Pilanci

ICASSP 2023accepted0 citations

Convex Optimization of Deep Polynomial and ReLU Activation Neural Networks

Abstract

We consider training multi-layer neural networks with polynomial and ReLU activation functions. We develop exact convex optimization formulations for three-layer and deeper architectures. Our formulations are based on semidefinite lifting and recent results on the hidden convexity of two-layer ReLU networks. We show that certain deep neural network architectures can be trained to global optimality in polynomial time by solving equivalent convex optimization problems.

BibTeX

@inproceedings{icassp2023_convexoptimizati,
  title = {Convex Optimization of Deep Polynomial and ReLU Activation Neural Networks},
  author = {Burak Bartan and Mert Pilanci},
  booktitle = {ICASSP 2023},
  year = {2023}
}