AIConfPaper
ICML 2022oral56 citations

A Dynamical System Perspective for Lipschitz Neural Networks

Laurent Meunier, Blaise J Delattre, Alexandre Araujo, Alexandre Allauzen

Source ↗PDF ↗Scholar ↗TeX ↗

Abstract

The Lipschitz constant of neural networks has been established as a key quantity to enforce the robustness to adversarial examples. In this paper, we tackle the problem of building $1$-Lipschitz Neural Networks. By studying Residual Networks from a continuous time dynamical system perspective, we provide a generic method to build $1$-Lipschitz Neural Networks and show that some previous approaches are special cases of this framework. Then, we extend this reasoning and show that ResNet flows derived from convex potentials define $1$-Lipschitz transformations, that lead us to define the

BibTeX
@InProceedings{pmlr-v162-meunier22a,
  title = 	 {A Dynamical System Perspective for {L}ipschitz Neural Networks},
  author =       {Meunier, Laurent and Delattre, Blaise J and Araujo, Alexandre and Allauzen, Alexandre},
  booktitle = 	 {Proceedings of the 39th International Conference on Machine Learning},
  pages = 	 {15484--15500},
  year = 	 {2022},
  editor = 	 {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan},
  volume = 	 {162},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {17--23 Jul},
  publisher =    {PMLR},
  pdf = 	 {https://proceedings.mlr.press/v162/meunier22a/meunier22a.pdf},
  url = 	 {https://proceedings.mlr.press/v162/meunier22a.html},
  abstract = 	 {The Lipschitz constant of neural networks has been established as a key quantity to enforce the robustness to adversarial examples. In this paper, we tackle the problem of building $1$-Lipschitz Neural Networks. By studying Residual Networks from a continuous time dynamical system perspective, we provide a generic method to build $1$-Lipschitz Neural Networks and show that some previous approaches are special cases of this framework. Then, we extend this reasoning and show that ResNet flows derived from convex potentials define $1$-Lipschitz transformations, that lead us to define the
A Dynamical System Perspective for Lipschitz Neural Networks · ICML 2022