Jonghyun Shin

Minimum Width for Universal Approximation using Squashable Activation Functions

The exact minimum width that allows for universal approximation of unbounded-depth networks is known only for ReLU and its variants. In this work, we study the minimum width of networks using general activation functions. Specifically, we focus on squashable functions that can approximate the identi…