2021
Learning Specialized Activation Functions With the Piecewise Linear Unit
ICCV 2021poster
The choice of activation functions is crucial for modern deep neural networks. Popular hand-designed activation functions like Rectified Linear Unit(ReLU) and its variants show promising performance in various tasks and models. Swish, the automatically discovered activation function, outperforms ReL…