2023
Expected Gradients of Maxout Networks and Consequences to Parameter Initialization
ICML 2023poster
We study the gradients of a maxout network with respect to inputs and parameters and obtain bounds for the moments depending on the architecture and the parameter distribution. We observe that the distribution of the input-output Jacobian depends on the input, which complicates a stable parameter in…