John Higgins

Set Norm and Equivariant Skip Connections: Putting the Deep in Deep Sets

Permutation invariant neural networks are a promising tool for predictive modeling of set data. We show, however, that existing architectures struggle to perform well when they are deep. In this work, we mathematically and empirically analyze normalization layers and residual connections in the cont…