2020
Information Geometry of Orthogonal Initializations and Training
ICLR 2020poster
Recently mean field theory has been successfully used to analyze properties of wide, random neural networks. It gave rise to a prescriptive theory for initializing feed-forward neural networks with orthogonal weights, which ensures that both the forward propagated activations and the bac…