Reiji Suda

Train-by-Reconnect: Decoupling Locations of Weights from Their Values

What makes untrained deep neural networks (DNNs) different from the trained performant ones? By zooming into the weights in well-trained DNNs, we found that it is the location of weights that holds most of the information encoded by the training. Motivated by this observation, we hypothesized that w…