Hemanth Saratchandran*

Weight Conditioning for Smooth Optimization of Neural Networks

"In this article, we introduce a novel normalization technique for neural network weight matrices, which we term weight conditioning. This approach aims to narrow the gap between the smallest and largest singular values of the weight matrices, resulting in better-conditioned matrices. The inspiratio…