Antoine Venaille

A New Initialization to Control Gradients in Sinusoidal Neural Networks

Proper initialisation strategy is of primary importance to mitigate gradient explosion or vanishing when training neural networks. Yet, the impact of initialisation parameters still lacks a precise theoretical understanding for several well-established architectures. Here, we propose a new initialis…