Alexey Voronin

On the Convergence Behavior of Preconditioned Gradient Descent Toward the Rich Learning Regime

Spectral bias, the tendency of neural networks to learn low frequencies first, can be both a blessing and a curse. While it enhances the generalization capabilities by suppressing high-frequency noise, it can be a limitation in scientific tasks that require capturing fine-scale structures. The delay…