2025
Investigating the Overlooked Hessian Structure: From CNNs to LLMs
ICML 2025poster
It is well-known that the Hessian of deep loss landscape matters to optimization and generalization of deep learning. Previous studies reported a rough Hessian structure in deep learning, which consists of two components, a small number of large eigenvalues and a large number of nearly-zero eigenval…