AAAI 2026technical0 citations
Unpacking the Implicit Norm Dynamics of Sharpness-Aware Minimization in Tensorized Models
Tianxiao Cao, Kyohei Atarashi, Hisashi Kashima
Abstract
Sharpness-Aware Minimization (SAM) has been proven to be an effective optimization technique for improving generalization in overparameterized models. While prior works have explored the implicit regularization of SAM in simple two-core scale-invariant settings, its behavior in more general tensorized or scale-invariant models remains underexplored. In this work, we leverage scale-invariance to analyze the norm dynamics of SAM in general tensorized models. We introduce the notion of Norm Deviation as a global measure of core norm imbalance, and derive its evolution under SAM using gradient flow analysis. We show that SAM
BibTeX
@inproceedings{aaai2026_unpackingtheimpl,
title = {Unpacking the Implicit Norm Dynamics of Sharpness-Aware Minimization in Tensorized Models},
author = {Tianxiao Cao and Kyohei Atarashi and Hisashi Kashima},
booktitle = {AAAI 2026},
year = {2026}
}