Shengxiang Deng

Hypergradient-free Training for Deep Equilibrium Models

Deep Equilibrium (DEQ) models hold great promise for expanding the capabilities of neural networks and showcasing competitive performances with less memory cost. However, the backward propagation steps in their training process typically involve expensive Jacobian-inverse calculations, resulting in…