Yu Xin Li

Fishers for Free? Approximating the Fisher Information Matrix by Recycling the Squared Gradient Accumulator

The diagonal of a model's Fisher Information Matrix (the "Fisher") has frequently been used as a way to measure parameter sensitivity. Typically, the Fisher is estimated by computing the squared gradient of the model's outputs with respect to its parameters, averaged over a few hundred or thousand e…