The Quotient Bayesian Learning Rule
Mykola Lukashchuk, Raphaël Trésor, Wouter W. L. Nuijten, Ismail Senoz, Bert de Vries
Abstract
This paper introduces the Quotient Bayesian Learning Rule, an extension of natural-gradient Bayesian updates to probability models that fall outside the exponential family. Building on the observation that many heavy-tailed and otherwise non-exponential distributions arise as marginals of minimal exponential families, we prove that such marginals inherit a unique Fisher–Rao information geometry via the quotient-manifold construction. Exploiting this geometry, we derive the Quotient Natural Gradient algorithm, which takes steepest-descent steps in the well-structured covering space, thereby guaranteeing parameterization-invariant optimization in the target space. Empirical results on the Student-$t$ distribution confirm that our method converges more rapidly and attains higher-quality solutions than previous variants of the Bayesian Learning Rule. These findings position quotient geometry as a unifying tool for efficient and principled inference across a broad class of latent-variable models.
BibTeX
@inproceedings{
lukashchuk2025the,
title={The Quotient Bayesian Learning Rule},
author={Mykola Lukashchuk and Rapha{\"e}l Tr{\'e}sor and Wouter W. L. Nuijten and Ismail Senoz and Bert de Vries},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
year={2025},
url={https://openreview.net/forum?id=XDisynd63Y}
}