5 accepted papers
Sparse recovery in linear systems underpins applications from signal processing to high-dimensional regression. Sparse Bayesian Learning, grounded in the principle of automatic relevance determination (ARD), offers a practical Bayesian mechanism for feature sparsity via marginal likelihood optimizat…
We propose a new algorithm enabling stable training under low-precision computations. We call this algorithm Log-normal Multiplicative Dynamics (LMD), and derive it by taking inspiration from the robustness of biological neural networks. Such networks use synapses whose size follow log-normal distri…
LLMs excel at predictive tasks and complex reasoning tasks, but many high-value deployments rely on decisions under uncertainty, for example, which tool to call, which expert to consult, or how many resources to invest. While the usefulness and feasibility of Bayesian approaches remain unclear for L…
Stochastic Variance Reduced Gradient (SVRG) and its variants aim to speed-up training by using gradient corrections. In their decade of existence, these methods have never been connected to any Bayesian methods, at least not at a fundamental level. Here, we fill this gap and show surprising new conn…
The Bayesian Learning Rule provides a framework for generic algorithm design but can be difficult to use for three reasons. First, it requires a specific parameterization of exponential family. Second, it uses gradients which can be difficult to compute. Third, its update may not always stay on the…