An Iterative Min-Min Optimization Method for Sparse Bayesian Learning
Yasen Wang, Junlin Li, Zuogong Yue, ye yuan
Abstract
As a well-known machine learning algorithm, sparse Bayesian learning (SBL) can find sparse representations in linearly probabilistic models by imposing a sparsity-promoting prior on model coefficients. However, classical SBL algorithms lack the essential theoretical guarantees of global convergence. To address this issue, we propose an iterative Min-Min optimization method to solve the marginal likelihood function (MLF) of SBL based on the concave-convex procedure. The method can optimize the hyperparameters related to both the prior and noise level analytically at each iteration by re-expressing MLF using auxiliary functions. Particularly, we demonstrate that the method globally converges to a local minimum or saddle point of MLF. With rigorous theoretical guarantees, the proposed novel SBL algorithm outperforms classical ones in finding sparse representations on simulation and real-world examples, ranging from sparse signal recovery to system identification and kernel regression.
BibTeX
@inproceedings{
wang2024an,
title={An Iterative Min-Min Optimization Method for Sparse Bayesian Learning},
author={Yasen Wang and Junlin Li and Zuogong Yue and ye yuan},
booktitle={Forty-first International Conference on Machine Learning},
year={2024},
url={https://openreview.net/forum?id=69RewQwWA9}
}