Continual Learning with Bayesian Neural Networks for Non-Stationary Data
Richard Kurle, Botond Cseke, Alexej Klushyn, Patrick van der Smagt, Stephan Günnemann
Abstract
This work addresses continual learning for non-stationary data, using Bayesian neural networks and memory-based online variational Bayes. We represent the posterior approximation of the network weights by a diagonal Gaussian distribution and a complementary memory of raw data. This raw data corresponds to likelihood terms that cannot be well approximated by the Gaussian. We introduce a novel method for sequentially updating both components of the posterior approximation. Furthermore, we propose Bayesian forgetting and a Gaussian diffusion process for adapting to non-stationary data. The experimental results show that our update method improves on existing approaches for streaming data. Additionally, the adaptation methods lead to better predictive performance for non-stationary data.
BibTeX
@inproceedings{
Kurle2020Continual,
title={Continual Learning with Bayesian Neural Networks for Non-Stationary Data},
author={Richard Kurle and Botond Cseke and Alexej Klushyn and Patrick van der Smagt and Stephan Günnemann},
booktitle={International Conference on Learning Representations},
year={2020},
url={https://openreview.net/forum?id=SJlsFpVtDB}
}