Gustav Ragnar Stoettrup Als

Towards Scalable Bayesian Transformers: Investigating stochastic subset selection for NLP

Bayesian deep learning provides a framework for quantifying uncertainty. However, the scale of modern neural networks applied in Natural Language Processing (NLP) limits the usability of Bayesian methods. Subnetwork inference aims to approximate the posterior by selecting a stochastic parameter subs…