2021
ReasonBERT: Pre-trained to Reason with Distant Supervision
EMNLP 2021main
We present ReasonBert, a pre-training method that augments language models with the ability to reason over long-range relations and multiple, possibly hybrid contexts. Unlike existing pre-training methods that only harvest learning signals from local contexts of naturally occurring texts, we propose…