ICLR 2020poster4 citations
Logic and the 2-Simplicial Transformer
James Clift, Dmitry Doryn, Daniel Murfet, James Wallbridge
Abstract
We introduce the 2-simplicial Transformer, an extension of the Transformer which includes a form of higher-dimensional attention generalising the dot-product attention, and uses this attention to update entity representations with tensor products of value vectors. We show that this architecture is a useful inductive bias for logical reasoning in the context of deep reinforcement learning.
transformerlogicreinforcement learningreasoning
BibTeX
@inproceedings{
Clift2020Logic,
title={Logic and the 2-Simplicial Transformer},
author={James Clift and Dmitry Doryn and Daniel Murfet and James Wallbridge},
booktitle={International Conference on Learning Representations},
year={2020},
url={https://openreview.net/forum?id=rkecJ6VFvr}
}