← Search

Saeyoon Oh

2 accepted papers

2021

Transformers Generalize DeepSets and Can be Extended to Graphs & Hypergraphs

NeurIPS 2021poster

We present a generalization of Transformers to any-order permutation invariant data (sets, graphs, and hypergraphs). We begin by observing that Transformers generalize DeepSets, or first-order (set-input) permutation invariant MLPs. Then, based on recently characterized higher-order invariant MLPs,…