Nguyen Duy Khuong — accepted AI-conference papers

2023

Reusable Slotwise Mechanisms

Trang Nguyen, Amin Mansouri, Kanika Madan, Nguyen Duy Khuong, Kartik Ahuja, Dianbo Liu +1

NeurIPS 2023poster

Agents with the ability to comprehend and reason about the dynamics of objects would be expected to exhibit improved robustness and generalization in novel scenarios. However, achieving this capability necessitates not only an effective scene representation but also an understanding of the mechanism…

Cited by 4 Source ↗PDF ↗Scholar ↗

2022

Improving Transformer with an Admixture of Attention Heads

Tan Minh Nguyen, Tam Minh Nguyen, Hai Ngoc Do, Khai Nguyen, Vishwanath Saragadam, Minh Pham +3

NeurIPS 2022accept

Transformers with multi-head self-attention have achieved remarkable success in sequence modeling and beyond. However, they suffer from high computational and memory complexities for computing the attention matrix at each head. Recently, it has been shown that those attention matrices lie on a low-d…

Cited by 29 Source ↗PDF ↗Scholar ↗