Yuxi CAI — accepted AI-conference papers

2023

Cross-Layer Retrospective Retrieving via Layer Attention

Yanwen Fang, Yuxi CAI, Jintai Chen, Jingyu Zhao, Guangjian Tian, Guodong Li

ICLR 2023poster

More and more evidence has shown that strengthening layer interactions can enhance the representation power of a deep neural network, while self-attention excels at learning interdependencies by retrieving query-activated information. Motivated by this, we devise a cross-layer attention mechanism, c…

Cited by 15 Source ↗PDF ↗code ↗Scholar ↗

2023

Encoding Recurrence into Transformers

Feiqing Huang, Kexin Lu, Yuxi CAI, Zhen Qin, Yanwen Fang, Guangjian Tian +1

ICLR 2023top-5%

This paper novelly breaks down with ignorable loss an RNN layer into a sequence of simple RNNs, each of which can be further rewritten into a lightweight positional encoding matrix of a self-attention, named the Recurrence Encoding Matrix (REM). Thus, recurrent dynamics introduced by the RNN layer c…

Cited by 19 Source ↗PDF ↗Scholar ↗