Xiaoyue Xu — accepted AI-conference papers

2024

Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack

NeurIPS 2024poster

We introduce Lifelong ICL, a problem setting that challenges long-context language models (LMs) to learn a sequence of language tasks through in-context learning (ICL). We further introduce Task Haystack, an evaluation suite dedicated to assessing and diagnosing how long-context LMs utilizes context…

Cited by 4 Source ↗PDF ↗code ↗Scholar ↗

2023

Boosting Inference Efficiency: Unleashing the Power of Parameter-Shared Pre-trained Language Models

Weize Chen, Xiaoyue Xu, Xu Han, Yankai Lin, Ruobing Xie, Zhiyuan Liu +2

EMNLP 2023long findings

Parameter-shared pre-trained language models (PLMs) have emerged as a successful approach in resource-constrained environments, enabling substantial reductions in model storage and memory costs without significant performance compromise. However, it is important to note that parameter sharing does n…

Cited by 0 Source ↗Scholar ↗