AIConfPaper
← Search

Xiaozhe Yao

5 accepted papers

2025

Aurora-M: Open Source Continual Pre-training for Multilingual Language and Code

Taishi Nakamura, Mayank Mishra, Simone Tedeschi, Yekun Chai, Jason T. Stillerman, Felix Friedrich +35
COLING 2025industry

Pretrained language models are integral part of AI applications, but their high computational cost for training limits accessibility. Initiatives such as Bloom and StarCoder aim to democratize access to pretrained models for collaborative community development. Despite these efforts, such models enc…

Cited by 2Source ↗PDF ↗Scholar ↗
2025

Demystifying Cost-Efficiency in LLM Serving over Heterogeneous GPUs

YOUHE JIANG, Fangcheng Fu, Xiaozhe Yao, Guoliang HE, Xupeng Miao, Ana Klimovic +3
ICML 2025poster

Recent advancements in Large Language Models (LLMs) have led to increasingly diverse requests, accompanied with varying resource (compute and memory) demands to serve them. However, this in turn degrades the cost-efficiency of LLM serving as common practices primarily rely on homogeneous GPU resourc…

Cited by 4Source ↗PDF ↗Scholar ↗
2024

HexGen: Generative Inference of Large Language Model over Heterogeneous Environment

YOUHE JIANG, Ran Yan, Xiaozhe Yao, Yang Zhou, Beidi Chen, Binhang Yuan
ICML 2024poster

Serving generative inference of the large language model is a crucial component of contemporary AI applications. In this paper, our focus lies in deploying such services in a heterogeneous and cross-datacenter setting to mitigate the substantial inference costs typically associated with a single cen…

Cited by 9Source ↗PDF ↗code ↗Scholar ↗
2024

RedPajama: an Open Dataset for Training Large Language Models

Maurice Weber, Daniel Y Fu, Quentin Gregory Anthony, Yonatan Oren, Shane Adams, Anton Alexandrov +13
NeurIPS 2024spotlight

Large language models are increasingly becoming a cornerstone technology in artificial intelligence, the sciences, and society as a whole, yet the optimal strategies for dataset composition and filtering remain largely elusive. Many of the top-performing models lack transparency in their dataset cur…

Cited by 49Source ↗PDF ↗code ↗Scholar ↗
2023

DataPerf: Benchmarks for Data-Centric AI Development

Mark Mazumder, Colby Banbury, Xiaozhe Yao, Bojan Karlaš, William A Gaviria Rojas, Sudnya Diamos +41
NeurIPS 2023poster

Machine learning research has long focused on models rather than datasets, and prominent datasets are used for common ML tasks without regard to the breadth, difficulty, and faithfulness of the underlying problems. Neglecting the fundamental importance of data has given rise to inaccuracy, bias, and…

Cited by 147Source ↗PDF ↗code ↗Scholar ↗