DS SERVE: A Framework for Efficient and Scalable Neural Retrieval

Jinjian Liu; Yichuan Wang; Xinxi Lyu; Rulin Shao; Joseph E. Gonzalez; Matei Zaharia; Sewon Min

AAAI 2026technical0 citations

DS SERVE: A Framework for Efficient and Scalable Neural Retrieval

Jinjian Liu, Yichuan Wang, Xinxi Lyu, Rulin Shao, Joseph E. Gonzalez, Matei Zaharia, Sewon Min

Abstract

We present DS SERVE, a framework that transforms large-scale text datasets—comprising half a trillion tokens—into a high-performance neural retrieval system. DS SERVE offers both a web interface and API endpoints, achieving low latency with modest memory overhead on a single node. The framework also supports inference-time tradeoffs between latency, accuracy, and result diversity. We anticipate that DS SERVE will be broadly useful for a range of applications such as large-scale retrieval-augmented generation (RAG), training data attribution, training a search agent, and beyond.

BibTeX

@inproceedings{aaai2026_dsserveaframewor,
  title = {DS SERVE: A Framework for Efficient and Scalable Neural Retrieval},
  author = {Jinjian Liu and Yichuan Wang and Xinxi Lyu and Rulin Shao and Joseph E. Gonzalez and Matei Zaharia and Sewon Min},
  booktitle = {AAAI 2026},
  year = {2026}
}