Sam Havens — accepted AI-conference papers

2025

FreshStack: Building Realistic Benchmarks for Evaluating Retrieval on Technical Documents

Nandan Thakur, Jimmy Lin, Sam Havens, Michael Carbin, Omar Khattab, Andrew Drozdov

NeurIPS 2025poster

We introduce FreshStack, a holistic framework for automatically building information retrieval (IR) evaluation benchmarks by incorporating challenging questions and answers. FreshStack conducts the following steps: (1) automatic corpus collection from code and technical documentation, (2) nugget gen…

Cited by 0 Source ↗Scholar ↗

2023

MosaicBERT: A Bidirectional Encoder Optimized for Fast Pretraining

Jacob Portes, Alexander R Trott, Sam Havens, DANIEL KING, Abhinav Venigalla, Moin Nadeem +3

NeurIPS 2023poster

Although BERT-style encoder models are heavily used in NLP research, many researchers do not pretrain their own BERTs from scratch due to the high cost of training. In the past half-decade since BERT first rose to prominence, many advances have been made with other transformer architectures and trai…

Cited by 18 Source ↗PDF ↗code ↗Scholar ↗