Sean Michael McLeish — accepted AI-conference papers

2025

Gemstones: A Model Suite for Multi-Faceted Scaling Laws

Sean Michael McLeish, John Kirchenbauer, David Yu Miller, Siddharth Singh, Abhinav Bhatele, Micah Goldblum +2

NeurIPS 2025poster

Scaling laws are typically fit using a family of models with a narrow range of frozen hyperparameter choices. In this work we study scaling laws using multiple architectural shapes and hyperparameter choices, highlighting their impact on resulting prescriptions. As a primary artifact of our researc…

Cited by 0 Source ↗code ↗Scholar ↗

2025

Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach

Jonas Geiping, Sean Michael McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R. Bartoldson +3

NeurIPS 2025spotlight

We study a novel language model architecture that is capable of scaling test-time computation by implicitly reasoning in latent space. Our model works by iterating a recurrent block, thereby unrolling to arbitrary depth at test-time. This stands in contrast to mainstream reasoning models that scale…

Cited by 0 Source ↗code ↗Scholar ↗

2024

Transformers Can Do Arithmetic with the Right Embeddings

Sean Michael McLeish, Arpit Bansal, Alex Stein, Neel Jain, John Kirchenbauer, Brian R. Bartoldson +5

NeurIPS 2024poster

The poor performance of transformers on arithmetic tasks seems to stem in large part from their inability to keep track of the exact position of each digit inside of a large span of digits. We mend this problem by adding an embedding to each digit that encodes its position relative to the start of t…

Cited by 12 Source ↗PDF ↗code ↗Scholar ↗