Michael Kuchnik — accepted AI-conference papers

2026

How Text Quality Interventions Reshape Neural Scaling Laws for LLMs: Empirical Study

Newsha Ardalani, Feiyang Kang, Michael Kuchnik, Mostafa Elhoushi, Shubhabrata Sengupta, Shang-Wen Li +1

ICLR 2026poster

Neural scaling laws are widely used for performance projection and resource planning, yet their sensitivity to data quality interventions remains poorly understood. We present an empirical study of how interventions—deduplication, heuristic filtering, and LLM-guided rewriting—reshape scaling behavi…

Cited by 0 Source ↗Scholar ↗

2026

Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead

Feiyang Kang, Michael Kuchnik, Karthik Padthe, Marin Vlastelica, Ruoxi Jia, Carole-Jean Wu +1

ICLR 2026poster

In post-training for reasoning Large Language Models (LLMs), the current state of practice trains LLMs in two independent stages: Supervised Fine-Tuning (SFT) and Reinforcement Learning with Verifiable Rewards (RLVR, shortened as "RL" below). In this work, we challenge whether high SFT scores transl…

Cited by 0 Source ↗Scholar ↗

2025

AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench

Edan Toledo, Karen Hambardzumyan, Martin Josifoski, RISHI HAZRA, Nicolas Baldwin, Alexis Audran-Reiss +19

NeurIPS 2025spotlight

AI research agents are demonstrating great potential to accelerate scientific progress by automating the design, implementation, and training of machine learning models. We focus on methods for improving agents' performance on MLE-bench, a challenging benchmark where agents compete in Kaggle competi…

Cited by 0 Source ↗code ↗Scholar ↗

2025

Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls

Feiyang Kang, Newsha Ardalani, Michael Kuchnik, Youssef Emad, Mostafa Elhoushi, Shubhabrata Sengupta +4

EMNLP 2025

Training data plays a crucial role in Large Language Models (LLM) scaling, yet high quality data is of limited supply. Synthetic data techniques offer a potential path toward sidestepping these limitations.We conduct a large-scale empirical investigation (>1000 LLMs with >100k GPU hours) using a uni

Cited by 0 Source ↗PDF ↗code ↗Scholar ↗

2024

Croissant: A Metadata Format for ML-Ready Datasets

Mubashara Akhtar, Omar Benjelloun, Costanza Conforti, Luca Foschini, Joan Giner-Miguelez, Pieter Gijsbers +25

NeurIPS 2024spotlight

Data is a critical resource for machine learning (ML), yet working with data remains a key friction point. This paper introduces Croissant, a metadata format for datasets that creates a shared representation across ML tools, frameworks, and platforms. Croissant makes datasets more discoverable, por…

Cited by 37 Source ↗PDF ↗code ↗Scholar ↗

2019

Efficient Augmentation via Data Subsampling

Michael Kuchnik, Virginia Smith

ICLR 2019poster

Data augmentation is commonly used to encode invariances in learning methods. However, this process is often performed in an inefficient manner, as artificial examples are created by applying a number of transformations to all points in the training set. The resulting explosion of the dataset size c…

Cited by 33 Source ↗PDF ↗Scholar ↗