Zain Sarwar — accepted AI-conference papers

2026

Critique-Guided Distillation for Robust Reasoning via Refinement

Berkcan Kapusuzoglu, Supriyo Chakraborty, Zain Sarwar, Michael Lee, Sambit Sahu

ICML 2026poster

Supervised fine-tuning with expert demonstrations often produces models that imitate outputs without internalizing the reasoning processes needed for robust generalization. While critique-based approaches show promise, training models to generate critiques directly, such as Critique Fine-Tuning (CFT…

Cited by 0 Source ↗Scholar ↗

2025

Dense Backpropagation Improves Training for Sparse Mixture-of-Experts

Ashwinee Panda, Vatsal Baherwani, Zain Sarwar, Benjamin Thérien, Sambit Sahu, Tom Goldstein +1

NeurIPS 2025poster

Mixture of Experts (MoE) pretraining is more scalable than dense Transformer pretraining, because MoEs learn to route inputs to a sparse set of their feedforward parameters. However, this means that MoEs only receive a sparse backward update, leading to training instability and suboptimal performanc…

Cited by 0 Source ↗code ↗Scholar ↗