AIConfPaper
← Search

Seyed Iman Mirzadeh

10 accepted papers

2026

Barriers for Learning in an Evolving World: Mathematical Understanding of Loss of Plasticity

Amir Joudaki, Giulia Lanzillotta, Mohammad Samragh, Seyed Iman Mirzadeh, Keivan Alizadeh, Thomas Hofmann +2
ICLR 2026poster

Deep learning models excel in stationary settings but suffer from loss of plasticity (LoP) in non-stationary environments. While prior literature characterizes LoP through symptoms like rank collapse of representations, it often lacks a mechanistic explanation for why gradient descent fails to recov…

Cited by 0Source ↗code ↗Scholar ↗
2025

GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models

Seyed Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, Mehrdad Farajtabar
ICLR 2025poster

Recent advancements in Large Language Models (LLMs) have sparked interest in their mathematical reasoning capabilities. While performance on the widely popular GSM8K benchmark has improved, questions remain about whether reported evaluation metrics are reliable, and reasoning abilities of LLMs have…

Cited by 209Source ↗PDF ↗Scholar ↗
2025

The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity

Parshin Shojaee, Seyed Iman Mirzadeh, Keivan Alizadeh, Maxwell Horton, Samy Bengio, Mehrdad Farajtabar
NeurIPS 2025poster

Recent generations of frontier language models have introduced Large Reasoning Models (LRMs) that generate detailed thinking processes before providing answers. While these models demonstrate improved performance on reasoning benchmarks, their fundamental capabilities, scaling properties, and limita…

Cited by 0Source ↗Scholar ↗
2025

TiC-LM: A Web-Scale Benchmark for Time-Continual LLM Pretraining

Jeffrey Li, Mohammadreza Armandpour, Seyed Iman Mirzadeh, Sachin Mehta, Vaishaal Shankar, Raviteja Vemulapalli +5
ACL 2025long

Large Language Models (LLMs) trained on historical web data inevitably become outdated. We investigate evaluation strategies and update methods for LLMs as new data becomes available. We introduce a web-scale dataset for time-continual pretraining of LLMs derived from 114 dumps of Common Crawl (CC)…

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2024

LLM in a flash: Efficient Large Language Model Inference with Limited Memory

Keivan Alizadeh, Seyed Iman Mirzadeh, Dmitry Belenko, S. Khatamifard, Minsik Cho, Carlo C Del Mundo +2
ACL 2024long

Large language models (LLMs) are central to modern natural language processing, delivering exceptional performance in various tasks. However, their substantial computational and memory requirements present challenges, especially for devices with limited DRAM capacity. This paper tackles the challeng…

Cited by 113Source ↗PDF ↗Scholar ↗
2024

ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models

Seyed Iman Mirzadeh, Keivan Alizadeh-Vahid, Sachin Mehta, Carlo C del Mundo, Oncel Tuzel, Golnoosh Samei +2
ICLR 2024oral

Large Language Models (LLMs) with billions of parameters have drastically transformed AI applications. However, their demanding computation during inference has raised significant challenges for deployment on resource-constrained devices. Despite recent trends favoring alternative activation functio…

Cited by 85Source ↗PDF ↗Scholar ↗
2022

Wide Neural Networks Forget Less Catastrophically

Seyed Iman Mirzadeh, Arslan Chaudhry, Dong Yin, Huiyi Hu, Razvan Pascanu, Dilan Gorur +1
ICML 2022spotlight

A primary focus area in continual learning research is alleviating the "catastrophic forgetting" problem in neural networks by designing new algorithms that are more robust to the distribution shifts. While the recent progress in continual learning literature is encouraging, our understanding of wha…

Cited by 82Source ↗PDF ↗Scholar ↗
2021

Linear Mode Connectivity in Multitask and Continual Learning

Seyed Iman Mirzadeh, Mehrdad Farajtabar, Dilan Gorur, Razvan Pascanu, Hassan Ghasemzadeh
ICLR 2021poster

Continual (sequential) training and multitask (simultaneous) training are often attempting to solve the same overall objective: to find a solution that performs well on all considered tasks. The main difference is in the training regimes, where continual learning can only have access to one task at…

Cited by 155Source ↗PDF ↗code ↗Scholar ↗
2020

Optimal Policy for Deployment of Machine Learning Models on Energy-Bounded Systems

Seyed Iman Mirzadeh, Hassan Ghasemzadeh
IJCAI 2020poster

With the recent advances in both machine learning and embedded systems research, the demand to deploy computational models for real-time execution on edge devices has increased substantially. Without deploying computational models on edge devices, the frequent transmission of sensor data to the clou…

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2020

Understanding the Role of Training Regimes in Continual Learning

Seyed Iman Mirzadeh, Mehrdad Farajtabar, Razvan Pascanu, Hassan Ghasemzadeh
NeurIPS 2020poster

Catastrophic forgetting affects the training of neural networks, limiting their ability to learn multiple tasks sequentially. From the perspective of the well established plasticity-stability dilemma, neural networks tend to be overly plastic, lacking the stability necessary to prevent the forgettin…

Cited by 263Source ↗PDF ↗code ↗Scholar ↗