AIConfPaper
← Search

Abulhair Saparov

12 accepted papers

2025

Are Language Models Efficient Reasoners? A Perspective from Logic Programming

Andreas Opedal, Yanick Zengaffinen, Haruki Shirakami, Clemente Pasti, Mrinmaya Sachan, Abulhair Saparov +2
NeurIPS 2025poster

Modern language models (LMs) exhibit strong deductive reasoning capabilities, yet standard evaluations emphasize correctness while overlooking a key aspect of human-like reasoning: *efficiency*. In real-world reasoning scenarios, much of the available information is irrelevant, and effective deducti…

Cited by 0Source ↗Scholar ↗
2025

MathGAP: Out-of-Distribution Evaluation on Problems with Arbitrarily Complex Proofs

Andreas Opedal, Haruki Shirakami, Bernhard Schölkopf, Abulhair Saparov, Mrinmaya Sachan
ICLR 2025poster

Large language models (LLMs) can solve arithmetic word problems with high accuracy, but little is known about how well they generalize to more complex problems. This is difficult to study, as (i) much of the available evaluation data has already been seen by the most capable models during training,…

Cited by 3Source ↗PDF ↗Scholar ↗
2025

Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective

Zhoujun Cheng, Shibo Hao, Tianyang Liu, Fan Zhou, Yutao Xie, Feng Yao +18
NeurIPS 2025poster

Reinforcement learning (RL) has shown promise in enhancing large language model (LLM) reasoning, yet progress towards broader capabilities is limited by the availability of high-quality, multi-domain datasets. This work introduces \ours, a 92K RL-for-reasoning dataset designed to address this gap, c…

Cited by 0Source ↗Scholar ↗
2025

Transformers Struggle to Learn to Search

Abulhair Saparov, Srushti Ajay Pawar, Shreyas Pimpalgaonkar, Nitish Joshi, Richard Yuanzhe Pang, Vishakh Padmakumar +3
ICLR 2025poster

Search is an ability foundational in many important tasks, and recent studies have shown that large language models (LLMs) struggle to perform search robustly. It is unknown whether this inability is due to a lack of data, insufficient model parameters, or fundamental limitations of the transformer…

Cited by 1Source ↗PDF ↗code ↗Scholar ↗
2024

Do Language Models Exhibit the Same Cognitive Biases in Problem Solving as Human Learners?

Andreas Opedal, Alessandro Stolfo, Haruki Shirakami, Ying Jiao, Ryan Cotterell, Bernhard Schölkopf +2
ICML 2024poster

There is increasing interest in employing large language models (LLMs) as cognitive models. For such purposes, it is central to understand which properties of human cognition are well-modeled by LLMs, and which are not. In this work, we study the biases of LLMs in relation to those known in children…

Cited by 16Source ↗PDF ↗code ↗Scholar ↗
2024

Personas as a Way to Model Truthfulness in Language Models

Nitish Joshi, Javier Rando, Abulhair Saparov, Najoung Kim, He He
EMNLP 2024main

Large language models (LLMs) are trained on vast amounts of text from the internet, which contains both factual and misleading information about the world. While unintuitive from a classic view of LMs, recent work has shown that the truth value of a statement can be elicited from the model’s represe…

Cited by 28Source ↗PDF ↗code ↗Scholar ↗
2023

Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought

Abulhair Saparov, He He
ICLR 2023poster

Large language models (LLMs) have shown remarkable reasoning capabilities given chain-of-thought prompts (examples with intermediate reasoning steps). Existing benchmarks measure reasoning ability indirectly, by evaluating accuracy on downstream tasks such as mathematical reasoning. However, it is u…

Cited by 279Source ↗PDF ↗code ↗Scholar ↗
2023

Noisy Exemplars Make Large Language Models More Robust: A Domain-Agnostic Behavioral Analysis

Hongyi Zheng, Abulhair Saparov
EMNLP 2023short main

Recent advances in prompt engineering enable large language models (LLMs) to solve multi-hop logical reasoning problems with impressive accuracy. However, there is little existing work investigating the robustness of LLMs with few-shot prompting techniques. Therefore, we introduce a systematic appro…

Cited by 0Source ↗code ↗Scholar ↗
2023

Testing the General Deductive Reasoning Capacity of Large Language Models Using OOD Examples

Abulhair Saparov, Richard Yuanzhe Pang, Vishakh Padmakumar, Nitish Joshi, Mehran Kazemi, Najoung Kim +1
NeurIPS 2023poster

Given the intractably large size of the space of proofs, any model that is capable of general deductive reasoning must generalize to proofs of greater complexity. Recent studies have shown that large language models (LLMs) possess some abstract deductive reasoning ability given chain-of-thought prom…

Cited by 79Source ↗PDF ↗code ↗Scholar ↗
2020

Jelly Bean World: A Testbed for Never-Ending Learning

Emmanouil Antonios Platanios, Abulhair Saparov, Tom Mitchell
ICLR 2020poster

Machine learning has shown growing success in recent years. However, current machine learning systems are highly specialized, trained for particular problems or domains, and typically on a single narrow dataset. Human learning, on the other hand, is highly general and adaptable. Never-ending learnin…

Cited by 40Source ↗code ↗Scholar ↗