← Search

Z.y. Peng

5 accepted papers

2025

Can Large Language Models Detect Errors in Long Chain-of-Thought Reasoning?

ACL 2025long

Recently, o1-like models have drawn significant attention, where these models produce the long Chain-of-Thought (CoT) reasoning steps to improve the reasoning abilities of existing Large Language Models (LLMs). In this paper, to understand the qualities of these long CoTs and measure the critique ab…

2025

MTU-Bench: A Multi-granularity Tool-Use Benchmark for Large Language Models

ICLR 2025poster

Large Language Models (LLMs) have displayed massive improvements in reason- ing and decision-making skills and can hold natural conversations with users. Recently, many tool-use benchmark datasets have been proposed. However, existing datasets have the following limitations: (1). Insufficient evalua…

2025

SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplines

NeurIPS 2025poster

Large language models (LLMs) have demonstrated remarkable proficiency in mainstream academic disciplines such as mathematics, physics, and computer science. However, human knowledge encompasses over 200 specialized disciplines, far exceeding the scope of existing benchmarks. The capabilities of LLMs…

Cited by 215SourceScholar
2024

RoleAgent: Building, Interacting, and Benchmarking High-quality Role-Playing Agents from Scripts

NeurIPS 2024poster

Believable agents can empower interactive applications ranging from immersive environments to rehearsal spaces for interpersonal communication. Recently, generative agents have been proposed to simulate believable human behavior by using Large Language Models. However, the existing method heavily re…

Cited by 1SourcePDFScholar
2024

RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models

ACL 2024findings

The advent of Large Language Models (LLMs) has paved the way for complex tasks such as role-playing, which enhances user interactions by enabling models to imitate various characters. However, the closed-source nature of state-of-the-art LLMs and their general-purpose training limit role-playing opt…