← Search

Maxwell Crouse

10 accepted papers

2025

NESTFUL: A Benchmark for Evaluating LLMs on Nested Sequences of API Calls

EMNLP 2025

The resurgence of autonomous agents built using large language models (LLMs) to solve complex real-world tasks has brought increased focus on LLMs’ fundamental ability of tool or function calling. At the core of these agents, an LLM must plan, execute, and respond using external tools, APIs, and cus

2024

API-BLEND: A Comprehensive Corpora for Training and Benchmarking API LLMs

ACL 2024long

There is a growing need for Large Language Models (LLMs) to effectively use tools and external Application Programming Interfaces (APIs) to plan and complete tasks. As such, there is tremendous interest in methods that can acquire sufficient quantities of train and test data that involve calls to to…

2024

Granite-Function Calling Model: Introducing Function Calling Abilities via Multi-task Learning of Granular Tasks

EMNLP 2024industry

An emergent research trend explores the use of Large Language Models (LLMs) as the backbone of agentic systems (e.g., SWE-Bench, Agent-Bench). To fulfill LLMs’ potential as autonomous agents, they must be able to identify, call, and interact with a variety of external tools and application program i…

2023

An Ensemble Approach for Automated Theorem Proving Based on Efficient Name Invariant Graph Neural Representations

IJCAI 2023poster

Using reinforcement learning for automated theorem proving has recently received much attention. Current approaches use representations of logical statements that often rely on the names used in these statements and, as a result, the models are generally not transferable from one domain to another.…

2023

Laziness Is a Virtue When It Comes to Compositionality in Neural Semantic Parsing

ACL 2023long

Nearly all general-purpose neural semantic parsers generate logical forms in a strictly top-down autoregressive fashion. Though such systems have achieved impressive results across a variety of datasets and domains, recent works have called into question whether they are ultimately limited in their…

Cited by 3SourcePDFScholar
2023

MISMATCH: Fine-grained Evaluation of Machine-generated Text with Mismatch Error Types

ACL 2023findings

With the growing interest in large language models, the need for evaluating the quality of machine text compared to reference (typically human-generated) text has become focal attention. Most recent works focus either on task-specific evaluation metrics or study the properties of machine-generated t…

2023

Self-Supervised Rule Learning to Link Text Segments to Relational Elements of Structured Knowledge

EMNLP 2023long findings

We present a neuro-symbolic approach to self-learn rules that serve as interpretable knowledge to perform relation linking in knowledge base question answering systems. These rules define natural language text predicates as a weighted mixture of knowledge base paths. The weights learned during train…

Cited by 0SourceScholar
2022

X-FACTOR: A Cross-metric Evaluation of Factual Correctness in Abstractive Summarization

EMNLP 2022main

Abstractive summarization models often produce factually inconsistent summaries that are not supported by the original article. Recently, a number of fact-consistent evaluation techniques have been proposed to address this issue; however, a detailed analysis of how these metrics agree with one anoth…

Cited by 12SourcePDFScholar
2021

A Deep Reinforcement Learning Approach to First-Order Logic Theorem Proving

AAAI 2021technical

Automated theorem provers have traditionally relied on manually tuned heuristics to guide how they perform proof search. Deep reinforcement learning has been proposed as a way to obviate the need for such heuristics, however, its deployment in automated theorem proving remains a challenge. In this p…