← Search

John J Nay

2 accepted papers

2025

Language Models can Subtly Deceive Without Lying: A Case Study on Strategic Phrasing in Legislation

ACL 2025long

We explore the ability of large language models (LLMs) to engage in subtle deception through strategically phrasing and intentionally manipulating information. This harmful behavior can be hard to detect, unlike blatant lying or unintentional hallucination. We build a simple testbed mimicking a legi…

2023

LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models

NeurIPS 2023poster

The advent of large language models (LLMs) and their adoption by the legal community has given rise to the question: what types of legal reasoning can LLMs perform? To enable greater study of this question, we present LegalBench: a collaboratively constructed legal reasoning benchmark consisting of…