Ryan Andrew Chi — accepted AI-conference papers

2026

Persona Features Control Emergent Misalignment

Miles Wang, Tom Dupre la Tour, Olivia Watkins, Aleksandar Makelov, Ryan Andrew Chi, Samuel Miserendino +5

ICLR 2026poster

Understanding how language models generalize behaviors from their training to a broader deployment distribution is an important problem in AI safety. Betley et al. discovered that fine-tuning GPT-4o on intentionally insecure code causes "emergent misalignment," where models give stereotypically mali…

Cited by 0 Source ↗code ↗Scholar ↗

2024

LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low Resource and Extinct Languages

Andrew Michael Bean, Simeon Hellsten, Harry Mayne, Jabez Magomere, Ethan A Chi, Ryan Andrew Chi +2

NeurIPS 2024oral

In this paper, we present the LingOly benchmark, a novel benchmark for advanced reasoning abilities in large language models. Using challenging Linguistic Olympiad puzzles, we evaluate (i) capabilities for in-context identification and generalisation of linguistic patterns in very low-resource or ex…

Cited by 10 Source ↗PDF ↗code ↗Scholar ↗

2024

Premise Order Matters in Reasoning with Large Language Models

Xinyun Chen, Ryan Andrew Chi, Xuezhi Wang, Denny Zhou

ICML 2024poster

Large language models (LLMs) have accomplished remarkable reasoning performance in various domains. However, in the domain of reasoning tasks, we discover a frailty: LLMs are surprisingly brittle to the ordering of the premises, despite the fact that such ordering does not alter the underlying task.…

Cited by 64 Source ↗PDF ↗Scholar ↗