← Search

Isabel Papadimitriou

7 accepted papers

2025

Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models

ICML 2025poster

Sparse Autoencoders (SAEs) have emerged as a powerful framework for machine learning interpretability, enabling the unsupervised decomposition of model representations into a dictionary of abstract, human-interpretable concepts. However, we reveal a fundamental limitation: SAEs exhibit severe instab…

Cited by 2SourcePDFScholar
2025

Investigating the interaction of linguistic and mathematical reasoning in language models using multilingual number puzzles

EMNLP 2025

Across languages, numeral systems vary widely in how they construct and combine numbers. While humans consistently learn to navigate this diversity, large language models (LLMs) struggle with linguistic-mathematical puzzles involving cross-linguistic numeral systems, which humans can learn to solve

Cited by 0SourcePDFScholar
2025

Using Shapley interactions to understand how models use structure

ACL 2025long

Language is an intricately structured system, and a key goal of NLP interpretability is to provide methodological insights for understanding how language models internally represent this structure. In this paper, we use Shapley Taylor interaction indices (STII) in order to examine how language and s…

Cited by 0SourcePDFScholar
2024

Mission: Impossible Language Models

ACL 2024long

Chomsky and others have very directly claimed that large language models (LLMs) are equally capable of learning languages that are possible and impossible for humans to learn. However, there is very little published experimental evidence to support such a claim. Here, we develop a set of synthetic i…

2023

Injecting structural hints: Using language models to study inductive biases in language learning

EMNLP 2023long findings

Both humans and transformer language models are able to learn language without explicit structural supervision. What cognitive inductive biases make this learning possible? Here, we examine the effect of different inductive learning biases by actively controlling the inductive biases of artificial l…

Cited by 0SourcecodeScholar
2023

Oolong: Investigating What Makes Transfer Learning Hard with Controlled Studies

EMNLP 2023short main

When we transfer a pretrained language model to a new language, there are many axes of variation that change at once. To disentangle the impact of different factors like syntactic similarity and vocabulary similarity, we propose a set of \emph{controlled transfer studies}: we systematically transfor…

Cited by 0SourcecodeScholar
2022

When classifying grammatical role, BERT doesn’t care about word order... except when it matters

ACL 2022short

Because meaning can often be inferred from lexical semantics alone, word order is often a redundant cue in natural language. For example, the words chopped, chef, and onion are more likely used to convey “The chef chopped the onion,” not “The onion chopped the chef.” Recent work has shown large lang…