Karen Hambardzumyan — accepted AI-conference papers

2025

AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench

Edan Toledo, Karen Hambardzumyan, Martin Josifoski, RISHI HAZRA, Nicolas Baldwin, Alexis Audran-Reiss +19

NeurIPS 2025spotlight

AI research agents are demonstrating great potential to accelerate scientific progress by automating the design, implementation, and training of machine learning models. We focus on methods for improving agents' performance on MLE-bench, a challenging benchmark where agents compete in Kaggle competi…

Cited by 0 Source ↗code ↗Scholar ↗

2025

Combining Code Generating Large Language Models and Self-Play to Iteratively Refine Strategies in Games

Yoram Bachrach, Edan Toledo, Karen Hambardzumyan, Despoina Magka, Martin Josifoski, Minqi Jiang +8

IJCAI 2025

We propose a self-play approach to generating strategies for playing in multi-player games, where strategies are represented as computer code. We use large language models (LLMs) to generate pieces of code to play in the game, which we refer to as generated bots. We engage the LLM generated bots in

Cited by 0 Source ↗PDF ↗Scholar ↗

2025

Robust LLM safeguarding via refusal feature adversarial training

Lei Yu, Virginie Do, Karen Hambardzumyan, Nicola Cancedda

ICLR 2025poster

Large language models (LLMs) are vulnerable to adversarial attacks that can elicit harmful responses. Defending against such attacks remains challenging due to the opacity of jailbreaking mechanisms and the high computational cost of training LLMs robustly. We demonstrate that adversarial attacks sh…

Cited by 9 Source ↗PDF ↗Scholar ↗

2025

The Automated LLM Speedrunning Benchmark: Reproducing NanoGPT Improvements

Bingchen Zhao, Despoina Magka, Minqi Jiang, Xian Li, Roberta Raileanu, Tatiana Shavrina +17

NeurIPS 2025poster

Rapidly improving large language models (LLMs) have the potential to assist in scientific progress. One critical skill in this endeavor is the ability to faithfully reproduce existing work. To evaluate the capability of AI agents to reproduce complex code in an active research area, we introduce the…

Cited by 0 Source ↗code ↗Scholar ↗

2024

LM Transparency Tool: Interactive Tool for Analyzing Transformer Language Models

Igor Tufanov, Karen Hambardzumyan, Javier Ferrando, Elena Voita

ACL 2024system demonstrations

We present the LM Transparency Tool (LM-TT), an open-source interactive toolkit for analyzing the internal workings of Transformer-based language models. Differently from previously existing tools that focus on isolated parts of the decision-making process, our framework is designed to make the enti…

Cited by 8 Source ↗PDF ↗Scholar ↗

2023

Scaling Laws for Generative Mixed-Modal Language Models

Armen Aghajanyan, LILI YU, Alexis Conneau, Wei-Ning Hsu, Karen Hambardzumyan, Susan Zhang +4

ICML 2023poster

Generative language models define distributions over sequences of tokens that can represent essentially any combination of data modalities (e.g., any permutation of image tokens from VQ-VAEs, speech tokens from HuBERT, BPE tokens for language or code, and so on). To better understand the scaling pro…

Cited by 104 Source ↗PDF ↗Scholar ↗

2021

WARP: Word-level Adversarial ReProgramming

Karen Hambardzumyan, Hrant Khachatrian, Jonathan May

ACL 2021long

Transfer learning from pretrained language models recently became the dominant approach for solving many NLP tasks. A common approach to transfer learning for multiple tasks that maximize parameter sharing trains one or more task-specific layers on top of the language model. In this paper, we presen…

Cited by 386 Source ↗PDF ↗code ↗Scholar ↗