AIConfPaper
← Search

Ameya Prabhu

18 accepted papers

2026

Adaptive Attacks on Trusted Monitors Subvert AI Control Protocols

Mikhail Terekhov, Alexander Panfilov, Daniil Dzenhaliou, Caglar Gulcehre, Maksym Andriushchenko, Ameya Prabhu +1
ICLR 2026poster

AI control protocols serve as a defense mechanism to stop untrusted LLM agents from causing harm in autonomous settings. Prior work treats this as a security problem, stress testing with exploits that use the deployment context to subtly complete harmful side tasks, such as backdoor insertion. In pr…

Cited by 0Source ↗Scholar ↗
2026

Mapping Post-Training Forgetting in Language Models at Scale

Jackson Harmon, Andreas Hochlehnert, Matthias Bethge, Ameya Prabhu
ICLR 2026poster

Scaled post‑training now drives many of the largest capability gains in language models (LMs), yet its effect on pretrained knowledge remains poorly understood. Not all forgetting is equal: Forgetting one fact (e.g., a U.S. president or an API call) does not “average out” by recalling another. Hence…

Cited by 0Source ↗Scholar ↗
2026

Strategic Dishonesty Can Undermine AI Safety Evaluations of Frontier LLMs

Alexander Panfilov, Evgenii Kortukov, Kristina Nikolić, Matthias Bethge, Sebastian Lapuschkin, Wojciech Samek +3
ICLR 2026poster

Large language model (LLM) developers aim for their models to be honest, helpful, and harmless. However, when faced with malicious requests, models are trained to refuse, sacrificing helpfulness. We show that frontier LLMs can develop a preference for \textit{dishonesty} as a new strategy, even when…

Cited by 0Source ↗Scholar ↗
2025

Great Models Think Alike and this Undermines AI Oversight

Shashwat Goel, Joschka Strüber, Ilze Amanda Auzina, Karuna K Chandra, Ponnurangam Kumaraguru, Douwe Kiela +3
ICML 2025spotlight

As Language Model (LM) capabilities advance, evaluating and supervising them at scale is getting harder for humans. There is hope that other language models can automate both these tasks, which we refer to as *AI Oversight*. We study how model similarity affects both aspects of AI oversight by propo…

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2025

How to Merge Your Multimodal Models Over Time?

Sebastian Dziadzio, Vishaal Udandarao, Karsten Roth, Ameya Prabhu, Zeynep Akata, Samuel Albanie +1
CVPR 2025poster

Model merging combines expert models---each finetuned from a shared foundation model on diverse tasks and domains---into a single, more capable base model. However, existing model merging approaches assume all experts to be available simultaneously. In reality, new tasks and domains emerge continuou…

Cited by 4Source ↗PDF ↗code ↗Scholar ↗
2025

ONEBench to Test Them All: Sample-Level Benchmarking Over Open-Ended Capabilities

Adhiraj Ghosh, Sebastian Dziadzio, Ameya Prabhu, Vishaal Udandarao, Samuel Albanie, Matthias Bethge
ACL 2025long

Traditional fixed test datasets fall short in evaluating the open-ended capabilities of foundation models. To address this, we propose ONEBench (OpeN-Ended Benchmarking), a new paradigm that consolidates individual evaluation datasets into a unified, ever-expanding sample pool. ONEBench enables cust…

Cited by 0Source ↗PDF ↗Scholar ↗
2025

VGGSounder: Audio-Visual Evaluations for Foundation Models

Daniil Zverev, Thaddäus Wiedemer, Ameya Prabhu, Matthias Bethge, Wieland Brendel, A. Sophia Koepke
ICCV 2025poster

Designing effective foundation models requires high-quality evaluation datasets. With the emergence of audio-visual foundation models, reliable assessment of their multi-modal understanding is essential. The current gold standard for evaluating audio-visual understanding is the popular classificatio…

Cited by 0Source ↗PDF ↗Scholar ↗
2024

A Practitioner's Guide to Real-World Continual Multimodal Pretraining

Vishaal Udandarao, Karsten Roth, Sebastian Dziadzio, Ameya Prabhu, Mehdi Cherti, Oriol Vinyals +4
NeurIPS 2024poster

Multimodal foundation models serve numerous applications at the intersection of vision and language. Still, despite being pretrained on extensive data, they become outdated over time. To keep models updated, research into continual pretraining mainly explores scenarios with either (1) infrequent, in…

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2024

CiteME: Can Language Models Accurately Cite Scientific Claims?

Ori Press, Andreas Hochlehnert, Ameya Prabhu, Vishaal Udandarao, Ofir Press, Matthias Bethge
NeurIPS 2024poster

Thousands of new scientific papers are published each month. Such information overload complicates researcher efforts to stay current with the state-of-the-art as well as to verify and correctly attribute claims. We pose the following research question: Given a text excerpt referencing a paper, cou…

Cited by 8Source ↗PDF ↗code ↗Scholar ↗
2024

Efficient Lifelong Model Evaluation in an Era of Rapid Progress

Ameya Prabhu, Vishaal Udandarao, Philip Torr, Matthias Bethge, Adel Bibi, Samuel Albanie
NeurIPS 2024poster

Standardized benchmarks drive progress in machine learning. However, with repeated testing, the risk of overfitting grows as algorithms over-exploit benchmark idiosyncrasies. In our work, we seek to mitigate this challenge by compiling \textit{ever-expanding} large-scale benchmarks called \textit{Li…

Cited by 1Source ↗PDF ↗code ↗Scholar ↗
2024

No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance

Vishaal Udandarao, Ameya Prabhu, Adhiraj Ghosh, Yash Sharma, Philip Torr, Adel Bibi +2
NeurIPS 2024poster

Web-crawled pretraining datasets underlie the impressive "zero-shot" evaluation performance of multimodal models, such as CLIP for classification and Stable-Diffusion for image generation. However, it is unclear how meaningful the notion of "zero-shot" generalization is for such multimodal models, a…

Cited by 50Source ↗PDF ↗code ↗Scholar ↗
2024

RanDumb: Random Representations Outperform Online Continually Learned Representations

Ameya Prabhu, Shiven Sinha, Ponnurangam Kumaraguru, Philip Torr, Ozan Sener, Puneet K. Dokania
NeurIPS 2024poster

Continual learning has primarily focused on the issue of catastrophic forgetting and the associated stability-plasticity tradeoffs. However, little attention has been paid to the efficacy of continually learned representations, as representations are learned alongside classifiers throughout the lear…

Cited by 0Source ↗PDF ↗Scholar ↗
2023

Computationally Budgeted Continual Learning: What Does Matter?

Ameya Prabhu, Hasan Abed Al Kader Hammoud, Puneet K. Dokania, Philip H.S. Torr, Ser-Nam Lim, Bernard Ghanem +1
CVPR 2023poster

Continual Learning (CL) aims to sequentially train models on streams of incoming data that vary in distribution by preserving previous knowledge while adapting to new data. Current CL literature focuses on restricted access to previously seen data, while imposing no constraints on the computational…

Cited by 76Source ↗PDF ↗code ↗Scholar ↗
2023

Rapid Adaptation in Online Continual Learning: Are We Evaluating It Right?

Hasan Abed Al Kader Hammoud, Ameya Prabhu, Ser-Nam Lim, Philip H.S. Torr, Adel Bibi, Bernard Ghanem
ICCV 2023poster

We revisit the common practice of evaluating adaptation of Online Continual Learning (OCL) algorithms through the metric of online accuracy, which measures the accuracy of the model on the immediate next few samples. However, we show that this metric is unreliable, as even vacuous blind classifiers,…

Cited by 0PDF ↗code ↗Scholar ↗
2023

Real-Time Evaluation in Online Continual Learning: A New Hope

Yasir Ghunaim, Adel Bibi, Kumail Alhamoud, Motasem Alfarra, Hasan Abed Al Kader Hammoud, Ameya Prabhu +2
CVPR 2023highlight

Current evaluations of Continual Learning (CL) methods typically assume that there is no constraint on training time and computation. This is an unrealistic assumption for any real-world setting, which motivates us to propose: a practical real-time evaluation of continual learning, in which the stre…

Cited by 54Source ↗PDF ↗code ↗Scholar ↗
2021

No Cost Likelihood Manipulation at Test Time for Making Better Mistakes in Deep Networks

Shyamgopal Karthik, Ameya Prabhu, Puneet K. Dokania, Vineet Gandhi
ICLR 2021poster

There has been increasing interest in building deep hierarchy-aware classifiers that aim to quantify and reduce the severity of mistakes, and not just reduce the number of errors. The idea is to exploit the label hierarchy (e.g., the WordNet ontology) and consider graph distances as a proxy for mist…

Cited by 27Source ↗PDF ↗code ↗Scholar ↗
2020

GDumb: A Simple Approach that Questions Our Progress in Continual Learning

Ameya Prabhu, Philip H. S. Torr, Puneet K. Dokania
ECCV 2020poster

We discuss a general formulation for the Continual Learning (CL) problem for classification---a learning task where a stream provides samples to a learner and the goal of the learner, depending on the samples it receives, is to continually upgrade its knowledge about the old classes and learn new on…

Cited by 730Source ↗PDF ↗code ↗Scholar ↗
2018

Deep Expander Networks: Efficient Deep Networks from Graph Theory

Ameya Prabhu, Girish Varma, Anoop Namboodiri
ECCV 2018poster

Efficient CNN designs like ResNets and DenseNet were proposed to improve accuracy vs efficiency trade-offs. They essentially increased the connectivity, allowing efficient information flow across layers. Inspired by these techniques, we propose to model connections between filters of a CNN using gra…

Cited by 92Source ↗PDF ↗code ↗Scholar ↗