AIConfPaper
← Search

Maksym Andriushchenko

32 accepted papers

2026

Adaptive Attacks on Trusted Monitors Subvert AI Control Protocols

Mikhail Terekhov, Alexander Panfilov, Daniil Dzenhaliou, Caglar Gulcehre, Maksym Andriushchenko, Ameya Prabhu +1
ICLR 2026poster

AI control protocols serve as a defense mechanism to stop untrusted LLM agents from causing harm in autonomous settings. Prior work treats this as a security problem, stress testing with exploits that use the deployment context to subtly complete harmful side tasks, such as backdoor insertion. In pr…

Cited by 0Source ↗Scholar ↗
2026

Capability-Based Scaling Trends for LLM-Based Red-Teaming

Alexander Panfilov, Paul Kassianik, Maksym Andriushchenko, Jonas Geiping
ICLR 2026poster

As large language models grow in capability and agency, identifying vulnerabilities through red-teaming becomes vital for safe deployment. However, traditional prompt-engineering approaches may prove ineffective once red-teaming turns into a \emph{weak-to-strong} problem, where target models surpass…

Cited by 0Source ↗code ↗Scholar ↗
2026

Helpful to a Fault: Measuring Illicit Assistance in Multi-Turn, Multilingual LLM Agents

Nivya Talokar, Ayush Kumar Tarun, Murari Mandal, Maksym Andriushchenko, Antoine Bosselut
ICML 2026poster

LLM-based agents increasingly execute real-world workflows via tools and memory. Granting LLMs such powers enables ill-intended adversaries to likewise use these agents to carry out complex misuse scenarios. Existing agent-misuse benchmarks largely test single-prompt instructions, leaving a gap in m…

Cited by 0Source ↗Scholar ↗
2026

Monitoring Decomposition Attacks with Lightweight Sequential Monitors

Chen Yueh-Han, Nitish Joshi, Yulin Chen, Maksym Andriushchenko, Rico Angell, He He
ICLR 2026poster

As LLMs become more agentic, a critical risk emerges: attackers can \emph{decompose} harmful goals into stateful, benign subtasks that trick LLM agents into executing them without realizing the harmful intent in the same context. The challenge lies in the existing shallow safety alignment techniques…

Cited by 0Source ↗code ↗Scholar ↗
2026

PostTrainBench: Can LLM Agents Automate LLM Post-Training?

Ben Rank, Hardik Bhatnagar, Ameya Pandurang Prabhu, Shira Eisenberg, Karina Nguyen, Matthias Bethge +1
ICML 2026poster

Given the recent rapid progress of LLM agents like Claude Code or Codex CLI for software engineering, an important next question is whether they can automate AI research itself. In this paper, we study *post-training*, which is the critical step that turns base LLMs into useful assistants. We introd…

Cited by 0Source ↗Scholar ↗
2026

Strategic Dishonesty Can Undermine AI Safety Evaluations of Frontier LLMs

Alexander Panfilov, Evgenii Kortukov, Kristina Nikolić, Matthias Bethge, Sebastian Lapuschkin, Wojciech Samek +3
ICLR 2026poster

Large language model (LLM) developers aim for their models to be honest, helpful, and harmless. However, when faced with malicious requests, models are trained to refuse, sacrificing helpfulness. We show that frontier LLMs can develop a preference for \textit{dishonesty} as a new strategy, even when…

Cited by 0Source ↗Scholar ↗
2025

AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents

Maksym Andriushchenko, Alexandra Souly, Mateusz Dziemian, Derek Duenas, Maxwell Lin, Justin Wang +6
ICLR 2025poster

The robustness of LLMs to jailbreak attacks, where users design prompts to circumvent safety measures and misuse model capabilities, has been studied primarily for LLMs acting as simple chatbots. Meanwhile, LLM agents---which use external tools and can execute multi-stage tasks---may pose a greater…

Cited by 36Source ↗PDF ↗Scholar ↗
2025

Critical Influence of Overparameterization on Sharpness-aware Minimization

Sungbin Shin, Dongyeop Lee, Maksym Andriushchenko, Namhoon Lee
UAI 2025

Training overparameterized neural networks often yields solutions with varying generalization capabilities, even when achieving similar training losses. Recent evidence indicates a strong correlation between the sharpness of a minimum and its generalization error, leading to increased interest in op

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2025

Is In-Context Learning Sufficient for Instruction Following in LLMs?

Hao Zhao, Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion
ICLR 2025poster

In-context learning (ICL) allows LLMs to learn from examples without changing their weights: this is a particularly promising capability for long-context LLMs that can potentially learn from many examples. Recently, Lin et al. (2024) proposed URIAL, a method using only three in-context examples to a…

Cited by 14Source ↗PDF ↗code ↗Scholar ↗
2025

Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks

Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion
ICLR 2025poster

We show that even the most recent safety-aligned LLMs are not robust to simple *adaptive* jailbreaking attacks. First, we demonstrate how to successfully leverage access to *logprobs* for jailbreaking: we initially design an adversarial prompt template (sometimes adapted to the target LLM), and then…

Cited by 161Source ↗PDF ↗code ↗Scholar ↗
2025

OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents

Thomas Kuntz, Agatha Duzan, Hao Zhao, Francesco Croce, J Zico Kolter, Nicolas Flammarion +1
NeurIPS 2025spotlight

Computer use agents are LLM-based agents that can directly interact with a graphical user interface, by processing screenshots or accessibility trees. While these systems are gaining popularity, their safety has been largely overlooked, despite the fact that evaluating and understanding their potent…

Cited by 0Source ↗code ↗Scholar ↗
2024

Improving Alignment and Robustness with Circuit Breakers

Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko +3
NeurIPS 2024poster

AI systems can take harmful actions and are highly vulnerable to adversarial attacks. We present an approach, inspired by recent advances in representation engineering, that interrupts the models as they respond with harmful outputs with "circuit breakers." Existing techniques aimed at improving ali…

Cited by 65Source ↗PDF ↗Scholar ↗
2024

JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models

Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag +6
NeurIPS 2024poster

Jailbreak attacks cause large language models (LLMs) to generate harmful, unethical, or otherwise objectionable content. Evaluating these attacks presents a number of challenges, which the current collection of benchmarks and evaluation techniques do not adequately address. First, there is no clear…

Cited by 149Source ↗PDF ↗code ↗Scholar ↗
2024

Layer-wise linear mode connectivity

Linara Adilova, Maksym Andriushchenko, Michael Kamp, Asja Fischer, Martin Jaggi
ICLR 2024poster

Averaging neural network parameters is an intuitive method for fusing the knowledge of two independent models. It is most prominently used in federated learning. If models are averaged at the end of training, this can only lead to a good performing model if the loss surface of interest is very parti…

Cited by 20Source ↗PDF ↗code ↗Scholar ↗
2024

Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning

Hao Zhao, Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion
ICML 2024poster

There is a consensus that instruction fine-tuning of LLMs requires high-quality data, but what are they? LIMA (NeurIPS 2023) and AlpaGasus (ICLR 2024) are state-of-the-art methods for selecting such high-quality examples, either via manual curation or using GPT-3.5-Turbo as a quality scorer. We show…

Cited by 39Source ↗PDF ↗code ↗Scholar ↗
2024

Why Do We Need Weight Decay in Modern Deep Learning?

Francesco D'Angelo, Maksym Andriushchenko, Aditya Varre, Nicolas Flammarion
NeurIPS 2024poster

Weight decay is a broadly used technique for training state-of-the-art deep networks from image classification to large language models. Despite its widespread usage and being extensively studied in the classical literature, its role remains poorly understood for deep learning. In this work, we high…

Cited by 27Source ↗PDF ↗code ↗Scholar ↗
2023

A Modern Look at the Relationship between Sharpness and Generalization

Maksym Andriushchenko, Francesco Croce, Maximilian Müller, Matthias Hein, Nicolas Flammarion
ICML 2023poster

Sharpness of minima is a promising quantity that can correlate with generalization in deep networks and, when optimized during training, can improve generalization. However, standard sharpness is not invariant under reparametrizations of neural networks, and, to fix this, reparametrization-invariant…

Cited by 71Source ↗PDF ↗code ↗Scholar ↗
2023

SGD with Large Step Sizes Learns Sparse Features

Maksym Andriushchenko, Aditya Vardhan Varre, Loucas Pillaud-Vivien, Nicolas Flammarion
ICML 2023poster

We showcase important features of the dynamics of the Stochastic Gradient Descent (SGD) in the training of neural networks. We present empirical observations that commonly used large step sizes (i) may lead the iterates to jump from one side of a valley to the other causing *loss stabilization*, and…

Cited by 66Source ↗PDF ↗code ↗Scholar ↗
2023

Sharpness-Aware Minimization Leads to Low-Rank Features

Maksym Andriushchenko, Dara Bahri, Hossein Mobahi, Nicolas Flammarion
NeurIPS 2023poster

Sharpness-aware minimization (SAM) is a recently proposed method that minimizes the sharpness of the training loss of a neural network. While its generalization improvement is well-known and is the primary motivation, we uncover an additional intriguing effect of SAM: reduction of the feature rank w…

Cited by 31Source ↗PDF ↗code ↗Scholar ↗
2023

Transferable Adversarial Robustness for Categorical Data via Universal Robust Embeddings

Klim Kireev, Maksym Andriushchenko, Carmela Troncoso, Nicolas Flammarion
NeurIPS 2023poster

Research on adversarial robustness is primarily focused on image and text data. Yet, many scenarios in which lack of robustness can result in serious risks, such as fraud detection, medical diagnosis, or recommender systems often do not rely on images or text but instead on tabular data. Adversarial…

Cited by 5Source ↗PDF ↗code ↗Scholar ↗
2022

On the effectiveness of adversarial training against common corruptions

Klim Kireev, Maksym Andriushchenko, Nicolas Flammarion
UAI 2022poster

The literature on robustness towards common corruptions shows no consensus on whether adversarial training can improve the performance in this setting. First, we show that, when used with an appropriately selected perturbation radius, Lp adversarial training can serve as a strong baseline against co…

Cited by 124Source ↗PDF ↗code ↗Scholar ↗
2022

Sparse-RS: A Versatile Framework for Query-Efficient Sparse Black-Box Adversarial Attacks

Francesco Croce, Maksym Andriushchenko, Naman D. Singh, Nicolas Flammarion, Matthias Hein
AAAI 2022technical

We propose a versatile framework based on random search, Sparse-RS, for score-based sparse targeted and untargeted attacks in the black-box setting. Sparse-RS does not rely on substitute models and achieves state-of-the-art success rate and query efficiency for multiple sparse attack models: L0-boun…

Cited by 129Source ↗PDF ↗code ↗Scholar ↗
2021

On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines

Marius Mosbach, Maksym Andriushchenko, Dietrich Klakow
ICLR 2021poster

Fine-tuning pre-trained transformer-based language models such as BERT has become a common practice dominating leaderboards across various NLP benchmarks. Despite the strong empirical performance of fine-tuned models, fine-tuning is an unstable process: training the same model with multiple random s…

Cited by 453Source ↗PDF ↗code ↗Scholar ↗
2021

RobustBench: a standardized adversarial robustness benchmark

Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang +2
NeurIPS 2021poster

As a research community, we are still lacking a systematic understanding of the progress on adversarial robustness which often makes it hard to identify the most promising ideas in training robust models. A key challenge in benchmarking robustness is that its evaluation is often error-prone leading…

Cited by 843Source ↗code ↗Scholar ↗
2020

Square Attack: a query-efficient black-box adversarial attack via random search

Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, Matthias Hein
ECCV 2020poster

We propose the Square Attack, a score-based black-box $l_2$- and $l_\infty$- adversarial attack that does not rely on local gradient information and thus is not affected by gradient masking. Square Attack is based on a randomized search scheme which selects localized square-shaped updates at random…

Cited by 1243Source ↗PDF ↗code ↗Scholar ↗
2019

Provable Robustness of ReLU networks via Maximization of Linear Regions

Francesco Croce, Maksym Andriushchenko, Matthias Hein
AISTATS 2019poster

It has been shown that neural network classifiers are not robust. This raises concerns about their usage in safety-critical systems. We propose in this paper a regularization scheme for ReLU networks which provably improves the robustness of the classifier by maximizing the linear regions of the cla…

Cited by 195Source ↗PDF ↗code ↗Scholar ↗
2019

Provably robust boosted decision stumps and trees against adversarial attacks

Maksym Andriushchenko, Matthias Hein
NeurIPS 2019poster

The problem of adversarial robustness has been studied extensively for neural networks. However, for boosted decision trees and decision stumps there are almost no results, even though they are widely used in practice (e.g. XGBoost) due to their accuracy, interpretability, and efficiency. We show in…

Cited by 88Source ↗PDF ↗code ↗Scholar ↗
2019

Why ReLU Networks Yield High-Confidence Predictions Far Away From the Training Data and How to Mitigate the Problem

Matthias Hein, Maksym Andriushchenko, Julian Bitterwolf
CVPR 2019oral

Classifiers used in the wild, in particular for safety-critical systems, should not only have good generalization properties but also should know when they don't know, in particular make low confidence predictions far away from the training data. We show that ReLU type neural networks which yield a…

Cited by 717PDF ↗code ↗Scholar ↗
2017

Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation

Matthias Hein, Maksym Andriushchenko
NeurIPS 2017poster

Recent work has shown that state-of-the-art classifiers are quite brittle, in the sense that a small adversarial change of an originally with high confidence correctly classified input leads to a wrong classification again with high confidence. This raises concerns that such classifiers are vulnera…

Cited by 662Source ↗PDF ↗Scholar ↗