AIConfPaper
← Search

Christoph H. Lampert

30 accepted papers

2026

ASIDE: Architectural Separation of Instructions and Data in Language Models

Egor Zverev, Evgenii Kortukov, Alexander Panfilov, Alexandra Volkova, Rush Tabesh, Sebastian Lapuschkin +2
ICLR 2026poster

Despite their remarkable performance, large language models lack elementary safety features, making them susceptible to numerous malicious attacks. In particular, previous work has identified the absence of an intrinsic separation between instructions and data as the root cause of the success of pro…

Cited by 0Source ↗code ↗Scholar ↗
2026

Back to Square Roots: An Optimal Bound on the Matrix Factorization Error for Multi-Epoch Differentially Private SGD

Nikita Kalinin, Jalaj Upadhyay, Ryan McKenna, Christoph H. Lampert
ICLR 2026poster

Matrix factorization mechanisms for differentially private training have emerged as a promising approach to improve model utility under privacy constraints. In practical settings, models are typically trained over multiple epochs, requiring matrix factorizations that account for repeated participati…

Cited by 0Source ↗Scholar ↗
2025

Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?

Egor Zverev, Sahar Abdelnabi, Soroush Tabesh, Mario Fritz, Christoph H. Lampert
ICLR 2025poster

Large Language Models (LLMs) show impressive results in numerous practical applications, but they lack essential safety features that are common in other areas of computer science, particularly an explicit separation of instructions and data. This makes them vulnerable to manipulations such as indir…

Cited by 22Source ↗PDF ↗code ↗Scholar ↗
2025

Differentially Private Federated $k$-Means Clustering with Server-Side Data

Jonathan Scott, Christoph H. Lampert, David Saulpic
ICML 2025poster

Clustering is a cornerstone of data analysis that is particularly suited to identifying coherent subgroups or substructures in unlabeled data, as are generated continuously in large amounts these days. However, in many cases traditional clustering methods are not applicable, because data are increas…

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2025

Fast Rate Bounds for Multi-Task and Meta-Learning with Different Sample Sizes

Hossein Zakerinia, Christoph H. Lampert
NeurIPS 2025poster

We present new fast-rate PAC-Bayesian generalization bounds for multi-task and meta-learning in the unbalanced setting, i.e. when the tasks have training sets of different sizes, as is typically the case in real-world scenarios. Previously, only standard-rate bounds were known for this situation, wh…

Cited by 0Source ↗Scholar ↗
2025

Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformers

Peter Súkeník, Christoph H. Lampert, Marco Mondelli
NeurIPS 2025poster

The empirical emergence of neural collapse---a surprising symmetry in the feature representations of the training data in the penultimate layer of deep neural networks---has spurred a line of theoretical research aimed at its understanding. However, existing work focuses on data-agnostic models or,…

Cited by 0Source ↗Scholar ↗
2024

1-Lipschitz Layers Compared: Memory Speed and Certifiable Robustness

Bernd Prach, Fabio Brau, Giorgio Buttazzo, Christoph H. Lampert
CVPR 2024poster

The robustness of neural networks against input perturbations with bounded magnitude represents a serious concern in the deployment of deep learning models in safety-critical systems. Recently the scientific community has focused on enhancing certifiable robustness guarantees by crafting \ols neural…

Cited by 6Source ↗PDF ↗code ↗Scholar ↗
2024

Banded Square Root Matrix Factorization for Differentially Private Model Training

Nikita Kalinin, Christoph H. Lampert
NeurIPS 2024poster

Current state-of-the-art methods for differentially private model training are based on matrix factorization techniques. However, these methods suffer from high computational overhead because they require numerically solving a demanding optimization problem to determine an approximately optimal fact…

Cited by 6Source ↗PDF ↗Scholar ↗
2024

More Flexible PAC-Bayesian Meta-Learning by Learning Learning Algorithms

Hossein Zakerinia, Amin Behjati, Christoph H. Lampert
ICML 2024poster

We introduce a new framework for studying meta-learning methods using PAC-Bayesian theory. Its main advantage over previous work is that it allows for more flexibility in how the transfer of knowledge between tasks is realized. For previous approaches, this could only happen indirectly, by means of…

Cited by 4Source ↗PDF ↗code ↗Scholar ↗
2024

Neural collapse vs. low-rank bias: Is deep neural collapse really optimal?

Peter Súkeník, Christoph H. Lampert, Marco Mondelli
NeurIPS 2024poster

Deep neural networks (DNNs) exhibit a surprising structure in their final layer known as neural collapse (NC), and a growing body of works is currently investigated the propagation of neural collapse to earlier layers of DNNs -- a phenomenon called deep neural collapse (DNC). However, existing theor…

Cited by 1Source ↗PDF ↗Scholar ↗
2024

PeFLL: Personalized Federated Learning by Learning to Learn

Jonathan Scott, Hossein Zakerinia, Christoph H Lampert
ICLR 2024poster

We present PeFLL, a new personalized federated learning algorithm that improves over the state-of-the-art in three aspects: 1) it produces more accurate models, especially in the low-data regime, and not only for clients present during its training phase, but also for any that may emerge in the futu…

Cited by 12Source ↗PDF ↗code ↗Scholar ↗
2023

CrAM: A Compression-Aware Minimizer

Alexandra Peste, Adrian Vladu, Eldar Kurtic, Christoph H Lampert, Dan Alistarh
ICLR 2023poster

Deep neural networks (DNNs) often have to be compressed, via pruning and/or quantization, before they can be deployed in practical settings. In this work we propose a new compression-aware minimizer dubbed CrAM that modifies the optimization step in a principled way, in order to produce models whose…

Cited by 10Source ↗PDF ↗code ↗Scholar ↗
2023

Deep Neural Collapse Is Provably Optimal for the Deep Unconstrained Features Model

Peter Súkeník, Marco Mondelli, Christoph H Lampert
NeurIPS 2023spotlight

Neural collapse (NC) refers to the surprising structure of the last layer of deep neural networks in the terminal phase of gradient descent training. Recently, an increasing amount of experimental evidence has pointed to the propagation of NC to earlier layers of neural networks. However, while the…

Cited by 22Source ↗PDF ↗Scholar ↗
2022

Almost-Orthogonal Layers for Efficient General-Purpose Lipschitz Networks

Bernd Prach, Christoph H. Lampert
ECCV 2022poster

"It is a highly desirable property for deep networks to be robust against small input changes. One popular way to achieve this property is by designing networks with a small Lipschitz constant. In this work, we propose a new technique for constructing such Lipschitz networks that has a number of des…

Cited by 45Source ↗PDF ↗code ↗Scholar ↗
2020

Unsupervised object-centric video generation and decomposition in 3D

Paul Henderson, Christoph H. Lampert
NeurIPS 2020poster

A natural approach to generative modeling of videos is to represent them as a composition of moving objects. Recent works model a set of 2D sprites over a slowly-varying background, but without considering the underlying 3D scene that gives rise to them. We instead propose to model a video as the vi…

Cited by 46Source ↗PDF ↗code ↗Scholar ↗
2018

Learning Intelligent Dialogs for Bounding Box Annotation

Ksenia Konyushkova, Jasper Uijlings, Christoph H. Lampert, Vittorio Ferrari
CVPR 2018poster

We introduce Intelligent Annotation Dialogs for bounding box annotation. We train an agent to automatically choose a sequence of actions for a human annotator to produce a bounding box in a minimal amount of time. Specifically, we consider two actions: box verification, where the annotator verifies…

Cited by 52Source ↗PDF ↗code ↗Scholar ↗
2017

iCaRL: Incremental Classifier and Representation Learning

Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, Christoph H. Lampert
CVPR 2017spotlight

A major open problem on the road to artificial intelligence is the development of incrementally learning systems that learn about more and more concepts over time from a stream of data. In this work, we introduce a new training strategy, iCaRL, that allows learning in such a class-incremental way: o…

Cited by 4974PDF ↗code ↗Scholar ↗
2015

A Multi-Plane Block-Coordinate Frank-Wolfe Algorithm for Training Structural SVMs With a Costly Max-Oracle

Neel Shah, Vladimir Kolmogorov, Christoph H. Lampert
CVPR 2015poster

Structural support vector machines (SSVMs) are amongst the best performing methods for structured computer vision tasks, such as semantic image segmentation or human pose estimation. Training SSVMs, however, is computationally costly, because it requires repeated calls to a structured prediction sub…

Cited by 30Source ↗PDF ↗Scholar ↗