← Search

Luca Beurer-Kellner

6 accepted papers

2024

AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents

NeurIPS 2024poster

AI agents aim to solve complex tasks by combining text-based reasoning with external tool calls. Unfortunately, AI agents are vulnerable to prompt injection attacks where data returned by external tools hijacks the agent to execute malicious tasks. To measure the adversarial robustness of AI agents,…

2024

Controlled Text Generation via Language Model Arithmetic

ICLR 2024spotlight

As Large Language Models (LLMs) are deployed more widely, customization with respect to vocabulary, style, and character becomes more important. In this work, we introduce model arithmetic, a novel inference framework for composing and biasing LLMs without the need for model (re)training or highly s…

2024

Guiding LLMs The Right Way: Fast, Non-Invasive Constrained Generation

ICML 2024poster

To ensure that text generated by large language models (LLMs) is in an expected format, constrained decoding methods propose to enforce strict formal language constraints during generation. However, as we show in this work, not only do such methods often incur performance overhead during generation,…

2024

Prompt Sketching for Large Language Models

ICML 2024poster

Many recent prompting strategies for large language models (LLMs) query the model multiple times sequentially -- first to produce intermediate results and then the final answer. However, using these methods, both decoder and model are unaware of potential follow-up prompts, leading to disconnected a…

2022

Learning to Configure Computer Networks with Neural Algorithmic Reasoning

NeurIPS 2022accept

We present a new method for scaling automatic configuration of computer networks. The key idea is to relax the computationally hard search problem of finding a configuration that satisfies a given specification into an approximate objective amenable to learning-based techniques. Based on this idea,…

Cited by 22SourcePDFScholar
2022

On Distribution Shift in Learning-based Bug Detectors

ICML 2022spotlight

Deep learning has recently achieved initial success in program analysis tasks such as bug detection. Lacking real bugs, most existing works construct training and test data by injecting synthetic bugs into correct programs. Despite achieving high test accuracy (e.g., >90%), the resulting bug detecto…