Justin W. Lin — accepted AI-conference papers

2026

Comparing AI Agents to Cybersecurity Professionals in Real-World Penetration Testing

Justin W Lin, Eliot Krzysztof Jones, Donovan Julian Jasper, Ethan Jun-shen Ho, Anna Wu, Arnold Tianyi Yang +7

ICLR 2026poster

We present the first comprehensive evaluation of AI agents against human cybersecurity professionals in a live enterprise environment. We evaluate ten cybersecurity professionals alongside six existing AI agents and ARTEMIS, our new agent scaffold, on a large university network consisting of $\sim$8…

Cited by 0 Source ↗code ↗Scholar ↗

2026

EVMbench: Evaluating AI Agents on Smart Contract Security

Justin Wang, Andreas Bigger, Xiaohai Xu, Justin W. Lin, Andy Applebaum, Tejal Patwardhan +2

ICML 2026poster

Smart contracts on public blockchains now manage large amounts of value, and vulnerabilities in these systems can lead to substantial losses. As AI agents become more capable at reading, writing, and running code, it is natural to ask how well they can already navigate this landscape, both in ways t…

Cited by 0 Source ↗Scholar ↗

2025

Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models

Andy K Zhang, Neil Perry, Riya Dulepet, Joey Ji, Celeste Menders, Justin W Lin +19

ICLR 2025oral

Language Model (LM) agents for cybersecurity that are capable of autonomously identifying vulnerabilities and executing exploits have potential to cause real-world impact. Policymakers, model providers, and researchers in the AI and cybersecurity communities are interested in quantifying the capabil…

Cited by 33 Source ↗PDF ↗Scholar ↗