← Search

Ryan Burnell

2 accepted papers

2025

Language-Models-as-a-Service: Overview of a New Paradigm and its Challenges

AAAI 2025technical

Some of the most powerful language models currently are proprietary systems, accessible only via (typically restrictive) web or software programming interfaces. This is the LanguageModels-as-a-Service (LMaaS) paradigm. In contrast with scenarios where full model access is available, as in the case…

Cited by 18SourcePDFScholar
2022

Not a Number: Identifying Instance Features for Capability-Oriented Evaluation

IJCAI 2022poster

In AI evaluation, performance is often calculated by averaging across various instances. But to fully understand the capabilities of an AI system, we need to understand the factors that cause its pattern of success and failure. In this paper, we present a new methodology to identify and build inform…