Dominic Sobhani

Language Models can Categorize System Inputs for Performance Analysis

Language model systems are used to process diverse categories of input requests, ranging from improving creative writing to solving programming challenges. It would be useful to know which categories they are good at. However, existing evaluations compare model performance on pre-defined categories,…