2025
Just Put a Human in the Loop? Investigating LLM-Assisted Annotation for Subjective Tasks
ACL 2025finding
LLM use in annotation is becoming widespread, and given LLMs’ overall promising performance and speed, putting humans in the loop to simply “review” LLM annotations can be tempting. In subjective tasks with multiple plausible answers, this can impact both evaluation of LLM performance, and analysis…