RealBirdID: Benchmarking Bird Species Identification in the Era of MLLMs
Logan Lawrence, Oindrila Saha, Rangel Daroya, Mustafa Chasmai, Wuao Liu, Max Hamilton, Aaron Sun, Seoyun Jeong
Abstract
Fine-grained bird species identification in the wild is frequently unanswerable from a single image: key cues may be non-visual (e.g. vocalization), or obscured due to occlusion, camera angle, or low resolution. Yet today's multimodal systems are typically judged on answerable, in-schema cases, encouraging confident guesses rather than principled abstention. We propose the RealBirdID benchmark: given an image of a bird, a system should either answer with a species or abstain with a concrete, evidence-based rationale: "requires vocalization," "low quality image," or "view obstructed". For each genus, the dataset includes a validation split composed of curated unanswerable examples with labeled rationales, paired with a companion set of clearly answerable instances. We find that (1) the species identification on the answerable set is challenging for a variety of open-source and proprietary models (\leq13% accuracy for MLLMs including GPT-5 and Gemini-2.5 Pro), (2) models with greater classification ability are not necessarily more calibrated to abstain from unanswerable examples, and (3) that MLLMs generally fail at providing correct reasons even when they do abstain. RealBirdID establishes a focused target for abstention-aware fine-grained recognition and a recipe for measuring progress.
BibTeX
@inproceedings{cvpr2026_realbirdidbenchm,
title = {RealBirdID: Benchmarking Bird Species Identification in the Era of MLLMs},
author = {Logan Lawrence and Oindrila Saha and Rangel Daroya and Mustafa Chasmai and Wuao Liu and Max Hamilton and Aaron Sun and Seoyun Jeong and Fabien Delattre and Subhransu Maji and Grant Van Horn},
booktitle = {CVPR 2026},
year = {2026}
}