D$^2$O: A Dual Debiasing Operator for Training-Free Test-Time Adaptation of Vision–Language Models
Training-free test-time adaptation (TTA) for vision-language models (VLMs) can boost zero-shot classification under mild shifts but often collapses under severe environment/style shifts. We identify two shared failure modes: (i) retrieval confounding, where feature similarity is dominated by style a…