SOURCE-LINKED INTELLIGENCE

To Adapt or Not to Adapt? Selective Adaptation for Vision-Language Models

arXiv · AI, language, vision and robotics · article · Sep 8, 2026 · UTC

Test-time adaptation (TTA) has emerged as a prominent strategy for adapting vision-language models to distribution shifts during inference. We conduct a per-sample analysis of model predictions before and after adaptation, and observe two failure modes in existing TTA methods that echo previous work. Adaptations are frequently negligible, yielding no change in the model's predictions, and more severely, they can be detrimental by flipping previously correct predictions to incorrect ones. This naturally raises a question: Can we identify and skip such negligible or harmful adaptations? In this

Read original source ↗ Open in workspace

recordType

paper

region

Global