SOURCE-LINKED INTELLIGENCE

Harnessing Image Question Dependence for Better VLM Test-time Reinforcement Learning

arXiv · AI, language, vision and robotics · article · Sep 9, 2026 · UTC

Test-time reinforcement learning can adapt vision-language models (VLMs) to unlabeled target data, but its effectiveness is fundamentally limited by the reliability of self-generated learning signals. To assess the reliability of consensus-based learning signals, we analyze VLM test-time reinforcement learning across diverse VQA datasets and model sizes, revealing two limitations. First, gains from consensus-based test-time training largely come from answer normalization rather than content correction. Second, many initial VLM responses are incorrect due to the model's limited ability to joint

Read original source ↗ Open in workspace

recordType

paper

region

Global