Scale Can't Overcome Pragmatics: The Impact of Reporting Bias on Vision-Language Reasoning

Amita Kamath, Jack Hessel, Khyathi Chandu, Jena D. Hwang, Kai-Wei Chang et al.|February 26, 2026arXiv

Key Takeaway

Bigger models and more data won't automatically teach reasoning skills if your training data has systematic blind spots—you need intentional data...

Summary

Vision-language models struggle with reasoning tasks like counting and spatial understanding not because they're too small, but because their training data is biased toward how people naturally talk about images—omitting obvious details.

data evaluation reasoning

Key Terms

reporting-bias vision-language-models pragmatics