Reproduce “Evaluating Model Perception of Color Illusions in Photorealistic Scenes” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lingjun Mao, Zineng Tang, Alane Suhr
Researchers at UC Berkeley demonstrated that Vision-Language Models exhibit human-like susceptibility to color illusions in photorealistic scenes, even when queried for objective pixel values. They developed a method that allows VLMs to selectively align their color judgments with either human perception or actual pixel values based on explicit user prompts, achieving over 80% accuracy in both modes for contrast illusions.
Key insights
- VLMs' susceptibility to color illusions arises from a complex interplay of biases in their low-level visual processing and their high-level prior knowledge, including linguistic and commonsense information.
- Scaling up VLM models tends to increase their human-like perceptual biases, making them more prone to illusions.
- Instruction tuning, particularly a novel mixed training approach, can effectively teach VLMs to selectively align their color judgments with either objective pixel values or human perception based on explicit prompts.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.