Reproduce “Visual Contrastive Self-Distillation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yijun Liang, Yunjie Tian, Yijiang Li, Yuqi Jia, Kaishen Wang, Tianyi Zhou, Di Fu

VCSD trains vision-language models to use image-specific information through on-policy self-distillation. An exponential-moving-average teacher predicts next tokens with the original image and a same-size black image; their contrast shapes a plausible target for the student, without answer hints. On ViRL39K, its seven-benchmark aggregate reaches 76.26% for Qwen3-VL-8B, exceeding base and matched answer-hint OPSD, but not every benchmark.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.