Reproduce “Correct Labels Are Not Enough: Reliable Metadata Can Silently Control Model Behavior” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Renee Jia, Jinzhou Wu

The authors test whether fine-tuning teaches models to follow visible annotations as shortcuts despite correct training labels. They train with a tag marking the correct choice, then move it to a wrong choice at test while holding the task fixed. One model follows the moved tag on 97.0% of 300 items, a diagnostic rather than deployment evidence.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.