Reproduce “Post-Training Leaves Behavioral Shadows on Unrelated Decisions” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ziyang Zhang, Yubin Jing, Yuanhao Zeng, Yuyao Li, Haofan Wang, Yichen Gong

ATD tests whether unrelated word choices from a privately post-trained model can transmit task-specific capability. It selects near-tie prompts from a public ancestor, records the teacher’s greedy choice, and trains a same-ancestor student on prompt–word pairs. On HumanEval+, Qwen2.5-1.5B’s signal student scores 51.22% versus 45.88% for a nuisance-matched control, a +5.34-point difference; on MBPP+, the difference is +0.33 points, showing transfer is not reliable across tasks.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.