Reproduce “Why DiT Models Underperform as Representation Learners without Long Skip Connections” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Benyuan Meng, Yiliang Zhang, Jin-Wen Wu, Qianqian Xu, Qingming Huang, Longtao Huang

The authors test whether long skip connections (LSCs) improve vision features from diffusion models independently of generation quality. In controlled, from-scratch DiT experiments, LSCs route information around middle layers, where useful image features form. A 100k-step LSC model scores 38.85 on training-free SPair, versus 34.68 without LSC at similar, not identical, FID; the routing explanation remains a hypothesis.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.