Reproduce “Progressive$^2$: A Teacher-Student Progressive Co-Evolving Knowledge Distillation Method for Substantial Model Compression” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tiancong Cheng, Ying Zhang, Zhiwen Yu, Yifang Yin, Bin Guo
Progressive² distillation transfers knowledge from a large teacher to a compact student by introducing deeper teacher features in stages, with an optional sequence of progressively smaller students. On Cityscapes, its 1.57-million-parameter student reports 58.39% mIoU and 92.88% pixel accuracy, versus 55.68% and 91.91% for TAKD+DIST. The proposed mechanism is not established as causal.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.