Reproduce “You Don't Need Aligned Representations: Knowledge Distillation via Random Prototype Spaces” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hamza Etcibasi, Ramazan Gokberk Cinbis
RPKD distills a pretrained vision model into a smaller model by comparing teacher and student features through a shared, fixed bank of random prototypes. Fixed teacher-side projections and learnable student projections support global and spatial similarity losses. On ImageNet-100, frozen prototype variants outperform learned prototypes across four model pairs; this comparison does not prove that a moving target alone explains the difference.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.