Reproduce “Distilling Realizable Students from Unrealizable Teachers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yujin Kim, Nathaniel Chin, Arnav Vasudev, Sanjiban Choudhury
Researchers at Cornell University present CritiQ and ReTRy, two distinct methods designed to enable student policies with partial observations to learn from teachers possessing privileged full-state information. These methods achieve higher learning efficiency and success rates in robotic tasks by strategically managing teacher queries and dynamically refining training resets.
Key insights
- Blindly imitating a teacher with privileged information under state aliasing leads to policy degradation, as conflicting teacher actions for the same student observation accumulate and destabilize learning.
- Strategic interaction, such as selective querying, can mitigate information asymmetry by only seeking teacher guidance when necessary, preventing the amplification of inconsistent training signals.
- Leveraging the teacher to generate recovery trajectories from student-explored states allows for more efficient exploration and robust learning, as the student can always restart from a known path to success.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.