Reproduce “Active Test-time Vision-Language Navigation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Heeju Ko, Sungjune Kim, Gyeongrok Oh, Jeongyoon Yoon, Honglak Lee, Sujin Jang, Seungryong Kim, Sangpil Kim
A framework for Vision-Language Navigation, ATENA, integrates active learning with test-time adaptation, using a novel Mixture Entropy Optimization guided by sparse episodic human feedback and Self-Active Learning for autonomous supervision. This approach significantly improves navigation success rates and efficiency on challenging benchmarks like REVERIE, R2R, and R2R-CE, outperforming existing test-time adaptation methods by up to 44.98% in success rate for some policies while maintaining low latency.
Key insights
- Outcome-conditioned entropy optimization, using a mixture of action and pseudo-expert distributions (MEO), effectively amplifies corrective signals and enhances confidence calibration, leading to more robust adaptation than uniform entropy minimization.
- Sparse, episodic human feedback can be effectively leveraged to guide online test-time adaptation, addressing the practical constraints of detailed human supervision in real-world VLN scenarios.
- An uncertainty-guided query strategy combined with an autonomous self-prediction head (SAL) enables continuous self-supervision, reducing the overall dependence on human intervention and improving the scalability of active learning in VLN.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.