Reproduce “Turning Adaptation into Assets: Cross-Domain Bridging for Online Vision-Language Navigation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zixuan Hu, Xuantuo Huang, Yancheng Li, Yichun Hu, Shengyong Xu, Ling-Yu Duan
The Inter-Domain BridgE with Historical Assets (IDEA) framework improves Vision-and-Language Navigation (VLN) by turning adaptation into continuously accumulated and composed "knowledge assets," addressing catastrophic forgetting and negative transfer in dynamic environments. It consistently outperformed existing test-time adaptation methods on REVERIE, R2R, and R2R-CE, showing up to +2.8% SPL gain and being significantly more inference-efficient.
Key insights
- Online adaptation can be reframed as accumulating and composing transferable "knowledge assets" (soft visual prompts) rather than performing isolated, transient updates.
- A sensitivity-aware weighting mechanism, guided by Fisher Information, helps distill task-essential knowledge into assets, improving their transferability and preventing overfitting.
- A training-free adaptation bridge, constructed by optimally projecting target domain statistics onto the convex hull of historical assets, offers efficient and robust adaptation for novel environments.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.