Reproduce “Generalization to New Sequential Decision Making Tasks with In-Context Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sharath Chandra Raparthy, Eric Hambro, Robert Kirk, Mikael Henaff, Roberta Raileanu
Transformers are shown to leverage in-context learning to generalize to new sequential decision-making tasks using only a few demonstrations without parameter updates. This is achieved by pretraining on diverse offline multi-trajectory sequences, enabling adaptation to unseen MiniHack and Procgen environments.
Key insights
- Training on multi-trajectory sequences is crucial for transformers to develop in-context learning capabilities for sequential decision-making.
- Higher 'trajectory burstiness' (more similar demonstrations in context) significantly improves few-shot performance, indicating the model leverages relevant examples.
- Training with stochastic environment dynamics enhances generalization to unseen tasks, acting as a form of regularization.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.