Reproduce “Optimistic Task Inference for Behavior Foundation Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Thomas Rupf, Marco Bagatella, Marin Vlastelica, Andreas Krause
Optimistic Task Inference for BFMs (OpTI-BFM) introduces an online framework for Behavior Foundation Models to infer reward functions solely through environment interaction. This method efficiently recovers near-optimal policies within a few episodes, significantly reducing the need for extensive pre-labeled data and demonstrating adaptability to changing reward functions.
Key insights
- Online task inference for BFMs can be modeled as a linear contextual bandit problem by leveraging the approximate linearity of expected discounted returns with Universal Successor Features.
- Actively collecting reward-level feedback through environment interaction provides more informative data for task identification compared to passive data collection.
- The UCB approach effectively balances the trade-off between reducing uncertainty about the reward function and maximizing the agent's immediate performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.