Reproduce “Intention-Conditioned Flow Occupancy Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chongyi Zheng, Seohong Park, Sergey Levine, Benjamin Eysenbach
A framework called Intention-Conditioned Flow Occupancy Models (InFOM) enables pre-training of RL agents to learn long-term temporal dynamics and diverse user intentions from offline data. InFOM achieves a 1.8x median improvement in returns and a 36% increase in success rates on various benchmarks by using flow matching for future state prediction and implicit generalized policy improvement for fine-tuning.
Key insights
- Conditioning future state predictions on inferred latent intentions allows for effective disentanglement of diverse behaviors within heterogeneous offline datasets, enhancing adaptability to new tasks.
- Combining flow matching with a SARSA-style temporal difference loss provides an expressive and stable way to model multimodal future state distributions and long-horizon temporal dependencies, improving temporal reasoning.
- An implicit Generalized Policy Improvement (GPI) strategy, implemented with an expectile loss, offers a stable and computationally efficient method for policy extraction from continuous intention spaces, outperforming explicit GPI.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.