Reproduce “Imitating Latent Policies from Observation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ashley D. Edwards, Himanshu Sahni, Yannick Schroecker, Charles L. Isbell
ILPO enables agents to learn effective policies directly from state observations with minimal environment interaction by inferring latent actions offline and then quickly mapping them to real actions. This approach demonstrates superior sample efficiency compared to Behavioral Cloning from Observation across various discrete action environments.
Key insights
- Complex behaviors can be decoupled into abstract "latent actions" learned offline solely from state observations, which are then grounded to real actions with minimal online interaction.
- A specialized loss function for the latent forward dynamics model enables it to predict distinct future states for each latent action, effectively discovering meaningful behavioral modes without explicit labels.
- The two-stage architecture significantly improves sample efficiency by performing the computationally intensive learning of abstract behaviors offline, reserving online interaction only for action grounding.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.