Reproduce “FOSSIL: Harnessing Feedback on Suboptimal Samples for Data-Efficient Generalisation with Imitation Learning for Embodied Vision-and-Language Tasks” on OpenResearch

About this paper

Sabrina McCallum, Amit Parekh, Alessandro Suglia

FOSSIL, a framework developed by researchers at the University of Edinburgh and Heriot-Watt University, enables imitation learning policies to effectively utilize suboptimal demonstrations when combined with constructive language feedback, leading to improved compositional generalization and robustness in embodied vision-and-language tasks. The method demonstrates greater data efficiency and stronger scaling properties compared to traditional imitation learning and reinforcement learning baselines.

Key insights

• Suboptimal demonstrations can be transformed into valuable learning signals when explicitly contextualized with constructive language feedback, expanding the utility of imperfect data in imitation learning.
• Language feedback serves as a competitive and intuitive alternative or complement to scalar rewards, enhancing generalization and robustness in embodied agents while being suitable for human-AI interaction.
• Auxiliary self-supervised feedback prediction objectives enable agents to learn more robust internal representations, improving resilience to environmental perturbations and unreliable feedback.

Summary by alphaXiv.