Reproduce “Environmental drivers of systematicity and generalization in a situated agent” on OpenResearch

About this paper

Felix Hill, Andrew Lampinen, Rosalia Schneider, Stephen Clark, Matthew Botvinick, James L. McClelland, Adam Santoro

The authors test whether a conventional neural-network agent can combine learned object concepts and actions on unseen instructions through interaction in simulated 3D rooms. It learns from language, pixels, movement, and manipulation across varied views. In putting, it reaches over 90% accuracy on withheld combinations; grid-world and Unity tests score 0.40 and 0.63. The findings support environmental effects but remain simulation-based.

Key insights

• Systematic generalization in deep neural networks can emerge from conventional architectures when agents are situated in sufficiently rich, interactive, and multi-modal environments.
• The quantity and diversity of training experiences are critical drivers, improving the agent's ability to systematically interpret even abstract concepts like negation.
• Factors such as visual invariances from an egocentric perspective and the temporal richness of interactive perception significantly enhance generalization capabilities beyond static, decontextualized inputs.

Summary by alphaXiv.