Reproduce “Automated mapping of virtual environments with visual predictive coding” on OpenResearch

About this paper

James Gornet, Matthew Thomson

This research from Caltech demonstrates that training a neural network on a visual predictive coding task autonomously constructs accurate implicit cognitive maps from raw sequential sensory data. The system learns robust spatial representations and exhibits place cell-like activity, effectively navigating environments with visual ambiguities where image similarity-based methods fail.

Key insights

• Predictive coding offers a unified algorithmic framework for constructing diverse cognitive maps, from spatial to abstract, by inherently forcing the network to build a consistent world model.
• Temporal context and the predictive task are crucial for forming robust spatial maps, especially in visually degenerate environments, distinguishing it from simple image similarity encoding.
• The learned latent representations in predictive coding networks exhibit properties analogous to biological place cells, forming a unique combinatorial code that supports vector navigation.

Summary by alphaXiv.