Reproduce “Learning in Hybrid Active Inference Models” on OpenResearch

About this paper

Poppy Collis, Ryan Singh, Paul F Kinghorn, Christopher L Buckley

This paper presents a Hybrid Hierarchical Active Inference (HHA) agent that learns discrete abstractions from continuous dynamics using Recurrent Switching Linear Dynamical Systems (rSLDS) to facilitate efficient planning and control. The agent successfully navigates the Continuous Mountain Car task, demonstrating superior sample efficiency over model-free reinforcement learning baselines and leveraging information-theoretic exploration for faster system identification.

Key insights

• Meaningful discrete abstractions can be learned online from continuous dynamics via Recurrent Switching Linear Dynamical Systems (rSLDS) without prior knowledge or grid-based discretization, overcoming the curse of dimensionality.
• Temporally abstract discrete planning, combined with efficient pre-computed continuous control, allows for computationally efficient and robust navigation in continuous environments.
• Information-theoretic exploration operating on learned discrete representations significantly accelerates system identification and state-space coverage in sparse reward settings.

Summary by alphaXiv.