Reproduce “Active Inference and Reinforcement Learning: A unified inference on continuous state and action spaces under partial observability” on OpenResearch

About this paper

Parvin Malekzadeh, Konstantinos N. Plataniotis

Researchers at the University of Toronto introduced a unified inference framework that integrates Active Inference and Reinforcement Learning for continuous control in partially observable environments. The resulting algorithms, G-SAC and G-Dreamer, achieve enhanced performance, superior robustness to noisy observations, and effective intrinsic exploration compared to existing deep RL methods.

Key insights

• Active Inference's Expected Free Energy can be reformulated as a unified objective function that intrinsically balances reward maximization, policy entropy, and information gain, making it amenable to continuous POMDPs.
• Explicitly accounting for belief state uncertainty and incorporating an information-seeking term in the objective enables robust performance under partial observability and noise, often matching or exceeding agents with full state access.
• The unified framework provides a theoretical basis for extending powerful deep RL algorithms (like SAC and Dreamer) to effectively operate in more realistic, partially observable settings with intrinsic motivation.

Summary by alphaXiv.