Reproduce “Policy Evaluation in Decentralized POMDPs with Belief Sharing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mert Kayaalp, Fatima Ghadieh, Ali H. Sayed
A new fully decentralized algorithm allows agents in Partially Observable Markov Decision Processes (Dec-POMDPs) to evaluate policies by approximating global beliefs and learning value functions through local interactions. The approach offers analytical guarantees for belief disagreement and value function convergence, demonstrating performance competitive with a centralized training baseline in multi-sensor target tracking simulations.
Key insights
- Local beliefs can effectively approximate global posterior distributions through information diffusion, even in partially observable multi-agent systems with action-dependent state transitions.
- Fully decentralized value function learning using diffusion strategies can achieve parameter consensus and performance comparable to a centralized training baseline.
- Combining geometric averaging for probabilistic beliefs and arithmetic averaging for point estimates (value function parameters) is an effective strategy for decentralized learning over networks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.