Reproduce “Continual Deep Reinforcement Learning for Decentralized Satellite Routing” on OpenResearch

About this paper

Federico Lozano-Cuadra, Beatriz Soret, Israel Leyva-Mayorga, Petar Popovski

Researchers at the University of Malaga and Aalborg University developed a decentralized multi-agent deep reinforcement learning framework for routing in Low Earth Orbit satellite constellations. This framework enables satellites to make autonomous decisions using local information and continually adapt over time through model anticipation and federated learning, achieving end-to-end latency comparable to centralized shortest path algorithms while better handling congestion.

Key insights

• Decentralized MA-DRL, operating with only local 1-hop neighbor information, can achieve end-to-end latency highly comparable to centralized shortest path algorithms that assume full network knowledge.
• The integration of a continual learning framework with 'model anticipation' for short-term alignment and 'Federated Learning' for long-term alignment effectively counteracts model divergence in dynamic LEO satellite constellations.
• The DRL-based routing dynamically adapts to congestion by considering queue occupancy and exploiting alternative paths, leading to superior traffic distribution compared to static shortest path methods.

Summary by alphaXiv.