Reproduce “Improving Autonomous Separation Assurance through Distributed Reinforcement Learning with Attention Networks” on OpenResearch

About this paper

Marc W. Brittain, Luis E. Alvarez, Kara Breeden

Researchers at MIT Lincoln Laboratory developed SACD-A, a distributed reinforcement learning framework with attention networks for autonomous separation assurance in Advanced Air Mobility (AAM), consistently reducing airspace risk and achieving a 10x training throughput increase over prior methods. The framework demonstrates robust performance under surveillance and communication uncertainties while minimizing unnecessary maneuvers.

Key insights

• The combination of an off-policy maximum entropy DRL algorithm (SACD) with attention networks enables highly scalable and robust autonomous separation assurance capable of handling variable numbers of interacting agents under uncertainty.
• Distributed asynchronous training drastically improves the sample efficiency and throughput of DRL algorithm development, making complex, safety-critical policies more feasible to train and refine.
• Incorporating both speed and vertical maneuvers within the action space allows for a more comprehensive and operationally suitable conflict resolution strategy while minimizing unnecessary actions through a carefully designed reward function.

Summary by alphaXiv.