Reproduce “A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combat” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiajun Chai, Wenzhang Chen, Yuanheng Zhu, Zong-xin Yao, Dongbin Zhao
A hierarchical deep reinforcement learning framework is presented for 6-DOF UCAV air-to-air combat, employing a PPO-trained inner-loop flight controller for precise attitude tracking and an outer-loop combat strategy developed via fictitious self-play. This model-free approach yields flight control superior to PID and robust, high-winning-rate combat tactics against evolving adversaries.
Key insights
- Decomposing complex 6-DOF air combat into hierarchical RL loops effectively manages computational complexity, allowing for specialized learning at different abstraction levels (low-level control vs. high-level strategy).
- A well-designed composite reward function is crucial for training a stable and performant RL-based flight controller capable of smooth, precise attitude tracking, outperforming traditional PID methods.
- Fictitious self-play enables the development of robust and increasingly powerful combat strategies by training against an evolving distribution of opponents, which is essential for competitive, adversarial environments.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.