Reproduce “Implementing TD3 to train a Neural Network to fly a Quadcopter through an FPV Gate” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Patrick Thomas, Kevin Schroeder, Jonathan Black

Virginia Tech researchers applied the TD3 algorithm to develop a neural network velocity controller, enabling a quadcopter to autonomously navigate through a First Person View (FPV) racing gate in simulation. The trained policy successfully transferred to a physical drone, demonstrating controlled flight through a virtual gate in a mixed-reality environment after tuning the simulated vehicle dynamics.

Key insights

  • The TD3 algorithm demonstrated significant sample inefficiency, requiring over 1.5 million simulation steps to learn a robust policy, highlighting the high computational cost for complex behaviors.
  • Simulation fidelity is critical for successful sim-to-real transfer; initial discrepancies in simulated vehicle dynamics led to poor real-world performance, necessitating a refinement of the simulation model.
  • The algorithm was sensitive to hyperparameter choices, and careful tuning, along with specific network initialization techniques (e.g., actor output nodes), was essential for stable and effective learning.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.