Reproduce “Gym-TORAX: Open-source software for integrating reinforcement learning with plasma control simulators in tokamak research” on OpenResearch

About this paper

Antoine Mouchamps, Arthur Malherbe, Adrien Bolland, Damien Ernst

Gym-TORAX, an open-source Python package, integrates the TORAX plasma simulator with the Gymnasium API to facilitate the creation of reinforcement learning environments for tokamak control. This framework simplifies the application of RL to plasma dynamics, showing that an optimized PI controller can achieve an expected return of 3.79 compared to 3.40 for open-loop control in an ITER hybrid ramp-up scenario.

Key insights

• Integrating a lightweight and differentiable plasma simulator like TORAX with a standard RL framework (Gymnasium) significantly lowers the entry barrier for RL researchers into fusion control problems.
• The two-level discretization approach, where each RL interaction step encapsulates multiple internal TORAX simulation steps, effectively balances simulation fidelity with RL environment dynamics.
• Built-in error handling and action constraint mechanisms enhance the robustness and realism of the simulated plasma control environments, ensuring practical applicability for RL training.

Summary by alphaXiv.