Reproduce “Digital Twin Synchronization: Bridging the Sim-RL Agent to a Real-Time Robotic Additive Manufacturing Control” on OpenResearch

About this paper

Matsive Ali, Sandesh Giri, Sen Liu, Qin Yang

Researchers at the University of Louisiana at Lafayette and Bradley University developed a framework that integrates the Soft Actor-Critic (SAC) reinforcement learning algorithm with real-time digital twin synchronization for robotic control in additive manufacturing. This approach enables efficient policy learning, reduces training time via transfer learning, and achieves reliable sim-to-real transfer with approximately 20 milliseconds latency on a Viper X300s robot arm.

Key insights

• The hierarchical reward structure effectively guided the RL agent, allowing it to recover from local minima and achieve stable convergence in complex tasks.
• Transfer learning significantly accelerated policy convergence and improved performance in dynamic goal-following tasks, demonstrating its utility in reducing training time and computational resources.
• Real-time digital twin synchronization with a low latency of approximately 20 milliseconds allows for robust and reliable transfer of simulation-trained policies to physical robots, mitigating risks associated with direct physical training.

Summary by alphaXiv.