Reproduce “Differentiable Agent-Based Simulation for Gradient-Guided Simulation-Based Optimization” on OpenResearch

About this paper

Philipp Andelfinger

Philipp Andelfinger from the University of Rostock developed a method for transforming non-differentiable agent-based models (ABMs) into differentiable forms by approximating discrete elements with smooth functions, enabling gradient-based optimization. This approach allowed gradient-based methods to achieve faster convergence and better solution quality in high-dimensional traffic optimization problems, demonstrating a 6.3% improvement in vehicle progress and successfully training embedded neural networks for dynamic control.

Key insights

• By systematically replacing discrete ABM elements with smooth approximations, traditionally non-differentiable simulations can become amenable to efficient gradient-based optimization.
• Restricting differentiability to key aspects of an ABM (e.g., traffic light control) significantly reduces computational overhead while still providing useful gradients for optimization.
• The differentiable ABM framework enables direct, gradient-based training of neural networks embedded within simulation logic, offering a novel method for optimizing complex agent behaviors without reinforcement learning.

Summary by alphaXiv.