Reproduce “RLFTSim: Realistic and Controllable Multi-Agent Traffic Simulation via Reinforcement Learning Fine-Tuning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ehsan Ahmadi, Hunter Schofield, Behzad Khamidehi, Fazel Arasteh, Jinjun Shan, Lili Mou, Dongfeng Bai, Kasra Rezaee

RLFTSim introduces a reinforcement learning framework that fine-tunes pre-trained multi-agent traffic simulators to enhance realism and instill controllable behaviors. The method achieves state-of-the-art realism with a WOSAC RMM of 0.7867 and effectively distills diverse goal-conditioned maneuvers without relying on human feedback.

Key insights

  • The MLOO reward mechanism provides an unbiased gradient estimator and achieves quadratic variance reduction (O(1/N^2)), significantly improving the stability and sample efficiency of RL training for realism alignment.
  • Distilling controllability into learned traffic simulators is achievable by combining realism-aligned RL fine-tuning with goal-conditioned observations and a mixed reward signal, allowing agents to perform diverse, targeted maneuvers.
  • The framework is model-agnostic, demonstrating its ability to enhance realism and controllability across different base simulator architectures, including discrete-token and continuous-action models.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.