Reproduce “Virtual-Taobao: Virtualizing Real-world Online Retail Environment for Reinforcement Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jing-Cheng Shi, Yang Yu, Qing Da, Shi-Yong Chen, An-Xiang Zeng
Researchers developed Virtual Taobao, a simulation framework for online retail, to enable reinforcement learning (RL) policy training without incurring high real-world costs. This framework successfully generated RL policies that achieved up to 8% improvements in Total Turnover and Total Volume during real-world A/B testing on the Taobao platform.
Key insights
- Generating diverse and realistic initial customer requests is crucial for high-fidelity simulation and can be achieved by extending GANs with entropy and KL-divergence constraints (GAN-SD).
- Dynamic, interactive systems with co-adapting human user and platform policies can be effectively simulated using a multi-agent adversarial imitation learning approach (MAIL).
- Over-fitting to simulated environments can be reduced, and real-world generalization improved, by constraining learned policies to actions with plausible norms relative to historical data (ANC).
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.