Reproduce “Deep Reinforcement Learning for Sponsored Search Real-time Bidding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jun Zhao, Guang Qiu, Ziyu Guan, Wei Zhao, Xiaofei He
Alibaba Group researchers introduced a Deep Reinforcement Learning (DRL) framework for real-time bidding in sponsored search, effectively maximizing advertiser profit by addressing environmental non-stationarity through an hourly-aggregated Robust MDP and managing massive multi-agent competition. Online A/B tests on Alibaba's platform demonstrated a 13.01% improvement in Purchase Amount/Cost for the multi-agent DRL approach over traditional keyword-level bidding.
Key insights
- While impression-level data is highly stochastic, statistical features aggregated at an hourly level exhibit consistent, periodic patterns, forming a robust and stable Markov Decision Process for DRL.
- Instead of directly outputting bid prices, allowing the DRL agent to control parameters of a subordinate bidding model (control-by-model) provides a flexible and robust solution for real-time decision-making in dynamic environments.
- Combining individual competitive objectives with a collective cooperative objective enables effective learning for thousands of independent agents in a massive multi-agent environment, enhancing overall system efficiency and individual ad performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.