Reproduce “Model-based Constrained MDP for Budget Allocation in Sequential Incentive Marketing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shuai Xiao, Le Guo, Zaifan Jiang, Lei Lv, Yuanbo Chen, Jun Zhu, Shuang Yang
Ant Financial Services Group developed an off-policy, model-based constrained Markov Decision Process (CMDP) framework for sequential incentive allocation under budget limits, showing up to a 16.11% reward improvement over random strategies on synthetic data and a 9.40% cost reduction for equivalent rewards on real Alipay data.
Key insights
- The expected cumulative cost of an optimal policy is monotonically non-increasing with respect to the Lagrangian multiplier, enabling efficient bisection search for the optimal dual variable.
- A practical and robust off-policy learning framework for CMDPs is presented, allowing learning from historically logged data without risky online exploration, crucial for industrial applications involving monetary incentives.
- Integrating model-based planning with bisection search significantly accelerates the optimization process by avoiding repetitive policy retraining, making the approach computationally efficient for large-scale systems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.