Reproduce “Fast Slate Policy Optimization: Going Beyond Plackett-Luce” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Otmane Sakhi, David Rohde, Nicolas Chopin
Researchers from CREST-ENSAE and Criteo AI Lab introduced the Latent Gaussian Perturbation (LGP) policy, a new class for optimizing large-scale slate decision systems, demonstrating improved computational efficiency and higher expected rewards compared to Plackett-Luce policies. The LGP policy, particularly when accelerated by Approximate MIPS, achieves logarithmic complexity for sampling and gradient estimation, enabling faster training and better performance on datasets with millions of items.
Key insights
- Perturbing the latent query embedding, instead of individual item scores, leads to a policy with significantly lower gradient variance and faster computational characteristics (O(log P)) compared to Plackett-Luce policies.
- The gradient variance of LGP is largely independent of the slate size K, making it more robust for applications requiring longer recommendation lists.
- Fixing action embeddings (pre-computed using SVD) before training the user embedding function (h_theta(X)) significantly enhances optimization stability and efficiency, reducing gradient variance.
- LGP is agnostic to the reward function structure, allowing optimization for complex, arbitrary reward functions, unlike methods like PL-Rank that exploit specific reward properties.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.