Reproduce “On Global Convergence Rates for Federated Softmax Policy Gradient under Heterogeneous Environments” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Safwan Labbi, Paul Mangold, Daniil Tiapkin, Eric Moulines
The authors analyze federated softmax policy-gradient methods, where agents train locally and a server averages updates without sharing raw trajectories. They prove global optimality-gap guarantees for unregularized S-FedPG and entropy-regularized RS-FedPG, with communication-round rates of O(1/ε) and O(log(1/ε)), respectively. Guarantees hold above a heterogeneity-dependent accuracy floor; one constructed instance establishes a strict hierarchy among deterministic, stochastic, and history-dependent policies.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.