Reproduce “RL-IF: Tracking Non-local Training Data Influence for Online Reinforcement Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shixuan Liu, Cheng Tang, Yuzheng Hu, Fan Wu, Han Zhao, Jiaqi W. Ma
RINSE estimates each sample’s downstream influence across policy updates using RL-IF, a first-order approximation to a leave-one-out counterfactual, then filters experience for PPO training. It improves sample efficiency over standard PPO and local attribution on six control and navigation tasks. On Ant-v4, it uses 74.9 ± 3.7% of PPO’s data to reach performance targets, though added computation limits claims about elapsed time.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.