Reproduce “Proximal Curriculum with Task Correlations for Deep Reinforcement Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Georgios Tzannetos, Parameswaran Kamalaruban, Adish Singla
PROCURL-TARGET introduces a curriculum learning strategy for Deep Reinforcement Learning that combines the Zone of Proximal Development (ZPD) principle with task correlations to efficiently guide agents towards specific target task distributions. It achieves superior performance and faster convergence compared to state-of-the-art baselines across various complex, multi-task environments.
Key insights
- The expected improvement in a REINFORCE learner's performance can be approximated by the product of the agent's learning potential on a task and its correlation with the target task's policy gradient.
- The agent's 'learning potential' on a task naturally embodies the ZPD principle, being maximized for tasks that are neither too easy nor too hard.
- By balancing task difficulty (ZPD) and explicit progression towards the target distribution (via task correlations), the curriculum effectively prioritizes tasks most beneficial for achieving specific learning goals.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.