Reproduce “An Information-Theoretic Definition for Open-Ended Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Wanqiao Xu, Yifan Zhu, Benjamin Van Roy
The research introduces a formal information-theoretic definition of open-ended learning, proving that classical bandit environments are not open-ended under this framework and proposing an 'insatiable linear bandit' where a novel Truncated Thompson Sampling algorithm achieves sustained capability expansion.
Key insights
- Traditional bandit environments, even with infinite action spaces or unbounded rewards, do not permit open-ended learning due to sublinear growth of useful information.
- Linear information gain about an environment parameter is insufficient for open-endedness; the information must specifically enable sustained improvement in expected reward.
- Agents capable of open-ended learning must dynamically expand their "learning targets" or understanding of the environment's increasing complexities, rather than operating with fixed scope.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.