Reproduce “Can Jev be Your Q or Policy in Reinforcement Learning?” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yi Ma, Tianpei Yang, Yaodong Yang, Weixun Wang, Hongyao Tang
The paper tests frozen Jev as guidance within reinforcement-learning training: as an action reference, exploration judge, and replay rater. Across nine MiniGrid tasks and three Atari games, learners sometimes outperform direct Jev control. Results depend on task and supplied information; in FourRooms, direct Jev succeeds in 96% of episodes with engineered navigation features, versus 40% without them.
Key insights
- A decision model need not solve an environment reliably on its own to offer useful guidance to a learning agent; students sometimes benefit from Jev’s recommendations even when direct control is imperfect.
- The value of guidance depends on how Jev is queried and what context it receives. Removing route and progress information reduced FourRooms success from 96% to 40% and MultiRoom-N4 from 100% to 38%.
- Jev’s judgments can rank states or experiences, but converting those rankings into bonuses or replay priorities changes the learning objective or sample weighting and does not guarantee better performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.