Reproduce “Novelty Search for Deep Reinforcement Learning Policy Network Weights by Action Sequence Edit Metric Distance” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ethan C. Jackson, Mark Daley
Researchers from The University of Western Ontario and Vector Institute introduced a generalizable novelty search mechanism for deep reinforcement learning that utilizes agent action sequences and Levenshtein distance. This method, when integrated as a population resampling strategy, improved the performance of a genetic algorithm in two out of four Atari 2600 games and fostered behaviorally diverse policies.
Key insights
- Action sequences, combined with Levenshtein distance, offer a computationally efficient and generalizable metric for behavioral novelty, applicable across diverse RL environments without domain-specific knowledge.
- While pure novelty-driven selection can produce behaviorally distinct policies (e.g., longer agent lifespans), it does not consistently yield higher game scores compared to reward-based optimization.
- Integrating novelty search for stagnation detection and population resampling (Method II) effectively reorients the learning process, allowing reward-optimizing genetic algorithms to escape local optima and achieve improved policy performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.