Reproduce “Memory-Based Meta-Learning on Non-Stationary Distributions” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Tim Genewein, Grégoire Delétang, Anian Ruoss, Li Kevin Wenliang, Elliot Catt, Vincent Dutordoir, Jordi Grau-Moya, Laurent Orseau and 2 more

The authors test whether memory-based neural models can predict piecewise-stationary Bernoulli sequences with hidden change points. They train recurrent networks and Transformers by next-step log loss, allowing activations to adapt across sequences. On 10,000 length-256 sequences from the PTW prior, a PTW-trained LSTM has mean cumulative regret of 12.07 bits, close to PTW’s 11.91 bits, in this setting.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.