Reproduce “Prediction with a Short Memory” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Vatsal Sharan, Sham Kakade, Percy Liang, Gregory Valiant

The authors show that limited past–future mutual information lets an ideal order-ℓ Markov predictor match full-history prediction in KL error, at most I/ℓ. For HMMs, an empirical window-counting predictor achieves average ℓ1 error at most √ε with an O(log n/ε) window under a sequence-length condition. This is not a KL guarantee or evidence that short context suffices for all tasks.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.