Reproduce “Prediction with a Short Memory” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Vatsal Sharan, Sham Kakade, Percy Liang, Gregory Valiant
The authors show that limited past–future mutual information lets an ideal order-ℓ Markov predictor match full-history prediction in KL error, at most I/ℓ. For HMMs, an empirical window-counting predictor achieves average ℓ1 error at most √ε with an O(log n/ε) window under a sequence-length condition. This is not a KL guarantee or evidence that short context suffices for all tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.