Reproduce “Learning Long-Range Dependencies with Temporal Predictive Coding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tom Potter, Oliver Rhodes
The authors introduce tPC-RTRL, adding a trace of recurrent weights’ influence on earlier states while retaining local predictive-coding errors, to supply temporal credit absent from standard tPC. On a delayed-copy task, it matches BPTT at 99.99% validation accuracy, versus about 40% for tPC and spatial BP. On WikiText-103, it reaches 1.865 BPC versus BPTT’s 1.864, a near-match on this dataset.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.