Reproduce “Interestingness as an Inductive Heuristic for Future Compression Progress” on OpenResearch

About this paper

Vincent Herrmann, Jürgen Schmidhuber

Herrmann and Schmidhuber's research formalizes "interestingness" as an inductive heuristic, demonstrating that future compression progress can be predicted from past learning trajectories. They establish that the probability of future discoveries decays exponentially with increasing stagnation, with the Algorithmic Prior offering a more optimistic outlook for deeper insights compared to the Length and Speed Priors.

Key insights

• Interestingness has an inductive property: future compression breakthroughs can be statistically predicted from the history of past progress and the recency of the last observed compression drop (stagnation length).
• The probability of future compression progress (a 'drop' in the complexity profile) diminishes exponentially with increasing stagnation length, indicating that systems should prioritize areas where recent learning has occurred.
• The choice of universal prior significantly impacts the expected magnitude of future progress; the Algorithmic Prior is "more optimistic" about deeper, more substantial compression gains than the Length or Speed Priors.

Summary by alphaXiv.