Reproduce “Online Learning Without Prior Information” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ashok Cutkosky, Kwabena Boahen

Stanford University researchers defined a new frontier of lower bounds for online learning without prior information and developed a family of Follow-the-Regularized-Leader (FTRL) algorithms that achieve these bounds. The work reveals a fundamental trade-off between regret terms related to optimal parameter value and gradient growth, and an inherent exponential penalty, offering "nearly hyperparameter-free" solutions like FREEREX.

Key insights

  • A fundamental trade-off exists in online learning without prior information, where reducing the logarithmic factor in the {sqrt}T regret term increases an inherent exponential penalty linked to gradient growth.
  • The development of ({sigma}, ||{cdot}||)-adaptive regularizers, generalizing strong convexity, is key to constructing algorithms that achieve optimal performance in information-agnostic online convex optimization.
  • Algorithms designed under assumptions of known maximum gradient (L_max) are fragile; if this bound is violated, even moderately adversarial gradient sequences can lead to exponential regret.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.