Reproduce “BrachistoneLR: A Brachistochrone-Inspired Learning-Rate Schedule and a Controlled Benchmark of Scheduling Policies” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Md. Sadekur Rahman Roni, Md. Jalal uddin Chowdhury, Moutusi Dash Nimi
BrachistoneLR derives a smooth learning-rate decay from the brachistochrone analogy, but analytical expansion shows that it is cosine annealing with the period adjusted from E to E−1 epochs. In a controlled 72-configuration benchmark, it performs nearly identically to cosine and warmup-cosine schedules, while smooth decay methods generally outperform constant, step, and exponential schedules, especially on CIFAR-10.
Key insights
- The proposed schedule is not a separate optimization family: its only difference from standard cosine annealing is using E−1 rather than E as the period, producing a lower terminal learning rate.
- Smooth high-to-low schedules are more consistently effective than constant or calendar-based decay in the tested setting, but cosine annealing, warmup-cosine, and BrachistoneLR are effectively tied overall.
- The endpoint distinction matters mainly for short training horizons; the terminal-rate difference decreases approximately as E−2 and becomes negligible for long runs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.