Reproduce “Distance-Based Regularisation of Deep Networks for Fine-Tuning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Henry Gouk, Timothy M. Hospedales, Massimiliano Pontil

MARS-PGM fine-tunes pretrained networks by constraining each layer’s weight distance from initialization after every update. Using maximum absolute row sum distance, it projects updates within layer-specific limits; unlike penalties, this guarantees a hard boundary. Across seven datasets, EfficientNetB0’s average rank was 1.00 versus 5.00 for MARS-SP, though results vary by model and task.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.