Reproduce “Predicting Fine-Tuning Performance with Probing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zining Zhu, Soroosh Shahtalebi, Frank Rudzicz
The authors test whether probing measurements can predict language models’ fine-tuning performance across six GLUE classification tasks. They use probe accuracies from model layers as features for regression, comparing predictions with a random-feature control using five-fold cross-validation. Three selected features reduce QNLI prediction RMSE by 80.52%, but results vary across tasks and are evaluated within five model families.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.