Reproduce “FisherSFT: Data-Efficient Supervised Fine-Tuning of Language Models Using Information Gain” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Rohan Deb, Kiran Thekumparampil, Kousha Kalantari, Gaurush Hiranandani, Shoham Sabach, Branislav Kveton
Researchers developed FisherSFT, a method for data-efficient supervised fine-tuning of large language models by intelligently selecting training examples based on their information gain. This approach reduces data requirements by 25-50% in synthetic tasks and generates higher quality text in real-world LLM fine-tuning, consistently preferred by a judge LLM.
Key insights
- Optimal experimental design principles, specifically D-optimality based on the Fisher Information Matrix, can be adapted to efficiently select training data for LLM fine-tuning.
- The computationally prohibitive full Hessian of an LLM's log-likelihood can be effectively approximated by a parameter-independent lower bound involving only token embeddings.
- The `log det` objective function exhibits submodularity and monotonicity, enabling efficient greedy algorithms with theoretical approximation guarantees for data subset selection.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.