Reproduce “RuleR: Improving LLM Controllability by Rule-based Data Recycling” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ming Li, Han Chen, Chenguang Wang, Dang Nguyen, Dianqi Li, Tianyi Zhou
RULER, a data augmentation method, improves large language model controllability by automatically recycling existing supervised fine-tuning data with rule-based constraint examples, eliminating the need for human or additional LLM intervention. The approach consistently enhances constraint-following abilities on benchmarks like IFEval by approximately 10% in some configurations, while also preserving general instruction-following capabilities.
Key insights
- Existing SFT data can be effectively recycled and augmented with rule-based constraints to improve LLM controllability without human intervention or reliance on additional LLMs.
- Diversity in instruction templates for each rule enhances both specific constraint controllability and general instruction-following, suggesting that varied phrasing helps models generalize better.
- A moderate number of simultaneously applied rules can be beneficial for challenging LLMs to develop more robust constraint understanding, but an excessive number of constraints can impair overall performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.