Reproduce “Prompt-Based Length Controlled Generation with Multiple Control Types” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Renlong Jie, Xiaojun Meng, Lifeng Shang, Xin Jiang, Qun Liu
Researchers from Northwestern Polytechnical University and Huawei Noah’s Ark Lab developed a prompt-based framework enabling highly accurate and flexible length-controlled text generation in GPT-style large language models. The system effectively interprets diverse user instructions and maintains text quality, achieving significantly reduced length control errors across various control types compared to baseline methods.
Key insights
- A simple, rule-based reward model can effectively guide length control in LLMs for verifiable attributes, avoiding the complexity and cost of neural reward models.
- The integration of an SFT loss term during RL fine-tuning is crucial for preserving the quality of the primary generation task while improving length control accuracy.
- Combining RL fine-tuning with inference-time sample filtering offers a synergistic effect, leading to superior length control accuracy across multiple control types.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.