Reproduce “Length Value Model: Scalable Value Pretraining for Token-Level Length Modeling” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhen Zhang, Changyi Yang, Zijie Xia, Zhen Yang, Chengzhi Liu, Zhaotiao Weng, Yepeng Liu, Haobo Chen and 6 more
Researchers from UC Santa Barbara, Carnegie Mellon, LMSYS Org, and Apple introduced the Length Value Model (LenVM), a token-level value pretraining framework for estimating the remaining generation length in autoregressive models. LenVM improved adherence to length constraints, boosting the Length Score for "Equal To" constraints to 64.8 on LIFEBench-token, and achieved a more favorable performance-efficiency trade-off on reasoning benchmarks while preserving quality.
Key insights
- Modeling remaining generation length as a bounded, Bellman-consistent discounted return provides a robust and scalable token-level supervision signal without requiring human annotation.
- Attaching a value head to the base model enables dynamic, fine-grained control over output length at each decoding step, moving beyond static, sequence-level constraints.
- The framework provides interpretability by identifying "length tokens" and analyzing temporal-difference residuals, offering insights into the model's internal decisions regarding generation continuation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.