Reproduce “Test-Time Alignment for Large Language Models via Textual Model Predictive Control” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kuang-Da Wang, Teng-Ruei Chen, Yu Heng Hung, Guo-Xun Ko, Shuoyang Ding, Yueh-Hua Wu, Yu-Chiang Frank Wang, Chao-Han Huck Yang and 2 more
Researchers from National Yang Ming Chiao Tung University and NVIDIA developed Textual Model Predictive Control (TMPC), a test-time alignment framework for Large Language Models that applies principles from Model Predictive Control to text generation. TMPC achieved competitive or superior performance against strong baselines, including GPT-4o and DPO-fine-tuned LLaMA models, across machine translation, long-form response generation, and program synthesis, demonstrating improved efficiency and robustness.
Key insights
- Framing test-time alignment as a sequential decision-making problem allows for leveraging receding-horizon control strategies to manage the long-term planning inherent in text generation.
- Dynamically identifying and utilizing empirically validated intermediate textual units (subgoals) effectively mitigates the 'curse of horizon' and 'curse of dimensionality' challenges in test-time alignment.
- Integrating the base LLM as both the dynamics model and proposal distribution within the TMPC framework eliminates the need for additional model training or fine-tuning, improving resource efficiency.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.